Compare commits

..

6 Commits

Author SHA1 Message Date
Minghe
2298f39cca k3s on docker (#401)
* since we do the image build in initialize container of pod, so we have to make sure the image built can be access from kubelet on same node, containerd could not support that
* update docs
* bump version
* clean up
2019-12-07 01:42:13 +08:00
Minghe
23d68bc27b Refactor packer (#399) 2019-12-06 20:16:05 +08:00
Minghe
74c0423f0d add docs for rust example (#398) 2019-12-06 14:58:30 +08:00
Minghe
06f87c4d8e fix deploy to aks issue, and update docs (#396)
* fix deploy to aks issue, and update docs
* update docs
2019-12-06 11:54:27 +08:00
Minghe
35262de828 release v0.8.7 (#394) 2019-12-06 10:04:40 +08:00
dependabot-preview[bot]
34a495984c Bump github.com/olekukonko/tablewriter from 0.0.3 to 0.0.4 (#393)
Bumps [github.com/olekukonko/tablewriter](https://github.com/olekukonko/tablewriter) from 0.0.3 to 0.0.4.
- [Release notes](https://github.com/olekukonko/tablewriter/releases)
- [Commits](https://github.com/olekukonko/tablewriter/compare/v0.0.3...v0.0.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-06 09:45:27 +08:00
25 changed files with 598 additions and 502 deletions

View File

@@ -32,7 +32,7 @@ cli-test-ci:
./scripts/test_cli.sh 'js' ./scripts/test_cli.sh 'js'
cli-test: cli-test:
./scripts/test_cli.sh 'js rb py go php java d' ./scripts/test_cli.sh 'js rb py go php java d rs'
http-test: http-test:
./scripts/http_test.sh ./scripts/http_test.sh

View File

@@ -1,5 +1,6 @@
fx fx
------ ------
Poor man's function as a service. Poor man's function as a service.
<br/> <br/>
![CI](https://github.com/metrue/fx/workflows/ci/badge.svg) ![CI](https://github.com/metrue/fx/workflows/ci/badge.svg)
@@ -13,13 +14,12 @@ Poor man's function as a service.
- [Introduction](#introduction) - [Introduction](#introduction)
- [Installation](#installation) - [Installation](#installation)
- [Usage](#usage) - [Usage](#usage)
- [Manage Infrastructure](#manage-infrastructure)
- [Contribute](#contribute) - [Contribute](#contribute)
## Introduction ## Introduction
![workflow](https://raw.githubusercontent.com/metrue/fx/master/docs/fx-workflow.png)
fx is a tool to help you do Function as a Service on your own server, fx can make your stateless function a service in seconds, both Docker host and Kubernetes cluster supported. The most exciting thing is that you can write your functions with most programming languages. fx is a tool to help you do Function as a Service on your own server, fx can make your stateless function a service in seconds, both Docker host and Kubernetes cluster supported. The most exciting thing is that you can write your functions with most programming languages.
Feel free hacking fx to support the languages not listed. Welcome to tweet me [@_metrue](https://twitter.com/_metrue) on Twitter, [@metrue](https://www.weibo.com/u/2165714507) on Weibo. Feel free hacking fx to support the languages not listed. Welcome to tweet me [@_metrue](https://twitter.com/_metrue) on Twitter, [@metrue](https://www.weibo.com/u/2165714507) on Weibo.
@@ -79,7 +79,7 @@ USAGE:
fx [global options] command [command options] [arguments...] fx [global options] command [command options] [arguments...]
VERSION: VERSION:
0.8.4 0.8.7
COMMANDS: COMMANDS:
infra manage infrastructure infra manage infrastructure
@@ -96,44 +96,36 @@ GLOBAL OPTIONS:
--version, -v print the version --version, -v print the version
``` ```
1. Write a function ### Deploy your function to Docker
You can check out [examples](https://github.com/metrue/fx/tree/master/examples/functions) for reference. Let's write a function as an example, it calculates the sum of two numbers then returns:
```js
module.exports = (ctx) => {
ctx.body = 'hello world'
}
``` ```
Then save it to a file `func.js`. $ fx up --name hello-fx ./examples/functions/JavaScript/func.js
2. Deploy your function as a service +------------------------------------------------------------------+-----------+---------------+
| ID | NAME | ENDPOINT |
Give your service a port with `--port`, and name with `--name`, heath checking with `--healthcheck` if you want. +------------------------------------------------------------------+-----------+---------------+
| 5b24d36608ee392c937a61a530805f74551ddec304aea3aca2ffa0fabcf98cf3 | /hello-fx | 0.0.0.0:58328 |
```shell +------------------------------------------------------------------+-----------+---------------+
$ fx up -name fx_service_name -p 10001 --healthcheck func.js
2019/08/10 13:26:37 info Pack Service: ✓
2019/08/10 13:26:39 info Build Service: ✓
2019/08/10 13:26:39 info Run Service: ✓
2019/08/10 13:26:39 info Service (fx_service_name) is running on: 0.0.0.0:10001
2019/08/10 13:26:39 info up function fx_service_name(func.js) to machine localhost: ✓
``` ```
if you want see what the source code of your service looks like, you can export it into a dirctory, ### Deploy your function to Kubernetes
```shell ```
$ fx image export -o <path of dir> func.js $ KUBECONFIG=~/.kube/config ./build/fx up examples/functions/JavaScript/func.js --name hello-fx
2019/09/25 19:31:19 info exported to <path of dir>: ✓
+-------------------------------+------+----------------+
| ID | NAME | ENDPOINT |
+----+--------------------------+-----------------------+
| 5b24d36608ee392c937a | hello-fx | 10.0.242.75:80 |
+------------------------+-------------+----------------+
``` ```
3. Test your service ### Test your service
then you can test your service: then you can test your service:
```shell ```shell
$ curl -v 0.0.0.0:10001 $ curl -v 0.0.0.0:58328
GET / HTTP/1.1 GET / HTTP/1.1
@@ -155,39 +147,32 @@ hello world
``` ```
## Docker ## Manage Infrastructure
**fx** is originally designed to turn a function into a runnable Docker container in a easiest way, on a host with Docker running, you can just deploy your function with `fx up` command, **fx** is originally designed to turn a function into a runnable Docker container in a easiest way, on a host with Docker running, you can just deploy your function with `fx up` command, and now **fx** supports deploy function to be a service onto Kubernetes cluster infrasture, and we encourage you to do that other than on bare Docker environment, there are lots of advantage to run your function on Kubernetes like self-healing, load balancing, easy horizontal scaling, etc. It's pretty simple to deploy your function onto Kubernetes with **fx**, you just set KUBECONFIG in your enviroment.
By default. **fx** use localhost as target infrastructure to run your service, and you can also setup your remote virtual machines as **fx**'s infrastructure and deploy your functions onto it.
### `fx infra create`
You can create types (docker and k8s) of infrastructures for **fx** to deploy functions
```shell ```shell
fx up --name hello-svc --port 7777 hello.js # onto localhost $ fx infra create --name infra_us --type docker --host <user>@<ip> ## create docker type infrasture on <ip>
DOCKER_REMOTE_HOST_ADDR=xx.xx.xx.xx DOCKER_REMOTE_HOST_USER=xxxx DOCKER_REMOTE_HOST_PASSWORD=xxxx fx up --name hello-svc --port 7777 hello.js # onto remote host $ fx infra create --name infra_bj --type k8s --master <user>@<ip> --agents '<user1>@<ip1>,<user2>@<ip2>' ## create k8s type infrasture use <ip> as master node, and <ip1> and <ip2> as agents nodes
``` ```
## Kubernetes ### `fx infra use`
**fx** supports deploy function to be a service onto Kubernetes cluster infrasture, and we encourage you to do that other than on bare Docker environment, there are lots of advantage to run your function on Kubernetes like self-healing, load balancing, easy horizontal scaling, etc. It's pretty simple to deploy your function onto Kubernetes with **fx**, you just set KUBECONFIG in your enviroment. To use a infrastructure, you can use `fx infra use` command to activate it.
```shell ```shell
KUBECONFIG=<Your KUBECONFIG> fx deploy -n fx-service-abc_js -p 12349 examples/functions/JavaScript/func.js # function will be deploy to your Kubernetes cluster and expose a IP address of your loadbalencer fx infra use <infrastructure name>
``` ```
or and you can list your infrastructure with `fx infra list`
```shell ## Use Public Cloud Kubernetes Service as infrastructure to run your functions
$ export KUBECONFIG=<Your KUBECONFIG>
$ fx deploy -n fx-service-abc_js -p 12349 examples/functions/JavaScript/func.js # function will be deploy to your Kubernetes cluster and expose a IP address of your loadbalencer
```
* Local Kubernetes Cluster
Docker for Mac and Docker for Windows already support Kubernetes with single node cluster, we can use it directly, and the default `KUBECONFIG` is `~/.kube/config`.
```shell
$ export KUBECONFIG=~/.kube/config # then fx will take the config to deloy function
```
if you have multiple Kubernetes clusters configured, you have to set context correctly. FYI [configure-access-multiple-clusters](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/)
* Azure Kubernetes Service (AKS) * Azure Kubernetes Service (AKS)
@@ -224,8 +209,6 @@ But we would suggest you run `kubectl config current-context` to check if the cu
* Setup your own Kubernetes cluster * Setup your own Kubernetes cluster
![init workflow](https://raw.githubusercontent.com/metrue/fx/master/docs/fx-init-cluster.png)
```shell ```shell
fx infra create --type k3s --name fx-cluster-1 --master root@123.11.2.3 --agents 'root@1.1.1.1,root@2.2.2.2' fx infra create --type k3s --name fx-cluster-1 --master root@123.11.2.3 --agents 'root@1.1.1.1,root@2.2.2.2'
``` ```

View File

@@ -106,7 +106,7 @@ func (c *Config) AddK8SCloud(name string, kubeconfig []byte) error {
cloud := map[string]string{ cloud := map[string]string{
"type": "k8s", "type": "k8s",
"kubeConfig": kubecfg, "kubeconfig": kubecfg,
} }
return c.addCloud(name, cloud) return c.addCloud(name, cloud)

View File

@@ -37,27 +37,29 @@ type API struct {
// Create a API // Create a API
func Create(host string, port string) (*API, error) { func Create(host string, port string) (*API, error) {
version, err := utils.DockerVersion(host, port) addr := host + ":" + port
v, err := version(addr)
if err != nil { if err != nil {
return nil, err return nil, err
} }
endpoint := fmt.Sprintf("http://%s:%s/v%s", host, port, version) endpoint := fmt.Sprintf("http://%s:%s/v%s", host, port, v)
return &API{ return &API{
endpoint: endpoint, endpoint: endpoint,
version: version, version: v,
}, nil }, nil
} }
// MustCreate a api object, panic if not // MustCreate a api object, panic if not
func MustCreate(host string, port string) *API { func MustCreate(host string, port string) *API {
version, err := utils.DockerVersion(host, port) addr := host + ":" + port
v, err := version(addr)
if err != nil { if err != nil {
panic(err) panic(err)
} }
endpoint := fmt.Sprintf("http://%s:%s/v%s", host, port, version) endpoint := fmt.Sprintf("http://%s:%s/v%s", host, port, v)
return &API{ return &API{
endpoint: endpoint, endpoint: endpoint,
version: version, version: v,
} }
} }
@@ -131,7 +133,11 @@ func (api *API) post(path string, body []byte, expectStatus int, v interface{})
// Version get version of docker engine // Version get version of docker engine
func (api *API) Version(ctx context.Context) (string, error) { func (api *API) Version(ctx context.Context) (string, error) {
path := api.endpoint + "/version" return version(api.endpoint)
}
func version(endpoint string) (string, error) {
path := endpoint + "/version"
if !strings.HasPrefix(path, "http") { if !strings.HasPrefix(path, "http") {
path = "http://" + path path = "http://" + path
} }

Binary file not shown.

Before

Width:  |  Height:  |  Size: 294 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

47
examples/functions/Rust/README.md vendored Normal file
View File

@@ -0,0 +1,47 @@
# Make a Rust function a service with fx
Write a function like,
```rust
pub mod fns {
#[derive(Serialize)]
pub struct Response {
pub result: i32,
}
#[derive(Deserialize)]
pub struct Request {
pub a: i32,
pub b: i32,
}
pub fn func(req: Request) -> Response {
Response {
result: req.a + req.b,
}
}
}
```
then deploy it with `fx up` command,
```shell
$ fx up -p 8080 func.rs
```
test it using `curl`
```shell
$ curl -X 'POST' --header 'Content-Type: application/json' --data '{"a":1,"b":1}' '0.0.0.0:3000'
HTTP/1.1 200 OK
Content-Length: 12
Content-Type: application/json
Date: Fri, 06 Dec 2019 06:45:14 GMT
Server: Rocket
{
"result": 2
}
```

4
fx.go
View File

@@ -16,7 +16,7 @@ import (
"github.com/urfave/cli" "github.com/urfave/cli"
) )
const version = "0.8.6" const version = "0.8.73"
func init() { func init() {
go checkForUpdate() go checkForUpdate()
@@ -212,6 +212,7 @@ func main() {
Action: handle( Action: handle(
middlewares.LoadConfig, middlewares.LoadConfig,
middlewares.Provision, middlewares.Provision,
middlewares.Parse("image_build"),
handlers.BuildImage, handlers.BuildImage,
), ),
}, },
@@ -227,6 +228,7 @@ func main() {
Action: handle( Action: handle(
middlewares.LoadConfig, middlewares.LoadConfig,
middlewares.Provision, middlewares.Provision,
middlewares.Parse("image_export"),
handlers.ExportImage, handlers.ExportImage,
), ),
}, },

2
go.mod
View File

@@ -27,7 +27,7 @@ require (
github.com/mitchellh/go-homedir v1.1.0 github.com/mitchellh/go-homedir v1.1.0
github.com/morikuni/aec v1.0.0 // indirect github.com/morikuni/aec v1.0.0 // indirect
github.com/nwaples/rardecode v1.0.0 // indirect github.com/nwaples/rardecode v1.0.0 // indirect
github.com/olekukonko/tablewriter v0.0.3 github.com/olekukonko/tablewriter v0.0.4
github.com/opencontainers/go-digest v1.0.0-rc1 // indirect github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
github.com/opencontainers/image-spec v1.0.1 // indirect github.com/opencontainers/image-spec v1.0.1 // indirect
github.com/otiai10/copy v1.0.2 github.com/otiai10/copy v1.0.2

4
go.sum
View File

@@ -163,6 +163,8 @@ github.com/mattn/go-isatty v0.0.8 h1:HLtExJ+uU2HOZ+wI0Tt5DtUDrx8yhUqDcp7fYERX4CE
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-runewidth v0.0.6 h1:V2iyH+aX9C5fsYCpK60U8BYIvmhqxuOL3JZcqc1NB7k= github.com/mattn/go-runewidth v0.0.6 h1:V2iyH+aX9C5fsYCpK60U8BYIvmhqxuOL3JZcqc1NB7k=
github.com/mattn/go-runewidth v0.0.6/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI= github.com/mattn/go-runewidth v0.0.6/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/mattn/go-runewidth v0.0.7 h1:Ei8KR0497xHyKJPAv59M1dkC+rOZCMBJ+t3fZ+twI54=
github.com/mattn/go-runewidth v0.0.7/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/metrue/go-ssh-client v0.0.0-20191125030649-4ac058ee958b h1:JGD0sJ44XzhsT1voOg00zji4ubuMNcVNK3m7d9GI88k= github.com/metrue/go-ssh-client v0.0.0-20191125030649-4ac058ee958b h1:JGD0sJ44XzhsT1voOg00zji4ubuMNcVNK3m7d9GI88k=
github.com/metrue/go-ssh-client v0.0.0-20191125030649-4ac058ee958b/go.mod h1:ERHOEBrDy6+8vfoJjjmhdmBpOzdvvP7bLtwYTTK6LOs= github.com/metrue/go-ssh-client v0.0.0-20191125030649-4ac058ee958b/go.mod h1:ERHOEBrDy6+8vfoJjjmhdmBpOzdvvP7bLtwYTTK6LOs=
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE= github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
@@ -187,6 +189,8 @@ github.com/nwaples/rardecode v1.0.0 h1:r7vGuS5akxOnR4JQSkko62RJ1ReCMXxQRPtxsiFMB
github.com/nwaples/rardecode v1.0.0/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0= github.com/nwaples/rardecode v1.0.0/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0=
github.com/olekukonko/tablewriter v0.0.3 h1:i0LBnzgiChAWHJYTQAZJDOgf8MNxAVYZJ2m63SIDimI= github.com/olekukonko/tablewriter v0.0.3 h1:i0LBnzgiChAWHJYTQAZJDOgf8MNxAVYZJ2m63SIDimI=
github.com/olekukonko/tablewriter v0.0.3/go.mod h1:YZeBtGzYYEsCHp2LST/u/0NDwGkRoBtmn1cIWCJiS6M= github.com/olekukonko/tablewriter v0.0.3/go.mod h1:YZeBtGzYYEsCHp2LST/u/0NDwGkRoBtmn1cIWCJiS6M=
github.com/olekukonko/tablewriter v0.0.4 h1:vHD/YYe1Wolo78koG299f7V/VAS08c6IpCLn+Ejf/w8=
github.com/olekukonko/tablewriter v0.0.4/go.mod h1:zq6QwlOf5SlnkVbMSr5EoBv3636FWnp+qbPhuoO21uA=
github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=

View File

@@ -2,80 +2,74 @@ package handlers
import ( import (
"fmt" "fmt"
"io/ioutil"
"os" "os"
"time" "time"
"github.com/apex/log" "github.com/apex/log"
"github.com/google/uuid"
"github.com/metrue/fx/constants" "github.com/metrue/fx/constants"
containerruntimes "github.com/metrue/fx/container_runtimes" containerruntimes "github.com/metrue/fx/container_runtimes"
"github.com/metrue/fx/context" "github.com/metrue/fx/context"
"github.com/metrue/fx/packer" "github.com/metrue/fx/packer"
"github.com/metrue/fx/types" "github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/utils" "github.com/metrue/fx/utils"
"github.com/pkg/errors" "github.com/otiai10/copy"
) )
// BuildImage build image // BuildImage build image
func BuildImage(ctx context.Contexter) error { func BuildImage(ctx context.Contexter) (err error) {
cli := ctx.GetCliContext() spinner.Start("building")
funcFile := cli.Args().First() defer func() {
tag := cli.String("tag") spinner.Stop("building", err)
if tag == "" { }()
tag = uuid.New().String()
}
workdir := fmt.Sprintf("/tmp/fx-%d", time.Now().Unix()) workdir := fmt.Sprintf("/tmp/fx-%d", time.Now().Unix())
defer os.RemoveAll(workdir) defer os.RemoveAll(workdir)
body, err := ioutil.ReadFile(funcFile) sources := ctx.Get("sources").([]string)
if err != nil {
log.Fatalf("function code load failed: %v", err) if len(sources) == 0 {
return err return fmt.Errorf("source file/directory of function required")
} }
log.Infof("function code loaded: %v", constants.CheckedSymbol) if len(sources) == 1 &&
lang := utils.GetLangFromFileName(funcFile) utils.IsDir(sources[0]) &&
utils.HasDockerfile(sources[0]) {
fn := types.Func{Language: lang, Source: string(body)} if err := copy.Copy(sources[0], workdir); err != nil {
return err
if err := packer.PackIntoDir(fn, workdir); err != nil { }
log.Fatalf("could not pack function %v: %v", fn, err) } else {
return err if err := packer.Pack(workdir, sources...); err != nil {
}
docker, ok := ctx.Get("docker").(containerruntimes.ContainerRuntime)
if ok {
nameWithTag := tag + ":latest"
if err := docker.BuildImage(ctx.GetContext(), workdir, nameWithTag); err != nil {
return err return err
} }
log.Infof("image built: %v", constants.CheckedSymbol)
return nil
} }
return fmt.Errorf("no available docker cli")
docker := ctx.Get("docker").(containerruntimes.ContainerRuntime)
nameWithTag := ctx.Get("tag").(string) + ":latest"
if err := docker.BuildImage(ctx.GetContext(), workdir, nameWithTag); err != nil {
return err
}
log.Infof("image built: %s %v", nameWithTag, constants.CheckedSymbol)
return nil
} }
// ExportImage export service's code into a directory // ExportImage export service's code into a directory
func ExportImage(ctx context.Contexter) (err error) { func ExportImage(ctx context.Contexter) (err error) {
cli := ctx.GetCliContext() outputDir := ctx.Get("output").(string)
funcFile := cli.Args().First() sources := ctx.Get("sources").([]string)
outputDir := cli.String("output")
if outputDir == "" { if len(sources) == 0 {
log.Fatalf("output directory required") return fmt.Errorf("source file/directory of function required")
return nil }
if len(sources) == 1 &&
utils.IsDir(sources[0]) &&
utils.HasDockerfile(sources[0]) {
if err := copy.Copy(sources[0], outputDir); err != nil {
return err
}
} else {
if err := packer.Pack(outputDir, sources...); err != nil {
return err
}
} }
body, err := ioutil.ReadFile(funcFile)
if err != nil {
return errors.Wrap(err, "read source failed")
}
lang := utils.GetLangFromFileName(funcFile)
if err := packer.PackIntoDir(types.Func{Language: lang, Source: string(body)}, outputDir); err != nil {
log.Fatalf("write source code to file failed: %v", constants.UncheckedSymbol)
return err
}
log.Infof("exported to %v: %v", outputDir, constants.CheckedSymbol) log.Infof("exported to %v: %v", outputDir, constants.CheckedSymbol)
return nil return nil
} }

View File

@@ -35,7 +35,6 @@ func setupK8S(masterInfo string, agentsInfo string) ([]byte, error) {
} }
} }
fmt.Println(master, agents, len(agents))
k8sOperator := k8s.New(master, agents) k8sOperator := k8s.New(master, agents)
return k8sOperator.Provision() return k8sOperator.Provision()
} }

View File

@@ -9,8 +9,14 @@ import (
// Up command handle // Up command handle
func Up(ctx context.Contexter) (err error) { func Up(ctx context.Contexter) (err error) {
fn := ctx.Get("data").(string) fn, ok := ctx.Get("data").(string)
image := ctx.Get("image").(string) if !ok {
fn = ""
}
image, ok := ctx.Get("image").(string)
if !ok {
image = ""
}
name := ctx.Get("name").(string) name := ctx.Get("name").(string)
deployer := ctx.Get("deployer").(infra.Deployer) deployer := ctx.Get("deployer").(infra.Deployer)
bindings := ctx.Get("bindings").([]types.PortBinding) bindings := ctx.Get("bindings").([]types.PortBinding)

View File

@@ -62,7 +62,7 @@ func (k *Provisioner) HealthCheck() (bool, error) {
func (k *Provisioner) SetupMaster() error { func (k *Provisioner) SetupMaster() error {
sshKeyFile, _ := infra.GetSSHKeyFile() sshKeyFile, _ := infra.GetSSHKeyFile()
ssh := sshOperator.New(k.master.IP).WithUser(k.master.User).WithKey(sshKeyFile) ssh := sshOperator.New(k.master.IP).WithUser(k.master.User).WithKey(sshKeyFile)
installCmd := fmt.Sprintf("curl -sLS https://get.k3s.io | INSTALL_K3S_EXEC='server --tls-san %s' INSTALL_K3S_VERSION='%s' sh -", k.master.IP, version) installCmd := fmt.Sprintf("curl -sLS https://get.k3s.io | INSTALL_K3S_EXEC='server --docker --tls-san %s' INSTALL_K3S_VERSION='%s' sh -", k.master.IP, version)
if err := ssh.RunCommand(infra.Sudo(installCmd, k.master.User), sshOperator.CommandOptions{ if err := ssh.RunCommand(infra.Sudo(installCmd, k.master.User), sshOperator.CommandOptions{
Stdout: os.Stdout, Stdout: os.Stdout,
Stdin: os.Stdin, Stdin: os.Stdin,
@@ -97,7 +97,7 @@ func (k *Provisioner) SetupAgent() error {
if err != nil { if err != nil {
return err return err
} }
const k3sExtraArgs = "" const k3sExtraArgs = "--docker"
joinCmd := fmt.Sprintf("curl -fL https://get.k3s.io/ | K3S_URL='https://%s:6443' K3S_TOKEN='%s' INSTALL_K3S_VERSION='%s' sh -s - %s", k.master.IP, tok, version, k3sExtraArgs) joinCmd := fmt.Sprintf("curl -fL https://get.k3s.io/ | K3S_URL='https://%s:6443' K3S_TOKEN='%s' INSTALL_K3S_VERSION='%s' sh -s - %s", k.master.IP, tok, version, k3sExtraArgs)
for _, agent := range k.agents { for _, agent := range k.agents {
ssh := sshOperator.New(agent.IP).WithUser(agent.User).WithKey(sshKeyFile) ssh := sshOperator.New(agent.IP).WithUser(agent.User).WithKey(sshKeyFile)

View File

@@ -5,10 +5,13 @@ import (
"os" "os"
"time" "time"
"github.com/metrue/fx/config"
containerruntimes "github.com/metrue/fx/container_runtimes" containerruntimes "github.com/metrue/fx/container_runtimes"
"github.com/metrue/fx/context" "github.com/metrue/fx/context"
"github.com/metrue/fx/packer" "github.com/metrue/fx/packer"
"github.com/metrue/fx/pkg/spinner" "github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/utils"
"github.com/otiai10/copy"
) )
// Build image // Build image
@@ -19,40 +22,68 @@ func Build(ctx context.Contexter) (err error) {
spinner.Stop(task, err) spinner.Stop(task, err)
}() }()
name := ctx.Get("name").(string)
docker := ctx.Get("docker").(containerruntimes.ContainerRuntime)
workdir := fmt.Sprintf("/tmp/fx-%d", time.Now().Unix()) workdir := fmt.Sprintf("/tmp/fx-%d", time.Now().Unix())
defer os.RemoveAll(workdir) defer os.RemoveAll(workdir)
if err := packer.Pack(workdir, ctx.Get("sources").([]string)...); err != nil { // Cases supports
return err // 1. a single file function
// fx up func.js
// 2. a directory with Docker in it
// fx up ./func/
// 3. a directory without Dockerfile in it, but has fx handle function file
// 4. a fx handlefunction file and its dependencies files or/and directory
// fx up func.js helper.js ./lib/
// When only one directory given and there is a Dockerfile in given directory, treat it as a containerized project and skip packing
sources := ctx.Get("sources").([]string)
if len(sources) == 0 {
return fmt.Errorf("source file/directory of function required")
} }
data, err := packer.PackIntoK8SConfigMapFile(workdir) if len(sources) == 1 &&
if err != nil { utils.IsDir(sources[0]) &&
return err utils.HasDockerfile(sources[0]) {
} if err := copy.Copy(sources[0], workdir); err != nil {
ctx.Set("data", data) return err
}
if err := docker.BuildImage(ctx.GetContext(), workdir, name); err != nil { } else {
return err if err := packer.Pack(workdir, sources...); err != nil {
} return err
nameWithTag := name + ":latest"
if err := docker.TagImage(ctx.GetContext(), name, nameWithTag); err != nil {
return err
}
ctx.Set("image", nameWithTag)
if os.Getenv("K3S") != "" {
username := os.Getenv("DOCKER_USERNAME")
password := os.Getenv("DOCKER_PASSWORD")
if username != "" && password != "" {
if _, err := docker.PushImage(ctx.GetContext(), name); err != nil {
return err
}
ctx.Set("image", username+"/"+name)
} }
} }
cloudType := ctx.Get("cloud_type").(string)
name := ctx.Get("name").(string)
if cloudType == config.CloudTypeK8S && os.Getenv("K3S") == "" {
data, err := packer.PackIntoK8SConfigMapFile(workdir)
if err != nil {
return err
}
ctx.Set("data", data)
} else {
docker := ctx.Get("docker").(containerruntimes.ContainerRuntime)
if err := docker.BuildImage(ctx.GetContext(), workdir, name); err != nil {
return err
}
nameWithTag := name + ":latest"
if err := docker.TagImage(ctx.GetContext(), name, nameWithTag); err != nil {
return err
}
ctx.Set("image", nameWithTag)
if os.Getenv("K3S") != "" {
username := os.Getenv("DOCKER_USERNAME")
password := os.Getenv("DOCKER_PASSWORD")
if username != "" && password != "" {
if _, err := docker.PushImage(ctx.GetContext(), name); err != nil {
return err
}
ctx.Set("image", username+"/"+name)
}
}
}
return nil return nil
} }

View File

@@ -1,10 +1,10 @@
package middlewares package middlewares
import ( import (
"os" "fmt"
"github.com/google/uuid"
"github.com/metrue/fx/context" "github.com/metrue/fx/context"
"github.com/pkg/errors"
) )
// Parse parse input // Parse parse input
@@ -17,13 +17,6 @@ func Parse(action string) func(ctx context.Contexter) (err error) {
for _, s := range cli.Args() { for _, s := range cli.Args() {
sources = append(sources, s) sources = append(sources, s)
} }
if len(sources) == 0 {
pwd, err := os.Getwd()
if err != nil {
return err
}
sources = append(sources, pwd)
}
ctx.Set("sources", sources) ctx.Set("sources", sources)
name := cli.String("name") name := cli.String("name")
ctx.Set("name", name) ctx.Set("name", name)
@@ -32,7 +25,7 @@ func Parse(action string) func(ctx context.Contexter) (err error) {
case "down": case "down":
services := cli.Args() services := cli.Args()
if len(services) == 0 { if len(services) == 0 {
return errors.New("service name required") return fmt.Errorf("service name required")
} }
svc := []string{} svc := []string{}
for _, service := range services { for _, service := range services {
@@ -42,6 +35,28 @@ func Parse(action string) func(ctx context.Contexter) (err error) {
case "list": case "list":
name := cli.Args().First() name := cli.Args().First()
ctx.Set("filter", name) ctx.Set("filter", name)
case "image_build":
sources := []string{}
for _, s := range cli.Args() {
sources = append(sources, s)
}
ctx.Set("sources", sources)
tag := cli.String("tag")
if tag == "" {
tag = uuid.New().String()
}
ctx.Set("tag", tag)
case "image_export":
sources := []string{}
for _, s := range cli.Args() {
sources = append(sources, s)
}
ctx.Set("sources", sources)
outputDir := cli.String("output")
if outputDir == "" {
return fmt.Errorf("output directory required")
}
ctx.Set("output", outputDir)
} }
return nil return nil

View File

@@ -20,7 +20,13 @@ func Provision(ctx context.Contexter) (err error) {
cloud := fxConfig.Clouds[fxConfig.CurrentCloud] cloud := fxConfig.Clouds[fxConfig.CurrentCloud]
var deployer infra.Deployer var deployer infra.Deployer
if cloud["type"] == config.CloudTypeDocker { if os.Getenv("KUBECONFIG") != "" {
deployer, err = k8sInfra.CreateDeployer(os.Getenv("KUBECONFIG"))
if err != nil {
return err
}
ctx.Set("cloud_type", config.CloudTypeK8S)
} else if cloud["type"] == config.CloudTypeDocker {
provisioner := dockerInfra.CreateProvisioner(cloud["host"], cloud["user"]) provisioner := dockerInfra.CreateProvisioner(cloud["host"], cloud["user"])
ok, err := provisioner.HealthCheck() ok, err := provisioner.HealthCheck()
if err != nil { if err != nil {
@@ -43,13 +49,13 @@ func Provision(ctx context.Contexter) (err error) {
if err != nil { if err != nil {
return err return err
} }
ctx.Set("cloud_type", config.CloudTypeDocker)
} else if cloud["type"] == config.CloudTypeK8S { } else if cloud["type"] == config.CloudTypeK8S {
if os.Getenv("KUBECONFIG") != "" { deployer, err = k8sInfra.CreateDeployer(cloud["kubeconfig"])
deployer, err = k8sInfra.CreateDeployer(cloud["kubeconfig"]) if err != nil {
if err != nil { return err
return err
}
} }
ctx.Set("cloud_type", config.CloudTypeK8S)
} else { } else {
return fmt.Errorf("unsupport cloud type %s, please make sure you config is correct", cloud["type"]) return fmt.Errorf("unsupport cloud type %s, please make sure you config is correct", cloud["type"])
} }

14
packer/doc.go Normal file
View File

@@ -0,0 +1,14 @@
/*
Packer takes source codes of a function, and pack them into a containerized service, that means there is Dockerfile generated in the output directory
e.g.
Pack(output, "hello.js") # a single file function
Pack(output, "hello.js", "helper.js") # multiple files function
Pack(output, "./func/") # a directory of function
Pack(output, "hello.js", "./func/") # a directory and files of function
*/
package packer

View File

@@ -1,63 +0,0 @@
package packer
import (
"fmt"
"path/filepath"
"strings"
"github.com/gobuffalo/packr"
"github.com/metrue/fx/types"
)
// DockerPacker pack a function source code to a Docker build-able project
type DockerPacker struct {
box packr.Box
}
func isHandler(name string) bool {
basename := filepath.Base(name)
nameWithoutExt := strings.TrimSuffix(basename, filepath.Ext(basename))
return nameWithoutExt == "fx" ||
nameWithoutExt == "Fx" || // Fx is for Java
nameWithoutExt == "mod" // mod.rs is for Rust
}
// NewDockerPacker new a Docker packer
func NewDockerPacker(box packr.Box) *DockerPacker {
return &DockerPacker{box: box}
}
// Pack pack a single function source code to be project
func (p *DockerPacker) Pack(serviceName string, fn types.Func) (types.Project, error) {
var files []types.ProjectSourceFile
for _, name := range p.box.List() {
prefix := fmt.Sprintf("%s/", fn.Language)
if strings.HasPrefix(name, prefix) {
content, err := p.box.FindString(name)
if err != nil {
return types.Project{}, err
}
// if preset's file is handler function of project, replace it with give one
if isHandler(name) {
files = append(files, types.ProjectSourceFile{
Path: strings.Replace(name, prefix, "", 1),
Body: fn.Source,
IsHandler: true,
})
} else {
files = append(files, types.ProjectSourceFile{
Path: strings.Replace(name, prefix, "", 1),
Body: content,
IsHandler: false,
})
}
}
}
return types.Project{
Name: serviceName,
Files: files,
Language: fn.Language,
}, nil
}

View File

@@ -1,64 +0,0 @@
package packer
import (
"testing"
"github.com/gobuffalo/packr"
"github.com/golang/mock/gomock"
"github.com/metrue/fx/types"
)
func TestDockerPacker(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
box := packr.NewBox("./images")
p := NewDockerPacker(box)
mockSource := `
module.exports = ({a, b}) => {
return a + b
}
`
fn := types.Func{
Language: "node",
Source: mockSource,
}
serviceName := "service-mock"
project, err := p.Pack(serviceName, fn)
if err != nil {
t.Fatal(err)
}
if project.Name != serviceName {
t.Fatalf("should get %s but got %s", serviceName, project.Name)
}
if project.Language != "node" {
t.Fatal("incorrect Language")
}
if len(project.Files) != 3 {
t.Fatal("node project should have 3 files")
}
for _, file := range project.Files {
if file.Path == "fx.js" {
if file.IsHandler == false {
t.Fatal("fx.js should be handler")
}
if file.Body != mockSource {
t.Fatalf("should get %s but got %v", mockSource, file.Body)
}
} else if file.Path == "Dockerfile" {
if file.IsHandler == true {
t.Fatalf("should get %v but got %v", false, file.IsHandler)
}
} else {
if file.IsHandler == true {
t.Fatalf("should get %v but %v", false, file.IsHandler)
}
}
}
}

View File

@@ -1,142 +1,207 @@
package packer package packer
import ( import (
"encoding/base64"
"encoding/json"
"fmt" "fmt"
"io/ioutil" "io/ioutil"
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"time"
"encoding/base64"
"encoding/json"
"github.com/gobuffalo/packr" "github.com/gobuffalo/packr"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils" "github.com/metrue/fx/utils"
"github.com/otiai10/copy" "github.com/otiai10/copy"
"github.com/pkg/errors"
) )
var presets packr.Box
func init() {
presets = packr.NewBox("./images")
}
// Pack pack a file or directory into a Docker project // Pack pack a file or directory into a Docker project
func Pack(output string, input ...string) error { func Pack(output string, input ...string) error {
if len(input) == 0 { if len(input) == 0 {
return fmt.Errorf("source file or directory required") return fmt.Errorf("source file or directory required")
} }
var lang string
for _, f := range input {
if utils.IsRegularFile(f) {
lang = langFromFileName(f)
} else if utils.IsDir(f) {
if err := filepath.Walk(f, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if utils.IsRegularFile(path) {
lang = langFromFileName(path)
}
return nil
}); err != nil {
return err
}
}
}
if lang == "" {
return fmt.Errorf("could not tell programe language of your input source codes")
}
if err := restore(output, lang); err != nil {
return err
}
if len(input) == 1 { if len(input) == 1 {
file := input[0] stat, err := os.Stat(input[0])
if err != nil {
return err
}
if stat.Mode().IsRegular() {
if err := filepath.Walk(output, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if isHandler(path) {
if err := copy.Copy(input[0], path); err != nil {
return err
}
}
return nil
}); err != nil {
return err
}
}
return nil
}
if !hasFxHandleFile(input...) {
msg := `it requires a fx handle file when input is not a single file function, e.g.  
fx.go for Golang
Fx.java for Java
fx.php for PHP
fx.py for Python
fx.js for JavaScript or Node
fx.rb for Ruby
fx.jl for Julia
fx.d for D`
return fmt.Errorf(msg)
}
if err := merge(output, input...); err != nil {
return err
}
return nil
}
func restore(output string, lang string) error {
for _, name := range presets.List() {
prefix := fmt.Sprintf("%s/", lang)
if strings.HasPrefix(name, prefix) {
content, err := presets.FindString(name)
if err != nil {
return err
}
filePath := filepath.Join(output, strings.Replace(name, prefix, "", 1))
if err := utils.EnsureFile(filePath); err != nil {
return err
}
if err := ioutil.WriteFile(filePath, []byte(content), 0666); err != nil {
return err
}
}
}
return nil
}
func merge(dest string, input ...string) error {
for _, file := range input {
stat, err := os.Stat(file) stat, err := os.Stat(file)
if err != nil { if err != nil {
return err return err
} }
if !stat.IsDir() { if stat.Mode().IsRegular() {
lang := utils.GetLangFromFileName(file) targetFilePath := filepath.Join(dest, stat.Name())
body, err := ioutil.ReadFile(file) if err := utils.EnsureFile(targetFilePath); err != nil {
if err != nil {
return errors.Wrap(err, "read source failed")
}
fn := types.Func{
Language: lang,
Source: string(body),
}
if err := PackIntoDir(fn, output); err != nil {
return err return err
} }
return nil body, err := ioutil.ReadFile(file)
} if err != nil {
} return err
}
if err := ioutil.WriteFile(targetFilePath, body, 0644); err != nil {
return err
}
} else if stat.Mode().IsDir() {
if err := filepath.Walk(file, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
workdir := fmt.Sprintf("./fx-%d", time.Now().Unix()) if err := copy.Copy(file, dest); err != nil {
defer os.RemoveAll(workdir) return err
}
for _, f := range input { return nil
if err := copy.Copy(f, filepath.Join(workdir, f)); err != nil { }); err != nil {
return err return err
} }
}
if dockerfile, has := hasDockerfileInDir(workdir); has {
return copy.Copy(filepath.Dir(dockerfile), output)
}
if f, has := hasFxHandleFileInDir(workdir); has {
lang := utils.GetLangFromFileName(f)
body, err := ioutil.ReadFile(f)
if err != nil {
return errors.Wrap(err, "read source failed")
}
fn := types.Func{
Language: lang,
Source: string(body),
}
if err := PackIntoDir(fn, output); err != nil {
return err
}
return copy.Copy(filepath.Dir(f), output)
}
return fmt.Errorf("input directories or files has no Dockerfile or file with fx as name, e.g. fx.js")
}
func hasDockerfileInDir(dir string) (string, bool) {
var dockerfile string
if err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
// nolint
if !info.IsDir() && info.Name() == "Dockerfile" {
dockerfile = path
}
return nil
}); err != nil {
return "", false
}
if dockerfile == "" {
return "", false
}
return dockerfile, true
}
func hasFxHandleFileInDir(dir string) (string, bool) {
var handleFile string
if err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if !info.IsDir() && isHandler(info.Name()) {
handleFile = path
}
return nil
}); err != nil {
return "", false
}
if handleFile == "" {
return "", false
}
return handleFile, true
}
// Pack a function to be a docker project which is web service, handle the imcome request with given function
func pack(svcName string, fn types.Func) (types.Project, error) {
box := packr.NewBox("./images")
pkr := NewDockerPacker(box)
return pkr.Pack(svcName, fn)
}
// PackIntoDir pack service code into directory
func PackIntoDir(fn types.Func, outputDir string) error {
project, err := pack("", fn)
if err != nil {
return err
}
for _, file := range project.Files {
tmpfn := filepath.Join(outputDir, file.Path)
if err := utils.EnsureFile(tmpfn); err != nil {
return err
}
if err := ioutil.WriteFile(tmpfn, []byte(file.Body), 0666); err != nil {
return err
} }
} }
return nil return nil
} }
func isHandler(name string) bool {
basename := filepath.Base(name)
nameWithoutExt := strings.TrimSuffix(basename, filepath.Ext(basename))
return nameWithoutExt == "fx" ||
nameWithoutExt == "Fx" || // Fx is for Java
nameWithoutExt == "mod" // mod.rs is for Rust
}
func langFromFileName(fileName string) string {
extLangMap := map[string]string{
".js": "node",
".go": "go",
".rb": "ruby",
".py": "python",
".php": "php",
".jl": "julia",
".java": "java",
".d": "d",
".rs": "rust",
}
return extLangMap[filepath.Ext(fileName)]
}
func hasFxHandleFile(input ...string) bool {
var handleFile string
for _, file := range input {
if utils.IsRegularFile(file) && isHandler(file) {
handleFile = file
break
} else if utils.IsDir(file) {
if err := filepath.Walk(file, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if utils.IsRegularFile(path) && isHandler(info.Name()) {
handleFile = path
}
return nil
}); err != nil {
return false
}
}
}
return handleFile != ""
}
// PackIntoK8SConfigMapFile pack function a K8S config map file // PackIntoK8SConfigMapFile pack function a K8S config map file
func PackIntoK8SConfigMapFile(dir string) (string, error) { func PackIntoK8SConfigMapFile(dir string) (string, error) {
tree := map[string]string{} tree := map[string]string{}
@@ -176,18 +241,3 @@ func TreeToDir(tree map[string]string, outputDir string) error {
} }
return nil return nil
} }
// PackIntoTar pack service code into directory
func PackIntoTar(fn types.Func, path string) error {
tarDir, err := ioutil.TempDir("/tmp", "fx-tar")
if err != nil {
return err
}
defer os.RemoveAll(tarDir)
if err := PackIntoDir(fn, tarDir); err != nil {
return err
}
return utils.TarDir(tarDir, path)
}

View File

@@ -1,10 +1,15 @@
package packer package packer
import ( import (
"fmt"
"io/ioutil"
"log"
"os" "os"
"os/exec"
"testing" "testing"
"time"
"github.com/metrue/fx/types" "github.com/metrue/fx/utils"
) )
func TestPacker(t *testing.T) { func TestPacker(t *testing.T) {
@@ -64,55 +69,6 @@ func TestPacker(t *testing.T) {
t.Fatalf("should report error when there is not Dockerfile or fx.[ext] in it") t.Fatalf("should report error when there is not Dockerfile or fx.[ext] in it")
} }
}) })
t.Run("pack", func(t *testing.T) {
mockSource := `
module.exports = ({a, b}) => {
return a + b
}
`
fn := types.Func{
Language: "node",
Source: mockSource,
}
serviceName := "service-mock"
project, err := pack(serviceName, fn)
if err != nil {
t.Fatal(err)
}
if project.Name != serviceName {
t.Fatalf("should get %s but got %s", serviceName, project.Name)
}
if project.Language != "node" {
t.Fatal("incorrect Language")
}
if len(project.Files) != 3 {
t.Fatal("node project should have 3 files")
}
for _, file := range project.Files {
if file.Path == "fx.js" {
if file.IsHandler == false {
t.Fatal("fx.js should be handler")
}
if file.Body != mockSource {
t.Fatalf("should get %s but got %v", mockSource, file.Body)
}
} else if file.Path == "Dockerfile" {
if file.IsHandler == true {
t.Fatalf("should get %v but got %v", false, file.IsHandler)
}
} else {
if file.IsHandler == true {
t.Fatalf("should get %v but %v", false, file.IsHandler)
}
}
}
})
} }
func TestTreeAndUnTree(t *testing.T) { func TestTreeAndUnTree(t *testing.T) {
@@ -121,3 +77,106 @@ func TestTreeAndUnTree(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
} }
func TestGenerate(t *testing.T) {
langs := []string{
"d",
"go",
"java",
"julia",
"node",
"php",
"python",
"ruby",
"rust",
}
for _, lang := range langs {
output := fmt.Sprintf("output-%s-%d", lang, time.Now().Unix())
defer func() {
os.RemoveAll(output)
}()
if err := restore(output, lang); err != nil {
t.Fatal(err)
}
diffCmd := exec.Command("diff", "-r", output, "./images/"+lang)
if stdoutStderr, err := diffCmd.CombinedOutput(); err != nil {
fmt.Printf("%s\n", stdoutStderr)
t.Fatal(err)
}
}
}
func TestMerge(t *testing.T) {
// TODO should check the merge result
t.Run("NoInput", func(t *testing.T) {
dest := "./dest"
_ = utils.EnsureDir("./dest")
defer func() {
os.RemoveAll(dest)
}()
if err := merge(dest); err != nil {
t.Fatal(err)
}
})
t.Run("Files", func(t *testing.T) {
dest := "./dest"
_ = utils.EnsureDir("./dest")
defer func() {
os.RemoveAll(dest)
}()
f1, err := ioutil.TempFile("", "fx.*.txt")
if err != nil {
log.Fatal(err)
}
defer os.Remove(f1.Name())
f2, err := ioutil.TempFile("", "fx.*.txt")
if err != nil {
log.Fatal(err)
}
defer os.Remove(f2.Name())
if err := merge(dest, f1.Name(), f2.Name()); err != nil {
t.Fatal(err)
}
})
t.Run("Directories", func(t *testing.T) {
dest := "./dest"
_ = utils.EnsureDir("./dest")
defer func() {
os.RemoveAll(dest)
}()
if err := merge(dest, "./fixture/p1"); err != nil {
t.Fatal(err)
}
})
t.Run("Files and Directories", func(t *testing.T) {
dest := "./dest"
_ = utils.EnsureDir("./dest")
defer func() {
os.RemoveAll(dest)
}()
f1, err := ioutil.TempFile("", "fx.*.txt")
if err != nil {
log.Fatal(err)
}
defer os.Remove(f1.Name())
f2, err := ioutil.TempFile("", "fx.*.txt")
if err != nil {
log.Fatal(err)
}
defer os.Remove(f2.Name())
if err := merge(dest, "./fixture/p1", f1.Name(), f2.Name()); err != nil {
t.Fatal(err)
}
})
}

View File

@@ -1,47 +1,24 @@
package utils package utils
import ( import (
"encoding/json" "os"
"fmt" "path/filepath"
"io/ioutil"
"net/http"
"strings"
"time"
dockerTypes "github.com/docker/docker/api/types"
) )
// DockerVersion docker verion // HasDockerfile check if there is Dockerfile in dir
func DockerVersion(host string, port string) (string, error) { func HasDockerfile(dir string) bool {
path := host + ":" + port + "/version" var dockerfile string
if !strings.HasPrefix(path, "http") { if err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
path = "http://" + path // nolint
if info.Mode().IsRegular() && info.Name() == "Dockerfile" {
dockerfile = path
}
return nil
}); err != nil {
return false
} }
if dockerfile == "" {
req, err := http.NewRequest("GET", path, nil) return false
if err != nil {
return "", err
} }
client := &http.Client{Timeout: 20 * time.Second} return true
resp, err := client.Do(req)
if err != nil {
return "", err
}
if resp.StatusCode != 200 {
return "", fmt.Errorf("request %s failed: %d - %s", path, resp.StatusCode, resp.Status)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return "", err
}
var res dockerTypes.Version
err = json.Unmarshal(body, &res)
if err != nil {
return "", err
}
return res.APIVersion, nil
} }

12
utils/docker_test.go Normal file
View File

@@ -0,0 +1,12 @@
package utils
import "testing"
func TestHasDockerfile(t *testing.T) {
dir := "tmp"
_ = EnsureDir(dir)
if HasDockerfile(dir) {
t.Fatalf("should get false but got true")
}
}

View File

@@ -176,8 +176,8 @@ func CopyDir(src string, dst string) (err error) {
return return
} }
// EnsurerDir Create Dir if not exist // EnsureDir Create Dir if not exist
func EnsurerDir(dir string) (err error) { func EnsureDir(dir string) (err error) {
if _, statError := os.Stat(dir); os.IsNotExist(statError) { if _, statError := os.Stat(dir); os.IsNotExist(statError) {
mkError := os.MkdirAll(dir, os.ModePerm) mkError := os.MkdirAll(dir, os.ModePerm)
return mkError return mkError
@@ -188,7 +188,7 @@ func EnsurerDir(dir string) (err error) {
// EnsureFile ensure a file // EnsureFile ensure a file
func EnsureFile(fullpath string) error { func EnsureFile(fullpath string) error {
dir := path.Dir(fullpath) dir := path.Dir(fullpath)
err := EnsurerDir(dir) err := EnsureDir(dir)
if err != nil { if err != nil {
return err return err
} }
@@ -199,6 +199,24 @@ func EnsureFile(fullpath string) error {
return nil return nil
} }
// IsDir if given path is a directory
func IsDir(dir string) bool {
stat, err := os.Stat(dir)
if err != nil {
return false
}
return stat.IsDir()
}
// IsRegularFile if given path is a regular
func IsRegularFile(file string) bool {
stat, err := os.Stat(file)
if err != nil {
return false
}
return stat.Mode().IsRegular()
}
// IsPathExists checks whether a path exists or if failed to check // IsPathExists checks whether a path exists or if failed to check
func IsPathExists(path string) (bool, error) { func IsPathExists(path string) (bool, error) {
_, err := os.Stat(path) _, err := os.Stat(path)