Compare commits

...

44 Commits

Author SHA1 Message Date
Minghe Huang
1b0626b95d Refactor provision layer 2019-12-12 19:13:52 +08:00
Minghe
2298f39cca k3s on docker (#401)
* since we do the image build in initialize container of pod, so we have to make sure the image built can be access from kubelet on same node, containerd could not support that
* update docs
* bump version
* clean up
2019-12-07 01:42:13 +08:00
Minghe
23d68bc27b Refactor packer (#399) 2019-12-06 20:16:05 +08:00
Minghe
74c0423f0d add docs for rust example (#398) 2019-12-06 14:58:30 +08:00
Minghe
06f87c4d8e fix deploy to aks issue, and update docs (#396)
* fix deploy to aks issue, and update docs
* update docs
2019-12-06 11:54:27 +08:00
Minghe
35262de828 release v0.8.7 (#394) 2019-12-06 10:04:40 +08:00
dependabot-preview[bot]
34a495984c Bump github.com/olekukonko/tablewriter from 0.0.3 to 0.0.4 (#393)
Bumps [github.com/olekukonko/tablewriter](https://github.com/olekukonko/tablewriter) from 0.0.3 to 0.0.4.
- [Release notes](https://github.com/olekukonko/tablewriter/releases)
- [Commits](https://github.com/olekukonko/tablewriter/compare/v0.0.3...v0.0.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-06 09:45:27 +08:00
Minghe
d7130c4e28 support a project with multiple files (#392)
* support a project with multiple files
* fix lint issue
* fix test
2019-12-05 17:57:39 +08:00
Minghe
c9630a53c3 should treat CICD code block seperately (#391)
* should treat CICD code block seperately

* auto provision when host is not health

* fix lint
2019-12-05 11:38:54 +08:00
Minghe
0522690472 merge provisioner and deployer into infra (#390) 2019-12-04 19:09:14 +08:00
Minghe
a8a0fbed32 add ping method to check infra is healthy or not (#388)
* add ping method to check infra is healthy or not
* merge k3s and k8s deployer (#389)
* fix typo
2019-12-04 17:28:03 +08:00
Minghe
26ae9585f6 move cli args parsing into middleware level (#387) 2019-12-04 11:56:07 +08:00
Minghe
b69bd699c8 combine coverage and unit test progres (#386) 2019-12-04 09:45:42 +08:00
Minghe
650ee5f63a move progress bar into deploy function (#385) 2019-12-04 08:43:49 +08:00
Minghe
e3c60cbb77 show the endpoint information when deployed successfully (#384) 2019-12-03 19:27:26 +08:00
Minghe
0daca43d10 release v0.8.6 (#383) 2019-12-03 17:18:11 +08:00
Minghe
d3c239dc54 clean up no use codes (#381) 2019-12-03 17:01:14 +08:00
Minghe
05ac2441da list service as table format (#382) 2019-12-03 15:41:03 +08:00
Minghe
c0009b1b64 request a port when not port given (#379)
* request a port when not port given
* make context an interface, for easy testing (#380)
2019-12-03 15:14:13 +08:00
dependabot-preview[bot]
82960824ef Bump github.com/urfave/cli from 1.22.1 to 1.22.2 (#370)
Bumps [github.com/urfave/cli](https://github.com/urfave/cli) from 1.22.1 to 1.22.2.
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/master/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v1.22.1...v1.22.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-29 08:54:02 +08:00
dependabot-preview[bot]
64b63cbd0f Bump gopkg.in/yaml.v2 from 2.2.4 to 2.2.7 (#378)
Bumps [gopkg.in/yaml.v2](https://github.com/go-yaml/yaml) from 2.2.4 to 2.2.7.
- [Release notes](https://github.com/go-yaml/yaml/releases)
- [Commits](https://github.com/go-yaml/yaml/compare/v2.2.4...v2.2.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-28 08:41:26 +08:00
Minghe
05771fb07f read kubeconfig from .fx/config or env (#377) 2019-11-27 20:47:49 +08:00
Minghe
d1f680dacd auto load config (#376) 2019-11-27 18:09:06 +08:00
Minghe
14c9397b70 use ssh to install, no dependency on k3sup (#367) 2019-11-27 15:40:18 +08:00
Minghe
eb5e724899 disable aks ci since environment not ready (#362) 2019-11-17 13:07:23 +08:00
Minghe
80619bd800 release 0.8.4 (#361) 2019-11-17 12:17:58 +08:00
Minghe
8e2cdfc607 update docs (#360) 2019-11-17 11:29:08 +08:00
Minghe
58f416b7b2 support custom cluster setup by k3s (#359)
* support custom cluster setup by k3s
* clean up
* fix lint issue
2019-11-17 10:50:55 +08:00
Minghe
b6cf39e3e5 support k8s operator (#358) 2019-11-17 08:56:58 +08:00
Minghe
41bc98ab64 fx on k3s (#357) 2019-11-17 00:10:55 +08:00
Minghe
b007ac315a refactor output (#356) 2019-11-15 22:23:27 +08:00
Minghe
940f6b8f72 enable progress bar (#355)
* enable progress bar
* add spinner pkg
* fix lint
2019-11-15 21:17:38 +08:00
Minghe
f9690b74a5 auto startup fx agent during setup progress (#354)
* auto start fx agent during setup progress

* bump version
2019-11-15 19:41:17 +08:00
Minghe
f2c58d545a add document on ubunut (#352)
* add document on ubunut

* fix typo
2019-11-15 13:29:31 +08:00
Siddhesh Poyarekar
4732426629 Point to instructions to build from source for non-x86 targets (#350)
* Fix typos
* Point to instructions to build from source for non-x86 targets

Installation instructions do not specify clearly enough the fact that
they are supported only on x86. Make it clearer and point to the Build
and Test section in Contribute for instructions on building fx.
2019-11-14 15:22:25 +08:00
Minghe
d4af4f67b2 fix syntax error (#349) 2019-11-12 12:29:25 +08:00
Minghe
6420e8b6c6 add workflow graph (#348) 2019-11-12 12:26:33 +08:00
Minghe
15c59fa31f bump version and update README (#347) 2019-11-12 10:47:26 +08:00
Minghe
294131b48f use seperate script (#346) 2019-11-12 09:42:18 +08:00
Minghe
48413abaa1 add cov upload (#345) 2019-11-11 18:50:53 +08:00
Minghe
d36b2b935b fix lint 2019-11-11 18:15:55 +08:00
Minghe
f493749689 Better fx run remote docker host (#338) 2019-11-11 15:35:52 +08:00
dependabot-preview[bot]
9de10bc885 Bump github.com/spf13/viper from 1.4.0 to 1.5.0 (#339)
Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.4.0 to 1.5.0.
- [Release notes](https://github.com/spf13/viper/releases)
- [Commits](https://github.com/spf13/viper/compare/v1.4.0...v1.5.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-02 08:45:15 +08:00
Minghe
2d5446686a fix docker workflow (#337) 2019-10-27 21:55:17 +08:00
126 changed files with 4803 additions and 2526 deletions

View File

@@ -1,73 +0,0 @@
defaults: &defaults
machine: true
environment:
IMPORT_PATH: "github.com/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME"
OUTPUT_DIR: "./build"
DIST_DIR: "./dist"
install_golang: &install_golang
run:
name: install Golang 1.11
command: |
sudo add-apt-repository ppa:gophers/archive
sudo apt-get update
sudo apt-get install golang-1.11-go
alias go="/usr/lib/go-1.11/bin/go"
go version
install_deps: &install_deps
run:
name: Install deps
command: |
/usr/lib/go-1.11/bin/go mod vendor
/usr/lib/go-1.11/bin/go get -u github.com/gobuffalo/packr/packr
install_httpie: &install_httpie
run:
name: install httpie
command: |
sudo apt-get -y update && sudo apt-get -y install httpie
install_jq: &install_jq
run:
name: install jq
command: |
sudo apt-get update && sudo apt-get -y install jq
build_binary: &build_binary
run:
name: build binary
command: |
/usr/lib/go-1.11/bin/go build -o ${OUTPUT_DIR}/fx fx.go
unit_test: &unit_test
run:
name: unit test
command: |
make unit-test
bash <(curl -s https://codecov.io/bash) -t ${CODECOV_TOKEN}
cli_test: &cli_test
run:
name: cli test
command: make cli-test
version: 2
jobs:
test:
<<: *defaults
steps:
- checkout
- *install_golang
- *install_deps
- *unit_test
- *build_binary
- run:
name: Pull images
command: make pull
- *cli_test
workflows:
version: 2
workflow:
jobs:
- test

View File

@@ -13,27 +13,24 @@ jobs:
- name: check out
uses: actions/checkout@master
- name: setup docker
- name: kind create a k8s cluster
run: |
./scripts/provision.sh
kind create cluster
- name: setup k8s and kind
- name: lint
run: |
export GOBIN=$(go env GOPATH)/bin
export PATH=$PATH:$GOBIN
mkdir -p $GOBIN
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
chmod +x kubectl && mv kubectl $GOBIN
wget https://github.com/kubernetes-sigs/kind/releases/download/v0.5.0/kind-linux-amd64 && chmod +x kind-linux-amd64 && mv kind-linux-amd64 $GOBIN/kind
./scripts/setup_kind.sh
docker run --rm -v $(pwd):/app -w /app golangci/golangci-lint \
golangci-lint run -v
- name: unit test
env:
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
run: |
export KUBECONFIG=/home/runner/.kube/kind-config-fx-test
DEBUG=true go test -v ./container_runtimes/... ./deploy/...
export KUBECONFIG="$(kind get kubeconfig-path)"
./scripts/coverage.sh
bash <(curl -s https://codecov.io/bash) -t ${CODECOV_TOKEN}
- name: build fx
run: |
@@ -46,34 +43,36 @@ jobs:
make docker-build
make test
# make docker-publish #TODO in release workflow
- name: lint
- name: test fx docker cloud
run: |
export GOBIN=$(go env GOPATH)/bin
export PATH=$PATH:$GOBIN
go get -u github.com/golangci/golangci-lint/cmd/golangci-lint
golangci-lint run
make start_docker_infra
make test_docker_infra
make stop_docker_infra
- name: test fx cli
env:
REMOTE_HOST_ADDR: ${{secrets.DOCKER_REMOTE_HOST_ADDR}}
REMOTE_HOST_USER: ${{secrets.DOCKER_REMOTE_HOST_USER}}
REMOTE_HOST_PASSWORD: ${{secrets.DOCKER_REMOTE_HOST_PASSWORD}}
run: |
echo $KUBECONFIG
unset KUBECONFIG
make cli-test
make cli-test-ci
- name: test AKS
env:
AKS_KUBECONFIG: ${{ secrets.AKS_KUBECONFIG }}
run: |
export KUBECONFIG=${HOME}/.kube/aks
echo ${AKS_KUBECONFIG} | base64 -d > $KUBECONFIG
if [[ -z "$AKS_KUBECONFIG" ]];then
echo "skip deploy test since no valid KUBECONFIG"
else
DEBUG=true ./build/fx up -n hello -p 12345 examples/functions/JavaScript/func.js
./build/fx down hello
rm ${KUBECONFIG}
fi
echo "skip since aks environment not ready yet"
# export KUBECONFIG=${HOME}/.kube/aks
# echo ${AKS_KUBECONFIG} | base64 -d > $KUBECONFIG
# if [[ -z "$AKS_KUBECONFIG" ]];then
# echo "skip deploy test since no valid KUBECONFIG"
# else
# DEBUG=true ./build/fx up -n hello -p 12345 examples/functions/JavaScript/func.js
# ./build/fx down hello
# rm ${KUBECONFIG}
# fi
Installation:
runs-on: ${{ matrix.os }}

View File

@@ -18,40 +18,27 @@ jobs:
- name: check out
uses: actions/checkout@master
- name: setup docker
- name: kind create a k8s cluster
run: |
./scripts/provision.sh
kind create cluster
- name: setup k8s and kind
- name: lint
run: |
export GOBIN=$(go env GOPATH)/bin
export PATH=$PATH:$GOBIN
mkdir -p $GOBIN
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
chmod +x kubectl && mv kubectl $GOBIN
wget https://github.com/kubernetes-sigs/kind/releases/download/v0.5.0/kind-linux-amd64 && chmod +x kind-linux-amd64 && mv kind-linux-amd64 $GOBIN/kind
./scripts/setup_kind.sh
docker run --rm -v $(pwd):/app -w /app golangci/golangci-lint \
golangci-lint run -v
- name: unit test
env:
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
run: |
export KUBECONFIG=/home/runner/.kube/kind-config-fx-test
DEBUG=true go test -v ./container_runtimes/... ./deploy/...
export KUBECONFIG="$(kind get kubeconfig-path)"
DEBUG=true go test -v ./...
- name: build fx
run: |
make build
- name: lint
run: |
export GOBIN=$(go env GOPATH)/bin
export PATH=$PATH:$GOBIN
go get -u github.com/golangci/golangci-lint/cmd/golangci-lint
golangci-lint run
- name: test fx cli
run: |
echo $KUBECONFIG
@@ -64,11 +51,12 @@ jobs:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
run: |
export KUBECONFIG=${HOME}/.kube/aks
echo ${AKS_KUBECONFIG} | base64 -d > $KUBECONFIG
DEBUG=true ./build/fx up -n hello -p 12345 examples/functions/JavaScript/func.js
./build/fx down hello
rm ${KUBECONFIG}
echo "skip since aks environment not ready yet"
# export KUBECONFIG=${HOME}/.kube/aks
# echo ${AKS_KUBECONFIG} | base64 -d > $KUBECONFIG
# DEBUG=true ./build/fx up -n hello -p 12345 examples/functions/JavaScript/func.js
# ./build/fx down hello
# rm ${KUBECONFIG}
Release:
runs-on: ${{ matrix.os }}
needs: [Test]

View File

@@ -1,5 +1,7 @@
OUTPUT_DIR=./build
DIST_DIR=./dist
OUTPUT_DIR ?=./build
DIST_DIR ?=./dist
DOCKER_REMOTE_HOST_ADDR ?= "127.0.0.1"
DOCKER_REMOTE_HOST_USER ?= $(whoami)
lint:
golangci-lint run
@@ -7,6 +9,9 @@ lint:
generate:
packr
b:
go build -o ${OUTPUT_DIR}/fx fx.go
build:
go build -o ${OUTPUT_DIR}/fx fx.go
@@ -23,8 +28,11 @@ clean:
unit-test:
./scripts/coverage.sh
cli-test-ci:
./scripts/test_cli.sh 'js'
cli-test:
./scripts/test_cli.sh
./scripts/test_cli.sh 'js rb py go php java d rs'
http-test:
./scripts/http_test.sh
@@ -32,3 +40,27 @@ http-test:
zip:
zip -r images.zip images/
.PHONY: test build start list clean generate
start_docker_infra:
docker build -t fx-docker-infra -f test/Dockerfile ./test
docker run --rm --name fx-docker-infra -p 2222:22 -v /var/run/docker.sock:/var/run/docker.sock -d fx-docker-infra
test_docker_infra:
CICD=true SSH_PORT=2222 SSH_KEY_FILE=./test/id_rsa ./build/fx infra create --name docker-local -t docker --host root@127.0.0.1
stop_docker_infra:
docker stop fx-docker-infra
start_k3s_infra:
multipass launch --name k3s-master --cpus 1 --mem 512M --disk 3G --cloud-init ./test/k3s/ssh-cloud-init.yaml
multipass launch --name k3s-worker1 --cpus 1 --mem 512M --disk 3G --cloud-init ./test/k3s/ssh-cloud-init.yaml
multipass launch --name k3s-worker2 --cpus 1 --mem 512M --disk 3G --cloud-init ./test/k3s/ssh-cloud-init.yaml
test_k3s_infra:
./scripts/test_k3s_infra.sh
stop_k3s_infra:
multipass delete k3s-master
multipass delete k3s-worker1
multipass delete k3s-worker2
multipass purge

161
README.md
View File

@@ -1,10 +1,10 @@
fx
------
Poor man's function as a service.
<br/>
![ci](https://github.com/metrue/fx/workflows/ci/badge.svg)
![build](https://circleci.com/gh/metrue/fx.svg?style=svg&circle-token=bd62abac47802f8504faa4cf8db43e4f117e7cd7)
[![codecov](https://codecov.io/gh/metrue/fx/branch/master/graph/badge.svg)](https://codecov.io/gh/metrue/fx)
![CI](https://github.com/metrue/fx/workflows/ci/badge.svg)
[![CodeCov](https://codecov.io/gh/metrue/fx/branch/master/graph/badge.svg)](https://codecov.io/gh/metrue/fx)
[![Go Report Card](https://goreportcard.com/badge/github.com/metrue/fx?style=flat-square)](https://goreportcard.com/report/github.com/metrue/fx)
[![Go Doc](https://img.shields.io/badge/godoc-reference-blue.svg?style=flat-square)](http://godoc.org/github.com/metrue/fx)
![](https://img.shields.io/github/license/metrue/fx.svg)
@@ -14,12 +14,13 @@ Poor man's function as a service.
- [Introduction](#introduction)
- [Installation](#installation)
- [Usage](#usage)
- [Manage Infrastructure](#manage-infrastructure)
- [Contribute](#contribute)
## Introduction
fx is a tool to help you do Function as a Service on your own server. fx can make your stateless function a service in seconds. The most exciting thing is that you can write your functions with most programming languages.
fx is a tool to help you do Function as a Service on your own server, fx can make your stateless function a service in seconds, both Docker host and Kubernetes cluster supported. The most exciting thing is that you can write your functions with most programming languages.
Feel free hacking fx to support the languages not listed. Welcome to tweet me [@_metrue](https://twitter.com/_metrue) on Twitter, [@metrue](https://www.weibo.com/u/2165714507) on Weibo.
@@ -39,6 +40,8 @@ Feel free hacking fx to support the languages not listed. Welcome to tweet me [@
# Installation
Binaries are available for Windows, MacOS and Linux/Unix on x86. For other architectures and platforms, follow instructions to [build fx from source](#buildtest).
* MacOS
```
@@ -58,9 +61,9 @@ curl -o- https://raw.githubusercontent.com/metrue/fx/master/scripts/install.sh |
curl -o- https://raw.githubusercontent.com/metrue/fx/master/scripts/install.sh | sudo bash
```
fx will be installed into /usr/local/bin, sometimes you may need `source ~/.zshrc` or `source ~/.bashrc` to make fx available in `$PAHT`.
fx will be installed into /usr/local/bin, sometimes you may need `source ~/.zshrc` or `source ~/.bashrc` to make fx available in `$PATH`.
* Window
* Windows
You can go the release page to [download](https://github.com/metrue/fx/releases) fx manually;
@@ -76,16 +79,16 @@ USAGE:
fx [global options] command [command options] [arguments...]
VERSION:
0.6.0
0.8.7
COMMANDS:
infra manage infrastructure of fx
image manage image of service
doctor health check for fx
up deploy a function or a group of functions
infra manage infrastructure
up deploy a function
down destroy a service
list, ls list deployed services
call run a function instantly
image manage image of service
doctor health check for fx
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
@@ -93,99 +96,36 @@ GLOBAL OPTIONS:
--version, -v print the version
```
1. List your current machines and activate you machine
```shell
$ fx infra ls # list machines
{
"localhost": {
"Host": "localhost",
"User": "",
"Password": "",
"Enabled": true,
"Provisioned": false
}
}
$ fx infra activate localhost # activate 'localhost'
2019/08/10 13:21:20 info Provision:pull python Docker base iamge: ✓
2019/08/10 13:21:21 info Provision:pull d Docker base image: ✓
2019/08/10 13:21:23 info Provision:pull java Docker base image: ✓
2019/08/10 13:21:28 info Provision:pull julia Docker base image: ✓
2019/08/10 13:21:31 info Provision:pull node Docker base image: ✓
2019/08/10 13:22:09 info Provision:pull go Docker base image: ✓
2019/08/10 13:22:09 info provision machine localhost: ✓
2019/08/10 13:22:09 info enble machine localhost: ✓
```
It may take seconds since `fx` needs to download some basic resources
*Note* you can add a remote host as fx machine also,
```
$ fx infra add --name my_aws_vm --host 13.121.202.227 --user root --password yourpassword
$ fx infra list
{
"my_aws_vm": {
"Host": "13.121.202.227",
"User": "root",
"Password": "yourpassword",
"Enabled": false,
"Provisioned": false
},
"localhost": {
"Host": "localhost",
"User": "",
"Password": "",
"Enabled": true,
"Provisioned": true
}
}
$ fx infra activate my_aws_vm
### Deploy your function to Docker
```
then your function will be deployed onto remote host also.
$ fx up --name hello-fx ./examples/functions/JavaScript/func.js
2. Write a function
You can check out [examples](https://github.com/metrue/fx/tree/master/examples/functions) for reference. Let's write a function as an example, it calculates the sum of two numbers then returns:
```js
module.exports = (ctx) => {
ctx.body = 'hello world'
}
```
Then save it to a file `func.js`.
3. Deploy your function as a service
Give your service a port with `--port`, and name with `--name`, heath checking with `--healthcheck` if you want.
```shell
$ fx up -name fx_service_name -p 10001 --healthcheck func.js
2019/08/10 13:26:37 info Pack Service: ✓
2019/08/10 13:26:39 info Build Service: ✓
2019/08/10 13:26:39 info Run Service: ✓
2019/08/10 13:26:39 info Service (fx_service_name) is running on: 0.0.0.0:10001
2019/08/10 13:26:39 info up function fx_service_name(func.js) to machine localhost: ✓
+------------------------------------------------------------------+-----------+---------------+
| ID | NAME | ENDPOINT |
+------------------------------------------------------------------+-----------+---------------+
| 5b24d36608ee392c937a61a530805f74551ddec304aea3aca2ffa0fabcf98cf3 | /hello-fx | 0.0.0.0:58328 |
+------------------------------------------------------------------+-----------+---------------+
```
if you want see what the source code of your service looks like, you can export it into a dirctory,
### Deploy your function to Kubernetes
```shell
$ fx image export -o <path of dir> func.js
2019/09/25 19:31:19 info exported to <path of dir>: ✓
```
$ KUBECONFIG=~/.kube/config ./build/fx up examples/functions/JavaScript/func.js --name hello-fx
+-------------------------------+------+----------------+
| ID | NAME | ENDPOINT |
+----+--------------------------+-----------------------+
| 5b24d36608ee392c937a | hello-fx | 10.0.242.75:80 |
+------------------------+-------------+----------------+
```
4. Test your service
### Test your service
then you can test your service:
```shell
$ curl -v 0.0.0.0:10001
$ curl -v 0.0.0.0:58328
GET / HTTP/1.1
@@ -207,34 +147,32 @@ hello world
```
## Docker
## Manage Infrastructure
TODO
**fx** is originally designed to turn a function into a runnable Docker container in a easiest way, on a host with Docker running, you can just deploy your function with `fx up` command, and now **fx** supports deploy function to be a service onto Kubernetes cluster infrasture, and we encourage you to do that other than on bare Docker environment, there are lots of advantage to run your function on Kubernetes like self-healing, load balancing, easy horizontal scaling, etc. It's pretty simple to deploy your function onto Kubernetes with **fx**, you just set KUBECONFIG in your enviroment.
## Kubernetes
By default. **fx** use localhost as target infrastructure to run your service, and you can also setup your remote virtual machines as **fx**'s infrastructure and deploy your functions onto it.
**fx** supports deploy function to be a service onto Kubernetes cluster infrasture, and we encourage you to do that other than on bare Docker environment, there are lots of advantage to run your function on Kubernetes like self-healing, load balancing, easy horizontal scaling, etc. It's pretty simple to deploy your function onto Kubernetes with **fx**, you just set KUBECONFIG in your enviroment.
### `fx infra create`
You can create types (docker and k8s) of infrastructures for **fx** to deploy functions
```shell
KUBECONFIG=<Your KUBECONFIG> fx deploy -n fx-service-abc_js -p 12349 examples/functions/JavaScript/func.js # function will be deploy to your Kubernetes cluster and expose a IP address of your loadbalencer
$ fx infra create --name infra_us --type docker --host <user>@<ip> ## create docker type infrasture on <ip>
$ fx infra create --name infra_bj --type k8s --master <user>@<ip> --agents '<user1>@<ip1>,<user2>@<ip2>' ## create k8s type infrasture use <ip> as master node, and <ip1> and <ip2> as agents nodes
```
or
### `fx infra use`
To use a infrastructure, you can use `fx infra use` command to activate it.
```shell
$ export KUBECONFIG=<Your KUBECONFIG>
$ fx deploy -n fx-service-abc_js -p 12349 examples/functions/JavaScript/func.js # function will be deploy to your Kubernetes cluster and expose a IP address of your loadbalencer
fx infra use <infrastructure name>
```
* Local Kubernetes Cluster
and you can list your infrastructure with `fx infra list`
Docker for Mac and Docker for Windows already support Kubernetes with single node cluster, we can use it directly, and the default `KUBECONFIG` is `~/.kube/config`.
```shell
$ export KUBECONFIG=~/.kube/config # then fx will take the config to deloy function
```
if you have multiple Kubernetes clusters configured, you have to set context correctly. FYI [configure-access-multiple-clusters](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/)
## Use Public Cloud Kubernetes Service as infrastructure to run your functions
* Azure Kubernetes Service (AKS)
@@ -269,6 +207,12 @@ But we would suggest you run `kubectl config current-context` to check if the cu
* Google Kubernetes Engine (GKET)
TODO
* Setup your own Kubernetes cluster
```shell
fx infra create --type k3s --name fx-cluster-1 --master root@123.11.2.3 --agents 'root@1.1.1.1,root@2.2.2.2'
```
## Contribute
fx uses [Project](https://github.com/metrue/fx/projects/4) to manage the development.
@@ -278,6 +222,7 @@ fx uses [Project](https://github.com/metrue/fx/projects/4) to manage the develop
Docker: make sure [Docker](https://docs.docker.com/engine/installation/) installed and running on your server.
<a name="buildtest"></a>
#### Build & Test
```

View File

@@ -1,213 +1,196 @@
package config
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
"os/user"
"path"
"github.com/spf13/viper"
dockerInfra "github.com/metrue/fx/infra/docker"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
"github.com/mitchellh/go-homedir"
)
// Configer interface
// Configer manage fx config
type Configer interface {
GetMachine(name string) (Host, error)
AddMachine(name string, host Host) error
RemoveHost(name string) error
ListActiveMachines() (map[string]Host, error)
ListMachines() (map[string]Host, error)
EnableMachine(name string) error
DisableMachine(name string) error
UpdateProvisionedStatus(name string, ok bool) error
GetCurrentCloud() ([]byte, error)
GetCurrentCloudType() (string, error)
GetKubeConfig() (string, error)
UseCloud(name string) error
View() ([]byte, error)
AddCloud(name string, meta []byte) error
}
// Config config of fx
type Config struct {
dir string
configFile string
container *Container
}
// New create a config
func New(dir string) *Config {
return &Config{dir: dir}
const defaultFxConfig = "~/.fx/config.yml"
// LoadDefault load default config
func LoadDefault() (*Config, error) {
configFile, err := homedir.Expand(defaultFxConfig)
if err != nil {
return nil, err
}
if os.Getenv("FX_CONFIG") != "" {
configFile = os.Getenv("FX_CONFIG")
}
if _, err := os.Stat(configFile); os.IsNotExist(err) {
if err := utils.EnsureFile(configFile); err != nil {
return nil, err
}
}
return load(configFile)
}
// Init config
func (c *Config) Init() error {
if err := os.MkdirAll(c.dir, os.ModePerm); err != nil {
func load(configFile string) (*Config, error) {
container, err := CreateContainer(configFile)
if err != nil {
return nil, err
}
config := &Config{
configFile: configFile,
container: container,
}
if container.get("clouds") == nil {
if err := config.writeDefaultConfig(); err != nil {
return nil, err
}
}
return config, nil
}
// Load config
func Load(configFile string) (*Config, error) {
if configFile == "" {
return nil, fmt.Errorf("invalid config file")
}
if _, err := os.Stat(configFile); os.IsNotExist(err) {
if err := utils.EnsureFile(configFile); err != nil {
return nil, err
}
}
return load(configFile)
}
// AddCloud add k8s cloud
func (c *Config) AddCloud(name string, meta []byte) error {
var cloudMeta map[string]interface{}
if err := json.Unmarshal(meta, &cloudMeta); err != nil {
return err
}
ext := "yaml"
name := "config"
viper.SetConfigType(ext)
viper.SetConfigName(name)
viper.AddConfigPath(c.dir)
cloudType, ok := cloudMeta["type"].(string)
if !ok || cloudType == "" {
return fmt.Errorf("unknown cloud type")
}
// detect if file exists
configFilePath := path.Join(c.dir, name+"."+ext)
if _, err := os.Stat(configFilePath); os.IsNotExist(err) {
fd, err := os.Create(configFilePath)
if err != nil {
if cloudType == types.CloudTypeK8S {
dir := path.Dir(c.configFile)
kubecfg := path.Join(dir, name+".kubeconfig")
if err := utils.EnsureFile(kubecfg); err != nil {
return err
}
fd.Close()
localhost := Host{
Host: "localhost",
Password: "",
User: "",
Enabled: true,
Provisioned: false,
config, ok := cloudMeta["config"].(string)
if !ok {
return fmt.Errorf("invalid k8s config")
}
if err := ioutil.WriteFile(kubecfg, []byte(config), 0666); err != nil {
return err
}
viper.Set("hosts", map[string]Host{"localhost": localhost})
return viper.WriteConfig()
}
if err := viper.ReadInConfig(); err != nil {
return fmt.Errorf("fatal error config file: %s", err)
if err := c.container.set("clouds."+name, cloudMeta); err != nil {
return err
}
return nil
}
// GetMachine get host by name
func (c *Config) GetMachine(name string) (Host, error) {
var hosts map[string]Host
if err := viper.UnmarshalKey("hosts", &hosts); err != nil {
return Host{}, err
// UseCloud set cloud instance with name as current context
func (c *Config) UseCloud(name string) error {
if name == "" {
return fmt.Errorf("could not use empty name")
}
host, ok := hosts[name]
if c.container.get("clouds."+name) == nil {
return fmt.Errorf("no such cloud with name: %s", name)
}
return c.container.set("current_cloud", name)
}
// View view current config
func (c *Config) View() ([]byte, error) {
return ioutil.ReadFile(c.configFile)
}
// GetCurrentCloud get current using cloud's meta
func (c *Config) GetCurrentCloud() ([]byte, error) {
name, ok := c.container.get("current_cloud").(string)
if !ok {
return Host{}, fmt.Errorf("no such host %v", name)
return nil, fmt.Errorf("no active cloud")
}
return host, nil
meta := c.container.get("clouds." + name)
if meta == nil {
return nil, fmt.Errorf("invalid config")
}
return json.Marshal(meta)
}
// ListActiveMachines list enabled machines
func (c *Config) ListActiveMachines() (map[string]Host, error) {
hosts, err := c.ListMachines()
if err != nil {
return map[string]Host{}, err
// GetCurrentCloudType get current cloud type
func (c *Config) GetCurrentCloudType() (string, error) {
name, ok := c.container.get("current_cloud").(string)
if !ok {
return "", fmt.Errorf("no active cloud")
}
lst := map[string]Host{}
for name, h := range hosts {
if h.Enabled {
lst[name] = h
}
}
return lst, nil
return c.container.get("clouds." + name + ".type").(string), nil
}
// AddMachine add host
func (c *Config) AddMachine(name string, host Host) error {
if !viper.IsSet("hosts") {
viper.Set("hosts", map[string]Host{})
// GetKubeConfig get kubeconfig
func (c *Config) GetKubeConfig() (string, error) {
name, ok := c.container.get("current_cloud").(string)
if !ok {
return "", fmt.Errorf("no active cloud")
}
hosts, err := c.ListMachines()
if err != nil {
return err
}
hosts[name] = host
viper.Set("hosts", hosts)
return viper.WriteConfig()
dir := path.Dir(c.configFile)
kubecfg := path.Join(dir, name+".kubeconfig")
return kubecfg, nil
}
// RemoveHost remote a host
func (c *Config) RemoveHost(name string) error {
hosts, err := c.ListMachines()
func (c *Config) writeDefaultConfig() error {
me, err := user.Current()
if err != nil {
return err
}
if len(hosts) == 1 {
return fmt.Errorf("only one host left now, at least one host required by fx")
defaultCloud := &dockerInfra.Cloud{
IP: "127.0.0.1",
User: me.Username,
Name: "default",
Type: types.CloudTypeDocker,
}
if _, ok := hosts[name]; ok {
delete(hosts, name)
viper.Set("hosts", hosts)
return viper.WriteConfig()
}
return fmt.Errorf("no such host %s", name)
}
// ListMachines list hosts
func (c *Config) ListMachines() (map[string]Host, error) {
var hosts map[string]Host
if err := viper.UnmarshalKey("hosts", &hosts); err != nil {
return nil, err
}
return hosts, nil
}
// EnableMachine enable a machine, after machine enabled, function will be deployed onto it when ever `fx up` invoked
func (c *Config) EnableMachine(name string) error {
host, err := c.GetMachine(name)
meta, err := defaultCloud.Dump()
if err != nil {
return err
}
host.Enabled = true
if !viper.IsSet("hosts") {
viper.Set("hosts", map[string]Host{})
}
hosts, err := c.ListMachines()
if err != nil {
if err := c.container.set("clouds", map[string]interface{}{}); err != nil {
return err
}
hosts[name] = host
viper.Set("hosts", hosts)
return viper.WriteConfig()
if err := c.AddCloud("default", meta); err != nil {
return err
}
return c.UseCloud("default")
}
// DisableMachine disable a machine, after machine disabled, function will not be deployed onto it
func (c *Config) DisableMachine(name string) error {
host, err := c.GetMachine(name)
if err != nil {
return err
}
host.Enabled = false
if !viper.IsSet("hosts") {
viper.Set("hosts", map[string]Host{})
}
hosts, err := c.ListMachines()
if err != nil {
return err
}
hosts[name] = host
viper.Set("hosts", hosts)
return viper.WriteConfig()
}
// UpdateProvisionedStatus update provisioned status
func (c *Config) UpdateProvisionedStatus(name string, ok bool) error {
host, err := c.GetMachine(name)
if err != nil {
return err
}
host.Provisioned = ok
if !viper.IsSet("hosts") {
viper.Set("hosts", map[string]Host{})
}
hosts, err := c.ListMachines()
if err != nil {
return err
}
hosts[name] = host
viper.Set("hosts", hosts)
return viper.WriteConfig()
}
// IsMachineProvisioned check if machine provisioned
func (c *Config) IsMachineProvisioned(name string) bool {
host, err := c.GetMachine(name)
if err != nil {
return false
}
return host.Provisioned
}
var (
_ Configer = &Config{}
)

View File

@@ -1,98 +1,111 @@
package config
import (
"encoding/json"
"fmt"
"os"
"os/user"
"reflect"
"testing"
k8sInfra "github.com/metrue/fx/infra/k8s"
"github.com/metrue/fx/types"
)
func TestConfig(t *testing.T) {
configPath := "/tmp/.fx"
configPath := "./tmp/config.yml"
defer func() {
if err := os.RemoveAll(configPath); err != nil {
if err := os.RemoveAll("./tmp/config.yml"); err != nil {
t.Fatal(err)
}
}()
c := New(configPath)
if err := c.Init(); err != nil {
t.Fatal(err)
}
hosts, err := c.ListMachines()
// default cloud
c, err := Load(configPath)
if err != nil {
t.Fatal(err)
}
if len(hosts) != 1 {
t.Fatalf("should have localhost as default machine")
}
host := hosts["localhost"]
if !reflect.DeepEqual(host, Host{Host: "localhost", Enabled: true}) {
t.Fatalf("should get %v but got %v", Host{Host: "localhost"}, host)
}
name := "remote-a"
h := Host{
Host: "192.168.1.1",
User: "user-a",
Password: "password-a",
Enabled: false,
}
if err := c.AddMachine(name, h); err != nil {
t.Fatal(err)
}
hosts, err = c.ListMachines()
defaultMeta, err := c.GetCurrentCloud()
if err != nil {
t.Fatal(err)
}
if len(hosts) != 2 {
t.Fatalf("should have %d machines now, but got %d", 2, len(hosts))
var cloudMeta map[string]string
if err := json.Unmarshal(defaultMeta, &cloudMeta); err != nil {
t.Fatal(err)
}
if cloudMeta["ip"] != "127.0.0.1" {
t.Fatalf("should get %s but got %s", "127.0.0.1", cloudMeta["ip"])
}
lst, err := c.ListActiveMachines()
me, _ := user.Current()
if cloudMeta["user"] != me.Username {
t.Fatalf("should get %s but got %s", me.Username, cloudMeta["user"])
}
if cloudMeta["type"] != types.CloudTypeDocker {
t.Fatalf("should get %s but got %s", types.CloudTypeDocker, cloudMeta["type"])
}
if cloudMeta["name"] != "default" {
t.Fatalf("should get %s but got %s", "default", cloudMeta["name"])
}
// add k8s cloud
kCloud := k8sInfra.Cloud{
Type: types.CloudTypeK8S,
Config: "sample kubeconfg",
Token: "",
URL: "",
Nodes: map[string]k8sInfra.Noder{
"master-node": &k8sInfra.Node{
IP: "1.1.1.1",
User: "user-1",
Type: "k3s-master",
Name: "master-node",
},
"agent-node-1": &k8sInfra.Node{
IP: "1.1.1.1",
User: "user-1",
Type: "k3s-agent",
Name: "agent-node-1",
},
},
}
kMeta, err := kCloud.Dump()
if err != nil {
t.Fatal(err)
}
if len(lst) != 1 {
t.Fatalf("should only have %d machine enabled, but got %d", 1, len(lst))
}
if err := c.EnableMachine(name); err != nil {
kName := "k8s-1"
if err := c.AddCloud(kName, kMeta); err != nil {
t.Fatal(err)
}
lst, err = c.ListActiveMachines()
curMeta, err := c.GetCurrentCloud()
if err != nil {
t.Fatal(err)
}
if len(lst) != 2 {
t.Fatalf("should only have %d machine enabled, but got %d", 2, len(lst))
if !reflect.DeepEqual(curMeta, defaultMeta) {
t.Fatalf("should get %v but got %v", defaultMeta, curMeta)
}
h.Enabled = true
if !reflect.DeepEqual(lst[name], h) {
t.Fatalf("should get %v but got %v", h, lst[name])
if err := c.UseCloud("cloud-not-existed"); err == nil {
t.Fatalf("should get error when there is not given cloud name")
}
if lst[name].Provisioned != false {
t.Fatalf("should get %v but got %v", false, lst[name].Provisioned)
}
if err := c.UpdateProvisionedStatus(name, true); err != nil {
if err := c.UseCloud(kName); err != nil {
t.Fatal(err)
}
updatedHost, err := c.GetMachine(name)
curMeta, err = c.GetCurrentCloud()
if err != nil {
t.Fatal(err)
}
if updatedHost.Provisioned != true {
t.Fatalf("should get %v but got %v", true, updatedHost.Provisioned)
if reflect.DeepEqual(curMeta, kMeta) {
t.Fatalf("should get %v but got %v", kMeta, curMeta)
}
body, err := c.View()
if err != nil {
t.Fatal(err)
}
fmt.Println(string(body))
}

73
config/container.go Normal file
View File

@@ -0,0 +1,73 @@
package config
import (
"fmt"
"path/filepath"
"strings"
"sync"
"github.com/metrue/fx/utils"
"github.com/spf13/viper"
)
// Container config container, wrap viper as a key-value store with lock
type Container struct {
mux sync.Mutex
store string
}
// CreateContainer new a container
func CreateContainer(storeFile string) (*Container, error) {
if err := utils.EnsureFile(storeFile); err != nil {
return nil, err
}
dir := filepath.Dir(storeFile)
ext := filepath.Ext(storeFile)
name := filepath.Base(storeFile)
viper.AddConfigPath(dir)
viper.SetConfigName(strings.Replace(name, ext, "", 1))
viper.SetConfigType(strings.Replace(ext, ".", "", 1))
if err := viper.ReadInConfig(); err != nil {
return nil, err
}
return &Container{
store: storeFile,
}, nil
}
func (c *Container) set(key string, value interface{}) error {
c.mux.Lock()
defer c.mux.Unlock()
if key == "" {
return fmt.Errorf("empty key not allowed")
}
keys := strings.Split(key, ".")
if len(keys) == 1 {
viper.Set(key, value)
} else {
prePath := keys[0]
for i := 1; i < len(keys)-2; i++ {
prePath += "." + keys[i]
}
if viper.Get(prePath) == nil {
return fmt.Errorf("%s not existed", prePath)
}
viper.Set(key, value)
}
// viper.Set(key, value)
if err := viper.WriteConfig(); err != nil {
return err
}
return nil
}
func (c *Container) get(key string) interface{} {
c.mux.Lock()
defer c.mux.Unlock()
return viper.Get(key)
}

84
config/container_test.go Normal file
View File

@@ -0,0 +1,84 @@
package config
import (
"os"
"testing"
)
func TestContainer(t *testing.T) {
configPath := "./tmp/container.yml"
defer func() {
if err := os.RemoveAll("./tmp/container.yml"); err != nil {
t.Fatal(err)
}
}()
c, err := CreateContainer(configPath)
if err != nil {
t.Fatal(err)
}
if err := c.set("", ""); err == nil {
t.Fatalf("should get error when key is empty")
}
if c.get("1") != nil {
t.Fatalf("should get %v but got %v", nil, c.get("key"))
}
// create
if err := c.set("1", "1"); err != nil {
t.Fatal(err)
}
// read
if c.get("1").(string) != "1" {
t.Fatalf("should get %s but got %s", "val-1", c.get("key"))
}
// invaliad set
if err := c.set("1.1", "1.1"); err != nil {
t.Fatal(err)
}
if c.get("1.1").(string) != "1.1" {
t.Fatalf("should get 1.1 but got %s", c.get("1.1"))
}
// update
if err := c.set("1", "11"); err != nil {
t.Fatal(err)
}
if c.get("1").(string) != "11" {
t.Fatalf("should get 11 but got %s", c.get("1").(string))
}
// nested set
if err := c.set("2.2.2.2", "2222"); err == nil {
t.Fatalf("should throw error since 2.2.2 not ready yet")
}
if err := c.set("2", map[string]interface{}{
"2": map[string]interface{}{
"2": "2",
},
}); err != nil {
t.Fatal(err)
}
if c.get("2.2.2").(string) != "2" {
t.Fatalf("should get 2 but got %s", c.get("2.2.2"))
}
if err := c.set("2.2.2.2", "2222"); err != nil {
t.Fatal(err)
}
if c.get("2.2.2.2").(string) != "2222" {
t.Fatalf("should get 2222 but got %s", c.get("2.2.2.2"))
}
if err := c.set("2.2.2.1", "1111"); err != nil {
t.Fatal(err)
}
if c.get("2.2.2.1").(string) != "1111" {
t.Fatalf("should get 1111 but got %s", c.get("2.2.2.1"))
}
}

View File

@@ -1,40 +0,0 @@
package config
// Host host entity
type Host struct {
Host string
User string
Password string
Enabled bool
Provisioned bool
}
// NewHost new a host
func NewHost(addr, user, password string) Host {
return Host{
Host: addr,
User: user,
Password: password,
Enabled: false,
Provisioned: false,
}
}
// Valid if host is valid
func (h Host) Valid() bool {
// TODO stronger check
return h.Host != ""
}
// IsLocal if host is localhost
func (h Host) IsLocal() bool {
if !h.Valid() {
return false
}
return h.Host == "127.0.0.1" || h.Host == "localhost"
}
// IsRemote is host is remote
func (h Host) IsRemote() bool {
return !h.IsLocal()
}

View File

@@ -1,149 +0,0 @@
// Code generated by MockGen. DO NOT EDIT.
// Source: ./config.go
// Package mock_config is a generated GoMock package.
package mock_config
import (
gomock "github.com/golang/mock/gomock"
config "github.com/metrue/fx/config"
reflect "reflect"
)
// MockConfiger is a mock of Configer interface
type MockConfiger struct {
ctrl *gomock.Controller
recorder *MockConfigerMockRecorder
}
// MockConfigerMockRecorder is the mock recorder for MockConfiger
type MockConfigerMockRecorder struct {
mock *MockConfiger
}
// NewMockConfiger creates a new mock instance
func NewMockConfiger(ctrl *gomock.Controller) *MockConfiger {
mock := &MockConfiger{ctrl: ctrl}
mock.recorder = &MockConfigerMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockConfiger) EXPECT() *MockConfigerMockRecorder {
return m.recorder
}
// GetMachine mocks base method
func (m *MockConfiger) GetMachine(name string) (config.Host, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetMachine", name)
ret0, _ := ret[0].(config.Host)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetMachine indicates an expected call of GetMachine
func (mr *MockConfigerMockRecorder) GetMachine(name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetMachine", reflect.TypeOf((*MockConfiger)(nil).GetMachine), name)
}
// AddMachine mocks base method
func (m *MockConfiger) AddMachine(name string, host config.Host) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "AddMachine", name, host)
ret0, _ := ret[0].(error)
return ret0
}
// AddMachine indicates an expected call of AddMachine
func (mr *MockConfigerMockRecorder) AddMachine(name, host interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "AddMachine", reflect.TypeOf((*MockConfiger)(nil).AddMachine), name, host)
}
// RemoveHost mocks base method
func (m *MockConfiger) RemoveHost(name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "RemoveHost", name)
ret0, _ := ret[0].(error)
return ret0
}
// RemoveHost indicates an expected call of RemoveHost
func (mr *MockConfigerMockRecorder) RemoveHost(name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "RemoveHost", reflect.TypeOf((*MockConfiger)(nil).RemoveHost), name)
}
// ListActiveMachines mocks base method
func (m *MockConfiger) ListActiveMachines() (map[string]config.Host, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "ListActiveMachines")
ret0, _ := ret[0].(map[string]config.Host)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// ListActiveMachines indicates an expected call of ListActiveMachines
func (mr *MockConfigerMockRecorder) ListActiveMachines() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ListActiveMachines", reflect.TypeOf((*MockConfiger)(nil).ListActiveMachines))
}
// ListMachines mocks base method
func (m *MockConfiger) ListMachines() (map[string]config.Host, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "ListMachines")
ret0, _ := ret[0].(map[string]config.Host)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// ListMachines indicates an expected call of ListMachines
func (mr *MockConfigerMockRecorder) ListMachines() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ListMachines", reflect.TypeOf((*MockConfiger)(nil).ListMachines))
}
// EnableMachine mocks base method
func (m *MockConfiger) EnableMachine(name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "EnableMachine", name)
ret0, _ := ret[0].(error)
return ret0
}
// EnableMachine indicates an expected call of EnableMachine
func (mr *MockConfigerMockRecorder) EnableMachine(name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "EnableMachine", reflect.TypeOf((*MockConfiger)(nil).EnableMachine), name)
}
// DisableMachine mocks base method
func (m *MockConfiger) DisableMachine(name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "DisableMachine", name)
ret0, _ := ret[0].(error)
return ret0
}
// DisableMachine indicates an expected call of DisableMachine
func (mr *MockConfigerMockRecorder) DisableMachine(name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "DisableMachine", reflect.TypeOf((*MockConfiger)(nil).DisableMachine), name)
}
// UpdateProvisionedStatus mocks base method
func (m *MockConfiger) UpdateProvisionedStatus(name string, ok bool) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "UpdateProvisionedStatus", name, ok)
ret0, _ := ret[0].(error)
return ret0
}
// UpdateProvisionedStatus indicates an expected call of UpdateProvisionedStatus
func (mr *MockConfigerMockRecorder) UpdateProvisionedStatus(name, ok interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "UpdateProvisionedStatus", reflect.TypeOf((*MockConfiger)(nil).UpdateProvisionedStatus), name, ok)
}

View File

@@ -1,19 +1,32 @@
package api
import (
"bufio"
"bytes"
"context"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/apex/log"
dockerTypes "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
dockerTypesContainer "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network"
"github.com/docker/go-connections/nat"
"github.com/google/go-querystring/query"
"github.com/google/uuid"
containerruntimes "github.com/metrue/fx/container_runtimes"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
"github.com/pkg/errors"
)
// API interact with dockerd http api
@@ -24,27 +37,29 @@ type API struct {
// Create a API
func Create(host string, port string) (*API, error) {
version, err := utils.DockerVersion(host, port)
addr := host + ":" + port
v, err := version(addr)
if err != nil {
return nil, err
}
endpoint := fmt.Sprintf("http://%s:%s/v%s", host, port, version)
endpoint := fmt.Sprintf("http://%s:%s/v%s", host, port, v)
return &API{
endpoint: endpoint,
version: version,
version: v,
}, nil
}
// MustCreate a api object, panic if not
func MustCreate(host string, port string) *API {
version, err := utils.DockerVersion(host, port)
addr := host + ":" + port
v, err := version(addr)
if err != nil {
panic(err)
}
endpoint := fmt.Sprintf("http://%s:%s/v%s", host, port, version)
endpoint := fmt.Sprintf("http://%s:%s/v%s", host, port, v)
return &API{
endpoint: endpoint,
version: version,
version: v,
}
}
@@ -116,8 +131,47 @@ func (api *API) post(path string, body []byte, expectStatus int, v interface{})
return nil
}
// List list service
func (api *API) list(name string) ([]types.Service, error) {
// Version get version of docker engine
func (api *API) Version(ctx context.Context) (string, error) {
return version(api.endpoint)
}
func version(endpoint string) (string, error) {
path := endpoint + "/version"
if !strings.HasPrefix(path, "http") {
path = "http://" + path
}
req, err := http.NewRequest("GET", path, nil)
if err != nil {
return "", err
}
client := &http.Client{Timeout: 20 * time.Second}
resp, err := client.Do(req)
if err != nil {
return "", err
}
if resp.StatusCode != 200 {
return "", fmt.Errorf("request %s failed: %d - %s", path, resp.StatusCode, resp.Status)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return "", err
}
var res dockerTypes.Version
err = json.Unmarshal(body, &res)
if err != nil {
return "", err
}
return res.APIVersion, nil
}
// ListContainer list service
func (api *API) ListContainer(ctx context.Context, name string) ([]types.Service, error) {
if name != "" {
info, err := api.inspect(name)
if err != nil {
@@ -141,7 +195,7 @@ func (api *API) list(name string) ([]types.Service, error) {
}
type filterItem struct {
Status []string `json:"url,omitempty"`
Status []string `json:"status,omitempty"`
Label []string `json:"label,omitempty"`
Name []string `json:"name,omitempty"`
}
@@ -193,3 +247,223 @@ func (api *API) list(name string) ([]types.Service, error) {
return services, nil
}
// BuildImage build image
func (api *API) BuildImage(ctx context.Context, workdir string, name string) error {
tarDir, err := ioutil.TempDir("/tmp", "fx-tar")
if err != nil {
return err
}
defer os.RemoveAll(tarDir)
imageID := uuid.New().String()
tarFilePath := filepath.Join(tarDir, fmt.Sprintf("%s.tar", imageID))
if err := utils.TarDir(workdir, tarFilePath); err != nil {
return err
}
dockerBuildContext, err := os.Open(tarFilePath)
if err != nil {
return err
}
defer dockerBuildContext.Close()
type buildQuery struct {
Labels string `url:"labels,omitempty"`
Tags string `url:"t,omitempty"`
Dockerfile string `url:"dockerfile,omitempty"`
}
// Apply default labels
labelsJSON, _ := json.Marshal(map[string]string{
"belong-to": "fx",
})
q := buildQuery{
Labels: string(labelsJSON),
Dockerfile: "Dockerfile",
}
qs, err := query.Values(q)
if err != nil {
return err
}
qs.Add("t", name)
qs.Add("t", imageID)
path := "/build"
url := fmt.Sprintf("%s%s?%s", api.endpoint, path, qs.Encode())
req, err := http.NewRequest("POST", url, dockerBuildContext)
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/x-tar")
client := &http.Client{Timeout: 600 * time.Second}
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
scanner := bufio.NewScanner(resp.Body)
for scanner.Scan() {
if os.Getenv("DEBUG") != "" {
log.Infof(scanner.Text())
}
}
if err := scanner.Err(); err != nil {
return err
}
return nil
}
// PushImage push a image
func (api *API) PushImage(ctx context.Context, name string) (string, error) {
return "", nil
}
// InspectImage inspect image
func (api *API) InspectImage(ctx context.Context, name string, image interface{}) error {
return nil
}
// TagImage tag image
func (api *API) TagImage(ctx context.Context, name string, tag string) error {
query := url.Values{}
query.Set("repo", name)
query.Set("tag", tag)
path := fmt.Sprintf("/images/%s/tag?%s", name, query.Encode())
url := fmt.Sprintf("%s%s", api.endpoint, path)
req, err := http.NewRequest("POST", url, nil)
if err != nil {
return err
}
client := &http.Client{Timeout: 10 * time.Second}
if _, err = client.Do(req); err != nil {
return err
}
return nil
}
// StartContainer start container
func (api *API) StartContainer(ctx context.Context, name string, image string, bindings []types.PortBinding) error {
networks, err := api.GetNetwork(fxNetworkName)
if err != nil {
return errors.Wrapf(err, "get network failed: %s", err)
}
if len(networks) == 0 {
if err := api.CreateNetwork(fxNetworkName); err != nil {
return errors.Wrapf(err, "error create network: %s", err)
}
}
networks, _ = api.GetNetwork(fxNetworkName)
endpoint := &network.EndpointSettings{
NetworkID: networks[0].ID,
}
networkConfig := &network.NetworkingConfig{
EndpointsConfig: map[string]*network.EndpointSettings{
"fx-net": endpoint,
},
}
portSet := nat.PortSet{}
portMap := nat.PortMap{}
for _, binding := range bindings {
bindings := []nat.PortBinding{
nat.PortBinding{
HostIP: types.DefaultHost,
HostPort: fmt.Sprintf("%d", binding.ServiceBindingPort),
},
}
port := nat.Port(fmt.Sprintf("%d/tcp", binding.ContainerExposePort))
portSet[port] = struct{}{}
portMap[port] = bindings
}
config := &dockerTypesContainer.Config{
Image: image,
ExposedPorts: portSet,
}
hostConfig := &dockerTypesContainer.HostConfig{
AutoRemove: true,
PortBindings: portMap,
}
req := ContainerCreateRequestPayload{
Config: config,
HostConfig: hostConfig,
NetworkingConfig: networkConfig,
}
body, err := json.Marshal(req)
if err != nil {
return errors.Wrap(err, "error mashal container create req")
}
// create container
path := fmt.Sprintf("/containers/create?name=%s", name)
var createRes container.ContainerCreateCreatedBody
if err := api.post(path, body, 201, &createRes); err != nil {
return errors.Wrap(err, "create container request failed")
}
if createRes.ID == "" {
return fmt.Errorf("container id is missing")
}
// start container
path = fmt.Sprintf("/containers/%s/start", createRes.ID)
url := fmt.Sprintf("%s%s", api.endpoint, path)
request, err := http.NewRequest("POST", url, nil)
if err != nil {
return errors.Wrap(err, "error new container create request")
}
client := &http.Client{Timeout: 20 * time.Second}
resp, err := client.Do(request)
if err != nil {
return errors.Wrap(err, "error do start container request")
}
defer resp.Body.Close()
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
if len(b) != 0 {
msg := fmt.Sprintf("start container met issue: %s", string(b))
return errors.New(msg)
}
if _, err = api.inspect(createRes.ID); err != nil {
msg := fmt.Sprintf("inspect container %s error", name)
return errors.Wrap(err, msg)
}
return nil
}
// StopContainer stop a container
func (api *API) StopContainer(ctx context.Context, name string) error {
return api.Stop(name)
}
// InspectContainer inspect container
func (api *API) InspectContainer(ctx context.Context, name string, container interface{}) error {
path := fmt.Sprintf("/containers/%s/json", name)
if err := api.get(path, "", &container); err != nil {
return err
}
return nil
}
var (
_ containerruntimes.ContainerRuntime = &API{}
)

View File

@@ -1,89 +1,31 @@
package api
import (
"context"
"os"
"testing"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/types"
"github.com/docker/docker/api/types"
)
func TestDockerHTTP(t *testing.T) {
host := config.Host{Host: "127.0.0.1"}
api, err := Create(host.Host, constants.AgentPort)
host := os.Getenv("DOCKER_ENGINE_HOST")
port := os.Getenv("DOCKER_ENGINE_PORT")
if host == "" ||
port == "" {
t.Skip("DOCKER_ENGINE_HOST and DOCKER_ENGINE_PORT required")
}
api, err := Create(host, port)
if err != nil {
t.Fatal(err)
}
serviceName := "a-test-service"
project := types.Project{
Name: serviceName,
Language: "node",
Files: []types.ProjectSourceFile{
types.ProjectSourceFile{
Path: "Dockerfile",
Body: `
FROM metrue/fx-node-base
COPY . .
EXPOSE 3000
CMD ["node", "app.js"]`,
IsHandler: false,
},
types.ProjectSourceFile{
Path: "app.js",
Body: `
const Koa = require('koa');
const bodyParser = require('koa-bodyparser');
const func = require('./fx');
const app = new Koa();
app.use(bodyParser());
app.use(ctx => {
const msg = func(ctx.request.body);
ctx.body = msg;
});
app.listen(3000);`,
IsHandler: false,
},
types.ProjectSourceFile{
Path: "fx.js",
Body: `
module.exports = (input) => {
return input.a + input.b
}
`,
IsHandler: true,
},
},
}
service, err := api.Build(project)
if err != nil {
name := "fx-agent"
var container types.ContainerJSON
if err := api.InspectContainer(context.Background(), name, &container); err != nil {
t.Fatal(err)
}
if err != nil {
t.Fatal(err)
}
if service.Name != serviceName {
t.Fatalf("should get %s but got %s", serviceName, service.Name)
}
if err := api.Run(9999, &service); err != nil {
t.Fatal(err)
}
services, err := api.list(serviceName)
if err != nil {
t.Fatal(err)
}
if len(services) != 1 {
t.Fatal("service number should be 1")
}
if err := api.Stop(serviceName); err != nil {
t.Fatal(err)
if container.Name != "/"+name {
t.Fatalf("should get %s but got %s", name, container.Name)
}
}

View File

@@ -1,60 +1,51 @@
package api
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"strings"
"time"
"github.com/apex/log"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
)
// Call function directly with given params
func (api *API) Call(file string, param string, project types.Project) error {
service, err := api.Build(project)
if err != nil {
log.Fatalf("Build Service: %v", err)
return err
}
log.Info("Build Service: \u2713")
if err := api.Run(9999, &service); err != nil {
log.Fatalf("Run Service: %v", err)
return err
}
log.Info("Run Service: \u2713")
params := utils.PairsToParams(strings.Fields(param))
body, err := json.Marshal(params)
if err != nil {
return err
}
// Wait 2 seconds for service startup
time.Sleep(time.Second * 2)
url := fmt.Sprintf("http://%s:%d", service.Host, service.Port)
r, err := http.NewRequest("POST", url, bytes.NewReader(body))
if err != nil {
return err
}
r.Header.Set("Content-Type", "application/json")
client := &http.Client{Timeout: 20 * time.Second}
resp, err := client.Do(r)
if err != nil {
log.Fatalf("Call Service: %v", err)
return err
}
buf, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalf("Call Service: %v", err)
return err
}
log.Info("Call Service: \u2713")
return utils.OutputJSON(string(buf))
return nil
// service, err := api.Build(project)
// if err != nil {
// log.Fatalf("Build Service: %v", err)
// return err
// }
// log.Info("Build Service: \u2713")
//
// if err := api.Run(9999, &service); err != nil {
// log.Fatalf("Run Service: %v", err)
// return err
// }
// log.Info("Run Service: \u2713")
//
// params := utils.PairsToParams(strings.Fields(param))
// body, err := json.Marshal(params)
// if err != nil {
// return err
// }
//
// // Wait 2 seconds for service startup
// time.Sleep(time.Second * 2)
//
// url := fmt.Sprintf("http://%s:%d", service.Host, service.Port)
// r, err := http.NewRequest("POST", url, bytes.NewReader(body))
// if err != nil {
// return err
// }
// r.Header.Set("Content-Type", "application/json")
// client := &http.Client{Timeout: 20 * time.Second}
// resp, err := client.Do(r)
// if err != nil {
// log.Fatalf("Call Service: %v", err)
// return err
// }
// buf, err := ioutil.ReadAll(resp.Body)
// if err != nil {
// log.Fatalf("Call Service: %v", err)
// return err
// }
// log.Info("Call Service: \u2713")
// return utils.OutputJSON(string(buf))
}

View File

@@ -1,22 +0,0 @@
package api
import (
"github.com/apex/log"
"github.com/metrue/fx/utils"
)
// List services
func (api *API) List(name string) error {
services, err := api.list(name)
if err != nil {
log.Fatalf("List Services: %v", err)
return err
}
for _, service := range services {
if err := utils.OutputJSON(service); err != nil {
return err
}
}
return nil
}

View File

@@ -1,35 +0,0 @@
package api
import (
"testing"
"github.com/golang/mock/gomock"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
gock "gopkg.in/h2non/gock.v1"
)
func TestNetwork(t *testing.T) {
defer gock.Off()
ctrl := gomock.NewController(t)
defer ctrl.Finish()
host := config.Host{Host: "127.0.0.1"}
api, err := Create(host.Host, constants.AgentPort)
if err != nil {
t.Fatal(err)
}
const network = "fx-net"
if err := api.CreateNetwork(network); err != nil {
t.Fatal(err)
}
nws, err := api.GetNetwork(network)
if err != nil {
t.Fatal(err)
}
if nws[0].Name != network {
t.Fatalf("should get %s but got %s", network, nws[0].Name)
}
}

View File

@@ -1,119 +1,59 @@
package api
import (
"bufio"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"os"
"path/filepath"
"time"
"github.com/apex/log"
"github.com/google/go-querystring/query"
"github.com/google/uuid"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
)
func makeTar(project types.Project, tarFilePath string) error {
dir, err := ioutil.TempDir("/tmp", "fx-build-dir")
if err != nil {
return err
}
defer os.RemoveAll(dir)
for _, file := range project.Files {
tmpfn := filepath.Join(dir, file.Path)
if err := utils.EnsureFile(tmpfn); err != nil {
return err
}
if err := ioutil.WriteFile(tmpfn, []byte(file.Body), 0666); err != nil {
return err
}
}
return utils.TarDir(dir, tarFilePath)
}
// Build build a project
func (api *API) Build(project types.Project) (types.Service, error) {
tarDir, err := ioutil.TempDir("/tmp", "fx-tar")
if err != nil {
return types.Service{}, err
}
defer os.RemoveAll(tarDir)
imageID := uuid.New().String()
tarFilePath := filepath.Join(tarDir, fmt.Sprintf("%s.tar", imageID))
if err := makeTar(project, tarFilePath); err != nil {
return types.Service{}, err
}
labels := map[string]string{
"belong-to": "fx",
}
if err := api.BuildImage(tarFilePath, imageID, labels); err != nil {
return types.Service{}, err
}
return types.Service{
Name: project.Name,
Image: imageID,
}, nil
}
// BuildImage build docker image
func (api *API) BuildImage(tarFile string, tag string, labels map[string]string) error {
dockerBuildContext, err := os.Open(tarFile)
if err != nil {
return err
}
defer dockerBuildContext.Close()
type buildQuery struct {
Labels string `url:"labels,omitempty"`
Tags string `url:"t,omitempty"`
Dockerfile string `url:"dockerfile,omitempty"`
}
// Apply default labels
labelsJSON, _ := json.Marshal(labels)
q := buildQuery{
Tags: tag,
Labels: string(labelsJSON),
Dockerfile: "Dockerfile",
}
qs, err := query.Values(q)
if err != nil {
return err
}
path := "/build"
url := fmt.Sprintf("%s%s?%s", api.endpoint, path, qs.Encode())
req, err := http.NewRequest("POST", url, dockerBuildContext)
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/x-tar")
client := &http.Client{Timeout: 600 * time.Second}
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
scanner := bufio.NewScanner(resp.Body)
for scanner.Scan() {
if os.Getenv("DEBUG") != "" {
log.Infof(scanner.Text())
}
}
if err := scanner.Err(); err != nil {
return err
}
return nil
}
// import (
// "fmt"
// "io/ioutil"
// "os"
// "path/filepath"
//
// "github.com/google/uuid"
// "github.com/metrue/fx/types"
// "github.com/metrue/fx/utils"
// )
//
// func makeTar(project types.Project, tarFilePath string) error {
// dir, err := ioutil.TempDir("/tmp", "fx-build-dir")
// if err != nil {
// return err
// }
//
// defer os.RemoveAll(dir)
//
// for _, file := range project.Files {
// tmpfn := filepath.Join(dir, file.Path)
// if err := utils.EnsureFile(tmpfn); err != nil {
// return err
// }
// if err := ioutil.WriteFile(tmpfn, []byte(file.Body), 0666); err != nil {
// return err
// }
// }
//
// return utils.TarDir(dir, tarFilePath)
// }
//
// // Build build a project
// func (api *API) Build(project types.Project) (types.Service, error) {
// tarDir, err := ioutil.TempDir("/tmp", "fx-tar")
// if err != nil {
// return types.Service{}, err
// }
// defer os.RemoveAll(tarDir)
//
// imageID := uuid.New().String()
// tarFilePath := filepath.Join(tarDir, fmt.Sprintf("%s.tar", imageID))
// if err := makeTar(project, tarFilePath); err != nil {
// return types.Service{}, err
// }
// labels := map[string]string{
// "belong-to": "fx",
// }
// if err := api.BuildImage(tarFilePath, imageID, labels); err != nil {
// return types.Service{}, err
// }
//
// return types.Service{
// Name: project.Name,
// Image: imageID,
// }, nil
// }

View File

@@ -1,166 +1,82 @@
package api
import (
"fmt"
"io/ioutil"
"net/http"
"os"
"path/filepath"
"strings"
"testing"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/types"
gock "gopkg.in/h2non/gock.v1"
)
func TestMakeTar(t *testing.T) {
serviceName := "mock-service-abc"
project := types.Project{
Name: serviceName,
Language: "node",
Files: []types.ProjectSourceFile{
types.ProjectSourceFile{
Path: "Dockerfile",
Body: `
FROM metrue/fx-node-base
COPY . .
EXPOSE 3000
CMD ["node", "app.js"]`,
IsHandler: false,
},
types.ProjectSourceFile{
Path: "app.js",
Body: `
const Koa = require('koa');
const bodyParser = require('koa-bodyparser');
const func = require('./fx');
const app = new Koa();
app.use(bodyParser());
app.use(ctx => {
const msg = func(ctx.request.body);
ctx.body = msg;
});
app.listen(3000);`,
IsHandler: false,
},
types.ProjectSourceFile{
Path: "fx.js",
Body: `
module.exports = (input) => {
return input.a + input.b
}
`,
IsHandler: true,
},
},
}
tarDir, err := ioutil.TempDir("/tmp", "fx-tar")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tarDir)
tarFilePath := filepath.Join(tarDir, fmt.Sprintf("%s.tar", serviceName))
if err := makeTar(project, tarFilePath); err != nil {
t.Fatal(err)
}
file, err := os.Open(tarFilePath)
if err != nil {
t.Fatal(err)
}
stat, err := file.Stat()
if err != nil {
t.Fatal(err)
}
if stat.Name() != serviceName+".tar" {
t.Fatalf("should get %s but got %s", serviceName+".tar", stat.Name())
}
if stat.Size() <= 0 {
t.Fatalf("tarfile invalid: size %d", stat.Size())
}
}
func TestBuild(t *testing.T) {
defer gock.Off()
host := config.Host{Host: "127.0.0.1"}
api, err := Create(host.Host, constants.AgentPort)
if err != nil {
t.Fatal(err)
}
url := "http://" + host.Host + ":" + constants.AgentPort
gock.New(url).
Post("/v" + api.version + "/build").
AddMatcher(func(req *http.Request, ereq *gock.Request) (bool, error) {
if strings.Contains(req.URL.String(), "/v"+api.version+"/build") {
return true, nil
}
return false, nil
}).
Reply(200).
JSON(map[string]string{
"stream": "Step 1/5...",
})
serviceName := "mock-service-abc"
project := types.Project{
Name: serviceName,
Language: "node",
Files: []types.ProjectSourceFile{
types.ProjectSourceFile{
Path: "Dockerfile",
Body: `
FROM metrue/fx-node-base
COPY . .
EXPOSE 3000
CMD ["node", "app.js"]`,
IsHandler: false,
},
types.ProjectSourceFile{
Path: "app.js",
Body: `
const Koa = require('koa');
const bodyParser = require('koa-bodyparser');
const func = require('./fx');
const app = new Koa();
app.use(bodyParser());
app.use(ctx => {
const msg = func(ctx.request.body);
ctx.body = msg;
});
app.listen(3000);`,
IsHandler: false,
},
types.ProjectSourceFile{
Path: "fx.js",
Body: `
module.exports = (input) => {
return input.a + input.b
}
`,
IsHandler: true,
},
},
}
service, err := api.Build(project)
if err != nil {
t.Fatal(err)
}
if service.Name != serviceName {
t.Fatalf("should get %s but got %s", serviceName, service.Name)
}
if service.Image == "" {
t.Fatal("service image should not be empty")
}
}
// import (
// "fmt"
// "io/ioutil"
// "os"
// "path/filepath"
// "testing"
//
// "github.com/metrue/fx/types"
// )
//
// func TestMakeTar(t *testing.T) {
// serviceName := "mock-service-abc"
// project := types.Project{
// Name: serviceName,
// Language: "node",
// Files: []types.ProjectSourceFile{
// types.ProjectSourceFile{
// Path: "Dockerfile",
// Body: `
// FROM metrue/fx-node-base
//
// COPY . .
// EXPOSE 3000
// CMD ["node", "app.js"]`,
// IsHandler: false,
// },
// types.ProjectSourceFile{
// Path: "app.js",
// Body: `
// const Koa = require('koa');
// const bodyParser = require('koa-bodyparser');
// const func = require('./fx');
//
// const app = new Koa();
// app.use(bodyParser());
// app.use(ctx => {
// const msg = func(ctx.request.body);
// ctx.body = msg;
// });
//
// app.listen(3000);`,
// IsHandler: false,
// },
// types.ProjectSourceFile{
// Path: "fx.js",
// Body: `
// module.exports = (input) => {
// return input.a + input.b
// }
// `,
// IsHandler: true,
// },
// },
// }
// tarDir, err := ioutil.TempDir("/tmp", "fx-tar")
// if err != nil {
// t.Fatal(err)
// }
// defer os.RemoveAll(tarDir)
//
// tarFilePath := filepath.Join(tarDir, fmt.Sprintf("%s.tar", serviceName))
// if err := makeTar(project, tarFilePath); err != nil {
// t.Fatal(err)
// }
//
// file, err := os.Open(tarFilePath)
// if err != nil {
// t.Fatal(err)
// }
// stat, err := file.Stat()
// if err != nil {
// t.Fatal(err)
// }
// if stat.Name() != serviceName+".tar" {
// t.Fatalf("should get %s but got %s", serviceName+".tar", stat.Name())
// }
// if stat.Size() <= 0 {
// t.Fatalf("tarfile invalid: size %d", stat.Size())
// }
// }

View File

@@ -1,55 +0,0 @@
package api
import (
"net/http"
"testing"
"github.com/golang/mock/gomock"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/types"
gock "gopkg.in/h2non/gock.v1"
)
func TestServiceRun(t *testing.T) {
defer gock.Off()
ctrl := gomock.NewController(t)
defer ctrl.Finish()
host := config.Host{Host: "127.0.0.1"}
api, err := Create(host.Host, constants.AgentPort)
if err != nil {
t.Fatal(err)
}
service := types.Service{
Name: "a-mock-service",
Image: "a-mock-image-id",
}
mockContainerID := "mock-container-id"
url := "http://" + host.Host + ":" + constants.AgentPort
gock.New(url).
Post("/v0.2.1/containers").
AddMatcher(func(req *http.Request, ereq *gock.Request) (m bool, e error) {
// TODO multiple matching not supported by gock
if req.URL.String() == url+"/v0.2.1/containers/"+mockContainerID+"/start" {
return true, nil
} else if req.URL.String() == url+"/v0.2.1/containers/create?name="+service.Name {
return true, nil
}
return false, nil
}).
Reply(201).
JSON(map[string]interface{}{
"Id": mockContainerID,
"Warnings": []string{},
})
// FIXME
if err := api.Run(9999, &service); err == nil {
t.Fatal(err)
}
}

View File

@@ -1,39 +0,0 @@
package api
import (
"net/http"
"strings"
"testing"
"github.com/golang/mock/gomock"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
gock "gopkg.in/h2non/gock.v1"
)
func TestStop(t *testing.T) {
defer gock.Off()
ctrl := gomock.NewController(t)
defer ctrl.Finish()
host := config.Host{Host: "127.0.0.1"}
api, err := Create(host.Host, constants.AgentPort)
if err != nil {
t.Fatal(err)
}
mockServiceName := "mock-service-name"
url := "http://" + host.Host + ":" + constants.AgentPort
gock.New(url).
Post("/v" + api.version + "/containers/" + mockServiceName + "/stop").
AddMatcher(func(req *http.Request, ereq *gock.Request) (m bool, e error) {
if strings.Contains(req.URL.String(), "/v"+api.version+"/containers/"+mockServiceName+"/stop") {
return true, nil
}
return false, nil
}).
Reply(204)
if err := api.Stop(mockServiceName); err != nil {
t.Fatal(err)
}
}

View File

@@ -1,81 +1,71 @@
package api
import (
"context"
"time"
"github.com/apex/log"
"github.com/docker/docker/api/types/container"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/types"
)
// UpOptions options for up
type UpOptions struct {
Body []byte
Lang string
Name string
Port int
HealtCheck bool
Project types.Project
}
// Up up a source code of function to be a service
func (api *API) Up(opt UpOptions) error {
service, err := api.Build(opt.Project)
if err != nil {
log.Fatalf("Build Service %s: %v", opt.Name, err)
return err
}
log.Infof("Build Service %s: %s", opt.Name, constants.CheckedSymbol)
if err := api.Run(opt.Port, &service); err != nil {
log.Fatalf("Run Service: %v", err)
return err
}
log.Infof("Run Service: %s", constants.CheckedSymbol)
log.Infof("Service (%s) is running on: %s:%d", service.Name, service.Host, service.Port)
if opt.HealtCheck {
go func() {
resultC, errC := api.ContainerWait(
context.Background(),
service.ID,
container.WaitConditionNextExit,
20*time.Second,
)
for {
select {
case res := <-resultC:
var msg string
if res.Error != nil {
msg = res.Error.Message
}
log.Warnf("container exited: Code(%d) %s %s", res.StatusCode, msg, constants.UncheckedSymbol)
case err := <-errC:
log.Fatalf("wait container status exit: %s, %v", constants.UncheckedSymbol, err)
}
}
}()
trys := 0
for {
if trys > 2 {
break
}
info, err := api.inspect(service.ID)
if err != nil {
log.Fatalf("healt checking failed: %v", err)
}
if info.State.Running {
log.Info("service is running")
} else {
log.Warnf("service is %s", info.State.Status)
}
time.Sleep(1 * time.Second)
trys++
}
}
return nil
}
// // UpOptions options for up
// type UpOptions struct {
// Body []byte
// Lang string
// Name string
// Port int
// HealtCheck bool
// Project types.Project
// }
//
// // Up up a source code of function to be a service
// func (api *API) Up(opt UpOptions) error {
// service, err := api.Build(opt.Project)
// if err != nil {
// log.Fatalf("Build Service %s: %v", opt.Name, err)
// return err
// }
// log.Infof("Build Service %s: %s", opt.Name, constants.CheckedSymbol)
//
// if err := api.Run(opt.Port, &service); err != nil {
// log.Fatalf("Run Service: %v", err)
// return err
// }
// log.Infof("Run Service: %s", constants.CheckedSymbol)
// log.Infof("Service (%s) is running on: %s:%d", service.Name, service.Host, service.Port)
//
// if opt.HealtCheck {
// go func() {
// resultC, errC := api.ContainerWait(
// context.Background(),
// service.ID,
// container.WaitConditionNextExit,
// 20*time.Second,
// )
// for {
// select {
// case res := <-resultC:
// var msg string
// if res.Error != nil {
// msg = res.Error.Message
// }
// log.Warnf("container exited: Code(%d) %s %s", res.StatusCode, msg, constants.UncheckedSymbol)
// case err := <-errC:
// log.Fatalf("wait container status exit: %s, %v", constants.UncheckedSymbol, err)
// }
// }
// }()
//
// trys := 0
// for {
// if trys > 2 {
// break
// }
// info, err := api.inspect(service.ID)
// if err != nil {
// log.Fatalf("healt checking failed: %v", err)
// }
// if info.State.Running {
// log.Info("service is running")
// } else {
// log.Warnf("service is %s", info.State.Status)
// }
// time.Sleep(1 * time.Second)
// trys++
// }
// }
//
// return nil
// }

View File

@@ -9,10 +9,12 @@ import (
"io/ioutil"
"os"
"path/filepath"
"strings"
"github.com/apex/log"
dockerTypes "github.com/docker/docker/api/types"
dockerTypesContainer "github.com/docker/docker/api/types/container"
dockerFilters "github.com/docker/docker/api/types/filters"
"github.com/docker/docker/client"
"github.com/docker/go-connections/nat"
"github.com/google/uuid"
@@ -74,7 +76,9 @@ func (d *Docker) BuildImage(ctx context.Context, workdir string, name string) er
if err != nil {
return err
}
log.Info(string(body))
if os.Getenv("DEBUG") != "" {
log.Info(string(body))
}
return nil
}
@@ -131,6 +135,11 @@ func (d *Docker) InspectImage(ctx context.Context, name string, img interface{})
return json.NewDecoder(rdr).Decode(&img)
}
// TagImage tag image
func (d *Docker) TagImage(ctx context.Context, name string, tag string) error {
return d.ImageTag(ctx, name, tag)
}
// StartContainer create and start a container from given image
func (d *Docker) StartContainer(ctx context.Context, name string, image string, ports []types.PortBinding) error {
portSet := nat.PortSet{}
@@ -180,7 +189,59 @@ func (d *Docker) StopContainer(ctx context.Context, name string) error {
// InspectContainer inspect a container
func (d *Docker) InspectContainer(ctx context.Context, name string, container interface{}) error {
return nil
res, err := d.ContainerInspect(ctx, name)
if err != nil {
return err
}
body, err := json.Marshal(res)
if err != nil {
return err
}
return json.Unmarshal(body, &container)
}
// ListContainer list containers
func (d *Docker) ListContainer(ctx context.Context, name string) ([]types.Service, error) {
args := dockerFilters.NewArgs(
dockerFilters.Arg("label", "belong-to=fx"),
)
containers, err := d.ContainerList(ctx, dockerTypes.ContainerListOptions{
Filters: args,
})
if err != nil {
return []types.Service{}, err
}
svs := make(map[string]types.Service)
for _, container := range containers {
// container name have extra forward slash
// https://github.com/moby/moby/issues/6705
if strings.HasPrefix(container.Names[0], fmt.Sprintf("/%s", name)) {
svs[container.Image] = types.Service{
Name: container.Names[0],
Image: container.Image,
ID: container.ID,
Host: container.Ports[0].IP,
Port: int(container.Ports[0].PublicPort),
State: container.State,
}
}
}
services := []types.Service{}
for _, s := range svs {
services = append(services, s)
}
return services, nil
}
// Version get version of docker engine
func (d *Docker) Version(ctx context.Context) (string, error) {
ping, err := d.Ping(ctx)
if err != nil {
return "", err
}
return ping.APIVersion, nil
}
var (

View File

@@ -8,6 +8,7 @@ import (
"time"
dockerTypes "github.com/docker/docker/api/types"
"github.com/metrue/fx/types"
)
func TestDocker(t *testing.T) {
@@ -42,6 +43,23 @@ func TestDocker(t *testing.T) {
t.Fatalf("should have built image with tag %s", name)
}
if err := cli.StartContainer(ctx, name, name, []types.PortBinding{
types.PortBinding{
ServiceBindingPort: 9000,
ContainerExposePort: 3000,
},
}); err != nil {
t.Fatal(err)
}
var container dockerTypes.ContainerJSON
if err := cli.InspectContainer(ctx, name, &container); err != nil {
t.Fatal(err)
}
if container.Name != "/"+name {
t.Fatalf("should get %s but got %s", "/"+name, container.Name)
}
username := os.Getenv("DOCKER_USERNAME")
password := os.Getenv("DOCKER_PASSWORD")
if username == "" || password == "" {

View File

@@ -10,8 +10,11 @@ import (
type ContainerRuntime interface {
BuildImage(ctx context.Context, workdir string, name string) error
PushImage(ctx context.Context, name string) (string, error)
InspectImage(ct context.Context, name string, img interface{}) error
InspectImage(ctx context.Context, name string, img interface{}) error
TagImage(ctx context.Context, name string, tag string) error
StartContainer(ctx context.Context, name string, image string, bindings []types.PortBinding) error
StopContainer(ctx context.Context, name string) error
InspectContainer(ctx context.Context, name string, container interface{}) error
ListContainer(ctx context.Context, filter string) ([]types.Service, error)
Version(ctx context.Context) (string, error)
}

76
context/context.go Normal file
View File

@@ -0,0 +1,76 @@
package context
import (
"context"
"github.com/urfave/cli"
)
type key string
const (
keyCliCtx = key("cmd_cli")
)
// Contexter ctx interface
type Contexter interface {
Get(k string) interface{}
Set(k string, v interface{})
Use(fn func(ctx *Context) error) error
GetContext() context.Context
GetCliContext() *cli.Context
}
// Context fx context
type Context struct {
context.Context
}
// NewContext new a context
func NewContext() *Context {
ctx := context.Background()
return &Context{ctx}
}
// FromCliContext create context from cli.Context
func FromCliContext(c *cli.Context) *Context {
ctx := NewContext()
ctx.WithCliContext(c)
return ctx
}
// WithCliContext set cli.Context
func (ctx *Context) WithCliContext(c *cli.Context) {
newCtx := context.WithValue(ctx.Context, keyCliCtx, c)
ctx.Context = newCtx
}
// GetCliContext get cli.Context
func (ctx *Context) GetCliContext() *cli.Context {
return ctx.Value(keyCliCtx).(*cli.Context)
}
// Set a value with name
func (ctx *Context) Set(name string, value interface{}) {
newCtx := context.WithValue(ctx.Context, name, value)
ctx.Context = newCtx
}
// Get a value
func (ctx *Context) Get(name string) interface{} {
return ctx.Context.Value(name)
}
// Use invole a middle
func (ctx *Context) Use(fn func(ctx *Context) error) error {
return fn(ctx)
}
// GetContext get context
func (ctx *Context) GetContext() context.Context {
return ctx.Context
}
var (
_ Contexter = &Context{}
)

29
context/context_test.go Normal file
View File

@@ -0,0 +1,29 @@
package context
import (
"testing"
"github.com/urfave/cli"
)
func TestContext(t *testing.T) {
ctx := NewContext()
cli := cli.NewContext(nil, nil, nil)
ctx.WithCliContext(cli)
c := ctx.GetCliContext()
if c != cli {
t.Fatalf("should get %v but got %v", cli, c)
}
key := "k_1"
value := "hello"
ctx.Set(key, "hello")
v := ctx.Get(key).(string)
if v != value {
t.Fatalf("should get %v but %v", value, v)
}
if ctx.GetContext() == nil {
t.Fatalf("should get context")
}
}

104
context/mocks/context.go Normal file
View File

@@ -0,0 +1,104 @@
// Code generated by MockGen. DO NOT EDIT.
// Source: context.go
// Package mock_context is a generated GoMock package.
package mock_context
import (
context "context"
gomock "github.com/golang/mock/gomock"
context0 "github.com/metrue/fx/context"
cli "github.com/urfave/cli"
reflect "reflect"
)
// MockContexter is a mock of Contexter interface
type MockContexter struct {
ctrl *gomock.Controller
recorder *MockContexterMockRecorder
}
// MockContexterMockRecorder is the mock recorder for MockContexter
type MockContexterMockRecorder struct {
mock *MockContexter
}
// NewMockContexter creates a new mock instance
func NewMockContexter(ctrl *gomock.Controller) *MockContexter {
mock := &MockContexter{ctrl: ctrl}
mock.recorder = &MockContexterMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockContexter) EXPECT() *MockContexterMockRecorder {
return m.recorder
}
// Get mocks base method
func (m *MockContexter) Get(k string) interface{} {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Get", k)
ret0, _ := ret[0].(interface{})
return ret0
}
// Get indicates an expected call of Get
func (mr *MockContexterMockRecorder) Get(k interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Get", reflect.TypeOf((*MockContexter)(nil).Get), k)
}
// Set mocks base method
func (m *MockContexter) Set(k string, v interface{}) {
m.ctrl.T.Helper()
m.ctrl.Call(m, "Set", k, v)
}
// Set indicates an expected call of Set
func (mr *MockContexterMockRecorder) Set(k, v interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Set", reflect.TypeOf((*MockContexter)(nil).Set), k, v)
}
// Use mocks base method
func (m *MockContexter) Use(fn func(*context0.Context) error) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Use", fn)
ret0, _ := ret[0].(error)
return ret0
}
// Use indicates an expected call of Use
func (mr *MockContexterMockRecorder) Use(fn interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Use", reflect.TypeOf((*MockContexter)(nil).Use), fn)
}
// GetContext mocks base method
func (m *MockContexter) GetContext() context.Context {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetContext")
ret0, _ := ret[0].(context.Context)
return ret0
}
// GetContext indicates an expected call of GetContext
func (mr *MockContexterMockRecorder) GetContext() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetContext", reflect.TypeOf((*MockContexter)(nil).GetContext))
}
// GetCliContext mocks base method
func (m *MockContexter) GetCliContext() *cli.Context {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetCliContext")
ret0, _ := ret[0].(*cli.Context)
return ret0
}
// GetCliContext indicates an expected call of GetCliContext
func (mr *MockContexterMockRecorder) GetCliContext() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetCliContext", reflect.TypeOf((*MockContexter)(nil).GetCliContext))
}

View File

@@ -15,9 +15,6 @@ import (
"github.com/metrue/fx/utils"
)
// Version binary version
var Version = "0.0.1"
func init() {
// TODO clean it up
os.Setenv("DEBUG", "true")
@@ -43,6 +40,7 @@ docker_packer <encrypt_docker_project_source_tree> <image_name>
}
var tree map[string]string
//nolint
if err := json.Unmarshal([]byte(str), &tree); err != nil {
log.Fatalf("could not unmarshal meta: %s", meta)
os.Exit(1)

View File

@@ -1,15 +0,0 @@
package deploy
import (
"context"
types "github.com/metrue/fx/types"
)
// Deployer make a image a service
type Deployer interface {
Deploy(ctx context.Context, fn types.Func, name string, bindings []types.PortBinding) error
Destroy(ctx context.Context, name string) error
Update(ctx context.Context, name string) error
GetStatus(ctx context.Context, name string) error
}

View File

@@ -1,109 +0,0 @@
package docker
import (
"context"
"fmt"
"log"
"os"
"time"
dockerTypes "github.com/docker/docker/api/types"
dockerHTTP "github.com/metrue/fx/container_runtimes/docker/http"
runtime "github.com/metrue/fx/container_runtimes/docker/sdk"
"github.com/metrue/fx/deploy"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
"github.com/pkg/errors"
)
// Docker manage container
type Docker struct {
localClient *runtime.Docker
}
// CreateClient create a docker instance
func CreateClient(ctx context.Context) (*Docker, error) {
cli, err := runtime.CreateClient(ctx)
if err != nil {
return nil, err
}
return &Docker{localClient: cli}, nil
}
// Deploy create a Docker container from given image, and bind the constants.FxContainerExposePort to given port
func (d *Docker) Deploy(ctx context.Context, fn types.Func, name string, ports []types.PortBinding) error {
// if DOCKER_REMOTE_HOST and DOCKER_REMOTE_PORT given
// it means user is going to deploy service to remote host
host := os.Getenv("DOCKER_REMOTE_HOST")
port := os.Getenv("DOCKER_REMOTE_PORT")
if port != "" && host != "" {
httpClient, err := dockerHTTP.Create(host, port)
if err != nil {
return err
}
project, err := packer.Pack(name, fn)
if err != nil {
return errors.Wrapf(err, "could pack function %v (%s)", name, fn)
}
return httpClient.Up(dockerHTTP.UpOptions{
Body: []byte(fn.Source),
Lang: fn.Language,
Name: name,
Port: int(ports[0].ServiceBindingPort),
HealtCheck: false,
Project: project,
})
}
workdir := fmt.Sprintf("/tmp/fx-%d", time.Now().Unix())
defer os.RemoveAll(workdir)
if err := packer.PackIntoDir(fn, workdir); err != nil {
log.Fatalf("could not pack function %v: %v", fn, err)
return err
}
if err := d.localClient.BuildImage(ctx, workdir, name); err != nil {
log.Fatalf("could not build image: %v", err)
return err
}
nameWithTag := name + ":latest"
if err := d.localClient.ImageTag(ctx, name, nameWithTag); err != nil {
log.Fatalf("could not tag image: %v", err)
return err
}
// when deploy a function on a bare Docker running without Kubernetes,
// image would be built on-demand on host locally, so there is no need to
// pull image from remote.
// But it takes some times waiting image ready after image built, we retry to make sure it ready here
var imgInfo dockerTypes.ImageInspect
if err := utils.RunWithRetry(func() error {
return d.localClient.InspectImage(ctx, name, &imgInfo)
}, time.Second*1, 5); err != nil {
return err
}
return d.localClient.StartContainer(ctx, name, name, ports)
}
// Update a container
func (d *Docker) Update(ctx context.Context, name string) error {
return nil
}
// Destroy stop and remove container
func (d *Docker) Destroy(ctx context.Context, name string) error {
return d.localClient.ContainerStop(ctx, name, nil)
}
// GetStatus get status of container
func (d *Docker) GetStatus(ctx context.Context, name string) error {
return nil
}
var (
_ deploy.Deployer = &Docker{}
)

View File

@@ -1,47 +0,0 @@
package docker
import (
"context"
"testing"
"time"
"github.com/metrue/fx/types"
)
func TestDocker(t *testing.T) {
ctx := context.Background()
cli, err := CreateClient(ctx)
if err != nil {
t.Fatal(err)
}
name := "helloworld"
bindings := []types.PortBinding{
types.PortBinding{
ServiceBindingPort: 80,
ContainerExposePort: 3000,
},
types.PortBinding{
ServiceBindingPort: 443,
ContainerExposePort: 3000,
},
}
fn := types.Func{
Language: "node",
Source: `
module.exports = (ctx) => {
ctx.body = 'hello world'
}
`,
}
if err := cli.Deploy(ctx, fn, name, bindings); err != nil {
t.Fatal(err)
}
time.Sleep(1 * time.Second)
if err := cli.Destroy(ctx, name); err != nil {
t.Fatal(err)
}
}

46
docs/ubuntu.md Normal file
View File

@@ -0,0 +1,46 @@
# fx on Ubuntu
> The guide is verified on Amazon Lightsail ubuntu 18.08 instance
## Install Docker
```shell
apt-get remove -y docker docker-engine docker.io containerd runc
apt-get update -y
apt-get install -y apt-transport-https ca-certificates curl software-properties-common lsb-core
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
apt-get update -y
apt-get install -y docker-ce
docker run hello-world
```
## Install fx
```shell
$ curl -o- https://raw.githubusercontent.com/metrue/fx/master/scripts/install.sh | sudo bash
```
## Deploy a function onto localhost
```shell
$ cat func.js
module.exports = (ctx) => {
ctx.body = 'hello world'
}
$ fx up -n test -p 2000 func.js
$ curl 127.0.0.1:2000
```
## Deploy a function onto remote host
* make sure your instance can be ssh login
* make sure your instance accept port 8866
then you can deploy function to remote host
```shell
DOCKER_REMOTE_HOST_ADDR=<your host> DOCKER_REMOTE_HOST_USER=<your user> DOCKER_REMOTE_HOST_PASSWORD=<your password> fx up -p 2000 test/functions/func.js
```

View File

@@ -2,7 +2,6 @@ package doctor
import (
"github.com/apex/log"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/pkg/command"
"github.com/metrue/go-ssh-client"
@@ -10,16 +9,23 @@ import (
// Doctor health checking
type Doctor struct {
host config.Host
host string
sshClient ssh.Client
}
func isLocal(host string) bool {
if host == "" {
return false
}
return host == "127.0.0.1" || host == "localhost" || host == "0.0.0.0"
}
// New a doctor
func New(host config.Host) *Doctor {
sshClient := ssh.New(host.Host).
WithUser(host.User).
WithPassword(host.Password)
func New(host, user, password string) *Doctor {
sshClient := ssh.New(host).
WithUser(user).
WithPassword(password)
return &Doctor{
host: host,
sshClient: sshClient,
@@ -32,7 +38,7 @@ func (d *Doctor) Start() error {
checkAgent := "docker inspect " + constants.AgentContainerName
cmds := []*command.Command{}
if d.host.IsRemote() {
if !isLocal(d.host) {
cmds = append(cmds,
command.New("check if dockerd is running", checkDocker, command.NewRemoteRunner(d.sshClient)),
command.New("check if fx agent is running", checkAgent, command.NewRemoteRunner(d.sshClient)),

47
examples/functions/Rust/README.md vendored Normal file
View File

@@ -0,0 +1,47 @@
# Make a Rust function a service with fx
Write a function like,
```rust
pub mod fns {
#[derive(Serialize)]
pub struct Response {
pub result: i32,
}
#[derive(Deserialize)]
pub struct Request {
pub a: i32,
pub b: i32,
}
pub fn func(req: Request) -> Response {
Response {
result: req.a + req.b,
}
}
}
```
then deploy it with `fx up` command,
```shell
$ fx up -p 8080 func.rs
```
test it using `curl`
```shell
$ curl -X 'POST' --header 'Content-Type: application/json' --data '{"a":1,"b":1}' '0.0.0.0:3000'
HTTP/1.1 200 OK
Content-Length: 12
Content-Type: application/json
Date: Fri, 06 Dec 2019 06:45:14 GMT
Server: Rocket
{
"result": 2
}
```

220
fx.go
View File

@@ -5,28 +5,32 @@ import (
"fmt"
"net/http"
"os"
"path"
"regexp"
"github.com/apex/log"
"github.com/google/uuid"
"github.com/metrue/fx/config"
aurora "github.com/logrusorgru/aurora"
"github.com/metrue/fx/context"
"github.com/metrue/fx/handlers"
"github.com/metrue/fx/middlewares"
"github.com/urfave/cli"
)
const version = "0.8.0"
var cfg *config.Config
const version = "0.8.74"
func init() {
go checkForUpdate()
configDir := path.Join(os.Getenv("HOME"), ".fx")
cfg := config.New(configDir)
}
if err := cfg.Init(); err != nil {
log.Fatalf("Init config failed %s", err)
os.Exit(1)
func handle(fns ...func(ctx context.Contexter) error) func(ctx *cli.Context) error {
return func(c *cli.Context) error {
ctx := context.FromCliContext(c)
for _, fn := range fns {
if err := fn(ctx); err != nil {
panic(err)
}
}
return nil
}
}
@@ -65,109 +69,71 @@ func main() {
app.Usage = "makes function as a service"
app.Version = version
defer func() {
if r := recover(); r != nil {
fmt.Println(aurora.Red("*****************"))
fmt.Println(r)
fmt.Println(aurora.Red("*****************"))
}
}()
app.Commands = []cli.Command{
{
Name: "infra",
Usage: "manage infrastructure of fx",
Usage: "manage infrastructure",
Subcommands: []cli.Command{
{
Name: "add",
Usage: "add a new machine",
Name: "create",
Usage: "create a infra for fx service",
Flags: []cli.Flag{
cli.StringFlag{
Name: "name, N",
Usage: "a alias name for this machine",
Name: "type, t",
Usage: "infracture type, 'docker', 'k8s' and 'k3s' support",
},
cli.StringFlag{
Name: "host, H",
Usage: "host name or IP address of a machine",
Name: "name, n",
Usage: "name to identify the infrastructure",
},
cli.StringFlag{
Name: "user, U",
Usage: "user name required for SSH login",
Name: "host",
Usage: "user and ip of your host, eg. 'root@182.12.1.12'",
},
cli.StringFlag{
Name: "password, P",
Usage: "password required for SSH login",
Name: "master",
Usage: "serve as master node in K3S cluster, eg. 'root@182.12.1.12'",
},
},
Action: func(c *cli.Context) error {
return handlers.AddHost(cfg)(c)
},
},
{
Name: "remove",
Usage: "remove an existing machine",
Action: func(c *cli.Context) error {
return handlers.RemoveHost(cfg)(c)
},
},
{
Name: "list",
Aliases: []string{"ls"},
Usage: "list machines",
Action: func(c *cli.Context) error {
return handlers.ListHosts(cfg)(c)
},
},
{
Name: "activate",
Usage: "enable a machine be a host of fx infrastructure",
Action: func(c *cli.Context) error {
return handlers.Activate(cfg)(c)
},
},
{
Name: "deactivate",
Usage: "disable a machine be a host of fx infrastructure",
Action: func(c *cli.Context) error {
return handlers.Deactivate(cfg)(c)
},
},
},
},
{
Name: "image",
Usage: "manage image of service",
Subcommands: []cli.Command{
{
Name: "build",
Usage: "build a image",
Flags: []cli.Flag{
cli.StringFlag{
Name: "tag, t",
Usage: "image tag",
Name: "agents",
Usage: "serve as agent node in K3S cluster, eg. 'root@187.1. 2. 3,root@123.3.2.1'",
},
},
Action: func(c *cli.Context) error {
return handlers.BuildImage(cfg)(c)
},
Action: handle(
middlewares.LoadConfig,
handlers.Setup,
),
},
{
Name: "export",
Usage: "export the Docker project of service",
Flags: []cli.Flag{
cli.StringFlag{
Name: "output, o",
Usage: "output directory",
},
},
Action: func(c *cli.Context) error {
return handlers.ExportImage()(c)
},
Name: "list",
Usage: "list all infrastructures",
Action: handle(
middlewares.LoadConfig,
handlers.ListInfra,
),
},
{
Name: "use",
Usage: "set current context to target cloud with given name",
Action: handle(
middlewares.LoadConfig,
handlers.UseInfra,
),
},
},
},
{
Name: "doctor",
Usage: "health check for fx",
Action: func(c *cli.Context) error {
return handlers.Doctor(cfg)(c)
},
},
{
Name: "up",
Usage: "deploy a function or a group of functions",
Usage: "deploy a function",
ArgsUsage: "[func.go func.js func.py func.rb ...]",
Flags: []cli.Flag{
cli.StringFlag{
@@ -188,25 +154,36 @@ func main() {
Usage: "force deploy a function or functions",
},
},
Action: func(c *cli.Context) error {
return handlers.Up(cfg)(c)
},
Action: handle(
middlewares.LoadConfig,
middlewares.Provision,
middlewares.Parse("up"),
middlewares.Binding,
middlewares.Build,
handlers.Up,
),
},
{
Name: "down",
Usage: "destroy a service",
ArgsUsage: "[service 1, service 2, ....]",
Action: func(c *cli.Context) error {
return handlers.Down(cfg)(c)
},
Action: handle(
middlewares.Parse("down"),
middlewares.LoadConfig,
middlewares.Provision,
handlers.Down,
),
},
{
Name: "list",
Aliases: []string{"ls"},
Usage: "list deployed services",
Action: func(c *cli.Context) error {
return handlers.List(cfg)(c)
},
Action: handle(
middlewares.Parse("list"),
middlewares.LoadConfig,
middlewares.Provision,
handlers.List,
),
},
{
Name: "call",
@@ -217,13 +194,54 @@ func main() {
Usage: "fx server host, default is localhost",
},
},
Action: func(c *cli.Context) error {
return handlers.Call(cfg)(c)
Action: handle(handlers.Call),
},
{
Name: "image",
Usage: "manage image of service",
Subcommands: []cli.Command{
{
Name: "build",
Usage: "build a image",
Flags: []cli.Flag{
cli.StringFlag{
Name: "tag, t",
Usage: "image tag",
},
},
Action: handle(
middlewares.LoadConfig,
middlewares.Provision,
middlewares.Parse("image_build"),
handlers.BuildImage,
),
},
{
Name: "export",
Usage: "export the Docker project of service",
Flags: []cli.Flag{
cli.StringFlag{
Name: "output, o",
Usage: "output directory",
},
},
Action: handle(
middlewares.LoadConfig,
middlewares.Provision,
middlewares.Parse("image_export"),
handlers.ExportImage,
),
},
},
},
{
Name: "doctor",
Usage: "health check for fx",
Action: handle(handlers.Doctor),
},
}
if err := app.Run(os.Args); err != nil {
log.Fatalf("fx startup with fatal: %v", err)
os.Exit(1)
}
}

19
go.mod
View File

@@ -6,6 +6,7 @@ require (
github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78 // indirect
github.com/Microsoft/go-winio v0.4.14 // indirect
github.com/apex/log v1.1.1
github.com/briandowns/spinner v1.7.0
github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/docker/docker v0.0.0-20190313072916-46036c230805
github.com/docker/go-connections v0.4.0
@@ -20,25 +21,29 @@ require (
github.com/googleapis/gnostic v0.3.1 // indirect
github.com/gorilla/mux v1.7.3 // indirect
github.com/imdario/mergo v0.3.7 // indirect
github.com/magiconair/properties v1.8.1 // indirect
github.com/metrue/go-ssh-client v0.0.0-20190810064746-98a7a27048f3
github.com/logrusorgru/aurora v0.0.0-20191017060258-dc85c304c434
github.com/metrue/go-ssh-client v0.0.0-20191209160027-5773243a8bc9
github.com/mholt/archiver v3.1.1+incompatible
github.com/mitchellh/go-homedir v1.1.0
github.com/morikuni/aec v1.0.0 // indirect
github.com/nwaples/rardecode v1.0.0 // indirect
github.com/olekukonko/tablewriter v0.0.4
github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
github.com/opencontainers/image-spec v1.0.1 // indirect
github.com/pelletier/go-toml v1.4.0 // indirect
github.com/otiai10/copy v1.0.2
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
github.com/pierrec/lz4 v0.0.0-20190222153722-062282ea0dcf // indirect
github.com/pkg/errors v0.8.1
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/spf13/viper v1.4.0
github.com/spf13/viper v1.3.2
github.com/stretchr/testify v1.4.0
github.com/ugorji/go v1.1.7 // indirect
github.com/urfave/cli v1.22.1
github.com/urfave/cli v1.22.2
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
golang.org/x/sys v0.0.0-20190712062909-fae7ac547cb7 // indirect
gopkg.in/h2non/gock.v1 v1.0.15
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4 // indirect
google.golang.org/grpc v1.21.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.2.7
gotest.tools v2.2.0+incompatible // indirect
k8s.io/api v0.0.0-20190925180651-d58b53da08f5
k8s.io/apimachinery v0.0.0-20190925235427-62598f38f24e

113
go.sum
View File

@@ -15,11 +15,8 @@ github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03
github.com/Microsoft/go-winio v0.4.14 h1:+hMXMk01us9KgxGb7ftKQt2Xpf5hH/yky+TDA+qxleU=
github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA=
github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/PuerkitoBio/purell v1.0.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
github.com/PuerkitoBio/urlesc v0.0.0-20160726150825-5bd2802263f2/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/apex/log v1.1.1 h1:BwhRZ0qbjYtTob0I+2M+smavV0kOC8XgcnGZcyL9liA=
github.com/apex/log v1.1.1/go.mod h1:Ls949n1HFtXfbDcjiTTFQqkVUrte0puoIBfO3SVgwOA=
github.com/aphistic/golf v0.0.0-20180712155816-02c07f170c5a/go.mod h1:3NqKYiepwy8kCu4PNA+aP7WUV72eXWJeP9/r3/K9aLE=
@@ -27,16 +24,12 @@ github.com/aphistic/sweet v0.2.0/go.mod h1:fWDlIh/isSE9n6EPsRmC0det+whmX6dJid3st
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
github.com/aws/aws-sdk-go v1.20.6/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aybabtme/rgbterm v0.0.0-20170906152045-cc83f3b3ce59/go.mod h1:q/89r3U2H7sSsE2t6Kca0lfwTK8JdoNGS/yzM/4iH5I=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/briandowns/spinner v1.7.0 h1:aan1hBBOoscry2TXAkgtxkJiq7Se0+9pt+TUWaPrB4g=
github.com/briandowns/spinner v1.7.0/go.mod h1://Zf9tMcxfRUA36V23M6YGEAv+kECGfvpnLTnb8n4XQ=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/cpuguy83/go-md2man v1.0.10 h1:BSKMNlYxDvnunlTymqtgONjNnaRV1sTpcovwwjF22jk=
github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d h1:U+s90UTSYgptZMwQh2aRr3LuazLJIa+Pg3Kc1ylSYVY=
@@ -46,7 +39,6 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug=
github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v0.0.0-20190313072916-46036c230805 h1:Imk7y5LY4ljn+DhwaPVj9d8kvAxiZw8DQGwNmivIom0=
@@ -64,24 +56,21 @@ github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdf
github.com/elazarl/goproxy v0.0.0-20170405201442-c4fc26588b6e/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc=
github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=
github.com/evanphx/json-patch v4.2.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gin-contrib/sse v0.0.0-20190301062529-5545eab6dad3 h1:t8FVkw33L+wilf2QiWkw0UV77qRpcH/JHPKGpKa2E8g=
github.com/gin-contrib/sse v0.0.0-20190301062529-5545eab6dad3/go.mod h1:VJ0WA2NBN22VlZ2dKZQPAPnyWw5XTlK1KymzLKsr59s=
github.com/gin-gonic/gin v1.4.0 h1:3tMoCCfM7ppqsR0ptz/wi1impNpT7/9wQtMZ8lr1mCQ=
github.com/gin-gonic/gin v1.4.0/go.mod h1:OW2EZn3DO8Ln9oIKOvM++LBO+5UPHJJDH72/q/3rZdM=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas=
github.com/go-openapi/jsonpointer v0.0.0-20160704185906-46af16f9f7b1/go.mod h1:+35s3my2LFTysnkMfxsJBAMHj/DoqoB9knIWoYG/Vk0=
github.com/go-openapi/jsonreference v0.0.0-20160704190145-13c6e3589ad9/go.mod h1:W3Z9FmVs9qj+KR4zFKmDPGiLdk1D9Rlm7cyMvf57TTg=
github.com/go-openapi/spec v0.0.0-20160808142527-6aced65f8501/go.mod h1:J8+jY1nAiCcj+friV/PDoE1/3eeccG9LYBs0tYvLOWc=
github.com/go-openapi/swag v0.0.0-20160704191624-1d0bd113de87/go.mod h1:DXUve3Dpr1UfpPtxFw+EFuQ41HhCWZfha5jSVRG7C7I=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/gobuffalo/envy v1.7.0 h1:GlXgaiBkmrYMHco6t4j7SacKO4XUjvh5pwXh0f4uxXU=
github.com/gobuffalo/envy v1.7.0/go.mod h1:n7DRkBerg/aorDM8kbduw5dN3oXGswK5liaSCx4T5NI=
github.com/gobuffalo/logger v1.0.0/go.mod h1:2zbswyIUa45I+c+FLXuWl9zSWEiVuthsk8ze5s8JvPs=
@@ -90,14 +79,10 @@ github.com/gobuffalo/packd v0.3.0/go.mod h1:zC7QkmNkYVGKPw4tHpBQ+ml7W/3tIebgeo1b
github.com/gobuffalo/packr v1.30.1 h1:hu1fuVR3fXEZR7rXNW3h8rqSML8EVAf6KNm0NKO/wKg=
github.com/gobuffalo/packr v1.30.1/go.mod h1:ljMyFO2EcrnzsHsN99cvbq055Y9OhRrIaviy289eRuk=
github.com/gobuffalo/packr/v2 v2.5.1/go.mod h1:8f9c96ITobJlPzI44jj+4tHnEKNt0xXWSVlXRN9X1Iw=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1 h1:/s5zKNz0uPFCZ5hddgPdo2TK2TVrUNMn0OOX8/aZMTE=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/gogo/protobuf v1.2.2-0.20190723190241-65acae22fc9d h1:3PaI8p3seN09VjbTYC/QWlUZdZ1qS1zGjy7LH2Wt07I=
github.com/gogo/protobuf v1.2.2-0.20190723190241-65acae22fc9d/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1 h1:G5FRp8JnTd7RQH5kemVNlMeyXQAztQ3mOWV95KxsXH8=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
@@ -111,7 +96,6 @@ github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5y
github.com/golang/snappy v0.0.1 h1:Qgr9rKW7uDUkrbSmQeiDsGa8SjGyCOGtuasMWwvp2P4=
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
@@ -131,13 +115,7 @@ github.com/googleapis/gnostic v0.3.1/go.mod h1:on+2t9HRStVgn95RSsFWFz+6Q0Snyqv1a
github.com/gophercloud/gophercloud v0.1.0/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8=
github.com/gorilla/mux v1.7.3 h1:gnP5JzjVOuiZD07fKKToCAOjS0yOpj/qPETTXCCS6hw=
github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
github.com/gregjones/httpcache v0.0.0-20170728041850-787624de3eb7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=
github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542 h1:2VTzZjLZBgl62/EtslCrtky5vbi9dd7HrQPQIx6wqiw=
github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542/go.mod h1:Ow0tF8D4Kplbc8s8sSb3V2oUCygFHVp8gC3Dn6U4MNI=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
@@ -150,17 +128,14 @@ github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANyt
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/joho/godotenv v1.3.0 h1:Zjp+RcGpHhGlrMbJzXTrZZPrWj+1vfm90La1wgB6Bhc=
github.com/joho/godotenv v1.3.0/go.mod h1:7hK45KPybAkOC6peb+G5yklZfMxEjkZhHbwpqxOKXbg=
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
github.com/jpillora/backoff v0.0.0-20180909062703-3050d21c67d7/go.mod h1:2iMrUgbbvHEiQClaW2NsSzMyGHqN+rDFqY705q49KG0=
github.com/json-iterator/go v0.0.0-20180612202835-f2b4162afba3/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/json-iterator/go v1.1.7 h1:KfgG9LzI+pYjr4xvmz/5H4FXjokeP+rlHLhv3iH62Fo=
github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/karrick/godirwalk v1.10.12 h1:BqUm+LuJcXjGv1d2mj3gBiQyrQ57a0rYoAmhvJQ7RDU=
github.com/karrick/godirwalk v1.10.12/go.mod h1:RoGL9dQei4vP9ilrpETWE8CLOZ1kiN0LhBygSwrAsHA=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
@@ -174,23 +149,31 @@ github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORN
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/logrusorgru/aurora v0.0.0-20191017060258-dc85c304c434 h1:im9kkmH0WWwxzegiv18gSUJbuXR9y028rXrWuPp6Jug=
github.com/logrusorgru/aurora v0.0.0-20191017060258-dc85c304c434/go.mod h1:7rIyQOR62GCctdiQpZ/zOJlFyk6y+94wXzv6RNZgaR4=
github.com/magiconair/properties v1.8.0 h1:LLgXmsheXeRoUOBOjtwPQCWIYqM/LU1ayDtDePerRcY=
github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/magiconair/properties v1.8.1 h1:ZC2Vc7/ZFkGmsVC9KvOjumD+G5lXy2RtTKyzRKO2BQ4=
github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/mailru/easyjson v0.0.0-20160728113105-d5b7844b561a/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mattn/go-colorable v0.1.1/go.mod h1:FuOcm+DKB9mbwrcAfNl7/TZVBZ6rcnceauSikq3lYCQ=
github.com/mattn/go-colorable v0.1.2 h1:/bC9yWikZXAL9uJdulbSfyVNIR3n3trXl+v8+1sx8mU=
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-isatty v0.0.5/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.7 h1:UvyT9uN+3r7yLEYSlJsbQGdsaB/a0DlgWP3pql6iwOc=
github.com/mattn/go-isatty v0.0.7/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.8 h1:HLtExJ+uU2HOZ+wI0Tt5DtUDrx8yhUqDcp7fYERX4CE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/metrue/go-ssh-client v0.0.0-20190810064746-98a7a27048f3 h1:IzZATG6TKa6amM5pr8HK7w/Ae4l0VBjmTwTmVbszWFw=
github.com/metrue/go-ssh-client v0.0.0-20190810064746-98a7a27048f3/go.mod h1:ERHOEBrDy6+8vfoJjjmhdmBpOzdvvP7bLtwYTTK6LOs=
github.com/mattn/go-runewidth v0.0.6 h1:V2iyH+aX9C5fsYCpK60U8BYIvmhqxuOL3JZcqc1NB7k=
github.com/mattn/go-runewidth v0.0.6/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/mattn/go-runewidth v0.0.7 h1:Ei8KR0497xHyKJPAv59M1dkC+rOZCMBJ+t3fZ+twI54=
github.com/mattn/go-runewidth v0.0.7/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/metrue/go-ssh-client v0.0.0-20191125030649-4ac058ee958b h1:JGD0sJ44XzhsT1voOg00zji4ubuMNcVNK3m7d9GI88k=
github.com/metrue/go-ssh-client v0.0.0-20191125030649-4ac058ee958b/go.mod h1:ERHOEBrDy6+8vfoJjjmhdmBpOzdvvP7bLtwYTTK6LOs=
github.com/metrue/go-ssh-client v0.0.0-20191209160027-5773243a8bc9 h1:HHfMhG77ZLn3FOH3AGXW/F5RpAABVH6Fr5mVZZ97S6w=
github.com/metrue/go-ssh-client v0.0.0-20191209160027-5773243a8bc9/go.mod h1:aPG/JtXTyLliKDDlkv+nzHbSbz2p2CBMAjNJRK4uhzY=
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
github.com/mholt/archiver v3.1.1+incompatible h1:1dCVxuqs0dJseYEhi5pl7MYPH9zDa1wBi7mF09cbNkU=
github.com/mholt/archiver v3.1.1+incompatible/go.mod h1:Dh2dOXnSdiLxRiPoVfIr/fI1TwETms9B8CTWfeh7ROU=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mitchellh/mapstructure v1.1.2 h1:fmNYVwqnSfB9mZU6OS2O6GsXM+wcskZDuKQzvN1EDeE=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
@@ -204,13 +187,13 @@ github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3Rllmb
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw=
github.com/nbio/st v0.0.0-20140626010706-e9e8d9816f32 h1:W6apQkHrMkS0Muv8G/TipAy/FJl/rCYT0+EuS8+Z0z4=
github.com/nbio/st v0.0.0-20140626010706-e9e8d9816f32/go.mod h1:9wM+0iRr9ahx58uYLpLIr5fm8diHn0JbqRycJi6w0Ms=
github.com/nwaples/rardecode v1.0.0 h1:r7vGuS5akxOnR4JQSkko62RJ1ReCMXxQRPtxsiFMBOs=
github.com/nwaples/rardecode v1.0.0/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
github.com/olekukonko/tablewriter v0.0.3 h1:i0LBnzgiChAWHJYTQAZJDOgf8MNxAVYZJ2m63SIDimI=
github.com/olekukonko/tablewriter v0.0.3/go.mod h1:YZeBtGzYYEsCHp2LST/u/0NDwGkRoBtmn1cIWCJiS6M=
github.com/olekukonko/tablewriter v0.0.4 h1:vHD/YYe1Wolo78koG299f7V/VAS08c6IpCLn+Ejf/w8=
github.com/olekukonko/tablewriter v0.0.4/go.mod h1:zq6QwlOf5SlnkVbMSr5EoBv3636FWnp+qbPhuoO21uA=
github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
@@ -220,29 +203,23 @@ github.com/opencontainers/go-digest v1.0.0-rc1 h1:WzifXhOVOEOuFYOJAW6aQqW0TooG2i
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVojFA6h/TRcI=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/otiai10/copy v1.0.2 h1:DDNipYy6RkIkjMwy+AWzgKiNTyj2RUI9yEMeETEpVyc=
github.com/otiai10/copy v1.0.2/go.mod h1:c7RpqBkwMom4bYTSkLSym4VSJz/XtncWRAj/J4PEIMY=
github.com/otiai10/curr v0.0.0-20150429015615-9b4961190c95/go.mod h1:9qAhocn7zKJG+0mI8eUu6xqkFDYS2kb2saOteoSB3cE=
github.com/otiai10/mint v1.3.0/go.mod h1:F5AjcsTsWUqX+Na9fpHb52P8pcRX2CI6A3ctIT91xUo=
github.com/pelletier/go-toml v1.2.0 h1:T5zMGML61Wp+FlcbWjRDT7yAxhJNAiPPLOFECq181zc=
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/pelletier/go-toml v1.4.0 h1:u3Z1r+oOXJIkxqw34zVhyPgjBsm6X2wn21NWs/HfSeg=
github.com/pelletier/go-toml v1.4.0/go.mod h1:PN7xzY2wHTK0K9p34ErDQMlFxa51Fk0OUruD3k1mMwo=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2 h1:JhzVVoYvbOACxoUmOs6V/G4D5nPVUW73rKvXxP4XUJc=
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2/go.mod h1:iIss55rKnNBTvrwdmkUpLnDpZoAHvWaiq5+iMmen4AE=
github.com/pierrec/lz4 v0.0.0-20190222153722-062282ea0dcf h1:0d7SseXGaeqFXfRTLbiCkuLhSGEHZyKpz1XD3e5lbSo=
github.com/pierrec/lz4 v0.0.0-20190222153722-062282ea0dcf/go.mod h1:3/3N9NVKO0jef7pBehbT1qWhCMrIgbYNnFAZCqQ5LRc=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/profile v1.2.1/go.mod h1:hJw3o1OdXxsrSjjVksARp5W95eeEaEfptyVZyv6JUPA=
github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
github.com/rogpeppe/fastuuid v1.1.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
github.com/rogpeppe/go-internal v1.1.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.3.0 h1:RR9dF3JtopPvtkroDZuVD7qquD0bnHlKSqaQhgwt8yk=
@@ -254,15 +231,12 @@ github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQD
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/shurcooL/sanitized_anchor_name v1.0.0 h1:PdmoCO6wvbs+7yrJyMORt4/BmY5IYyJwS/kOiWx8mHo=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/smartystreets/assertions v1.0.0/go.mod h1:kHHU4qYBaI3q23Pp3VPrmWhuIUrLW/7eUrw0BU5VaoM=
github.com/smartystreets/go-aws-auth v0.0.0-20180515143844-0c1422d1fdb9/go.mod h1:SnhjPscd9TpLiy1LpzGSKh3bXCfxxXuqd9xmQJy3slM=
github.com/smartystreets/gunit v1.0.0/go.mod h1:qwPWnhz6pn0NnRBP++URONOVyNkPyr4SauJk4cUOwJs=
github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
github.com/spf13/afero v1.2.2 h1:5jhuqJyZCZf2JRofRvN/nIFgIWNzPa3/Vz8mYylgbWc=
github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=
@@ -270,15 +244,13 @@ github.com/spf13/cast v1.3.0 h1:oget//CVOEoFewqQxwr0Ej5yjygnqGkvggSE/gB35Q8=
github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cobra v0.0.5 h1:f0B+LkLX6DtmRH1isoNA9VTtNUK9K8xYd28JNNfOv/s=
github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=
github.com/spf13/jwalterweatherman v1.0.0 h1:XHEdyB+EcvlqZamSM4ZOMGlc93t6AcsBEu9Gc1vn7yk=
github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
github.com/spf13/jwalterweatherman v1.1.0 h1:ue6voC5bR5F8YxI5S67j9i582FU4Qvo2bmqnqMYADFk=
github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=
github.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/viper v1.3.2 h1:VUFqw5KcqRf7i70GOzW7N+Q7+gxVBkSSqiXB12+JQ4M=
github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
github.com/spf13/viper v1.4.0 h1:yXHLWeravcrgGyFSyCgdYpXQ9dR9c/WED3pg1RhxqEU=
github.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -292,7 +264,6 @@ github.com/tj/assert v0.0.0-20171129193455-018094318fb0/go.mod h1:mZ9/Rh9oLWpLLD
github.com/tj/go-elastic v0.0.0-20171221160941-36157cbbebc2/go.mod h1:WjeM0Oo1eNAjXGDx2yma7uG2XoyRZTq1uv3M/o7imD0=
github.com/tj/go-kinesis v0.0.0-20171128231115-08b17f58cb1b/go.mod h1:/yhzCV0xPfx6jb1bBgRFjl5lytqVqZXEaeqWP8lTEao=
github.com/tj/go-spin v1.1.0/go.mod h1:Mg1mzmePZm4dva8Qz60H2lHwmJ2loum4VIrLgVnKwh4=
github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
github.com/ugorji/go v1.1.7 h1:/68gy2h+1mWMrwZFeD1kQialdSzAb432dtpeJ42ovdo=
github.com/ugorji/go v1.1.7/go.mod h1:kZn38zHttfInRq0xu/PH0az30d+z6vm202qpg1oXVMw=
@@ -303,16 +274,12 @@ github.com/ulikunitz/xz v0.5.6 h1:jGHAfXawEGZQ3blwU5wnWKQJvAraT7Ftq9EXjnXYgt8=
github.com/ulikunitz/xz v0.5.6/go.mod h1:2bypXElzHzzJZwzH67Y6wb67pO62Rzfn7BSiF4ABRW8=
github.com/urfave/cli v1.22.1 h1:+mkCCcOFKPnCmVYVcURKps1Xe+3zP90gSYGNfRkjoIY=
github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/urfave/cli v1.22.2 h1:gsqYFH8bb9ekPA12kRo0hfjngWQjkJPlN9R0N78BoUo=
github.com/urfave/cli v1.22.2/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 h1:nIPpBwaJSVYIxUFsDv3M8ofmx9yWTog9BfvIu0q41lo=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8/go.mod h1:HUYIGzjTL3rfEspMxjDjgmT5uz5wzYJKVo23qUhYTos=
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
@@ -330,15 +297,11 @@ golang.org/x/net v0.0.0-20170114055629-f2499483f923/go.mod h1:mL1N/T3taQHkDXs73r
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190522155817-f3200d17e092 h1:4QSRKanuywn15aTZvI/mIDEgPQpswuFndXpOj3rKEco=
golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859 h1:R/3boaszxrf1GEUWTVDzSKVwLmSJpwZ1yqXm8j0v2QI=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190812203447-cdfb69ac37fc h1:gkKoSkUmnU6bpS/VhkuO27bzQeSA51uaEfbOW5dNb68=
@@ -358,8 +321,6 @@ golang.org/x/sys v0.0.0-20170830134202-bb24a47a89ea/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190209173611-3b5209105503/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -379,7 +340,6 @@ golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4 h1:SvFZT6jyqRaOeXpc5h/JSfZenJ2O330aBsf7JfSUXmQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20181011042414-1f849cf54d09/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -388,6 +348,7 @@ golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190624180213-70d37148ca0c h1:KfpJVdWhuRqNk4XVXzjXf2KAV4TBEP77SYdFGjeGuIE=
golang.org/x/tools v0.0.0-20190624180213-70d37148ca0c/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
@@ -401,7 +362,6 @@ google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRn
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.21.0 h1:G+97AoqBnmZIT91cLG/EkCoK9NSelj64P8bOHHNmGn0=
google.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -411,21 +371,22 @@ gopkg.in/go-playground/assert.v1 v1.2.1 h1:xoYuJVE7KT85PYWrN730RguIQO0ePzVRfFMXa
gopkg.in/go-playground/assert.v1 v1.2.1/go.mod h1:9RXL0bg/zibRAgZUYszZSwO/z8Y/a8bDuhia5mkpMnE=
gopkg.in/go-playground/validator.v8 v8.18.2 h1:lFB4DoMU6B626w8ny76MV7VX6W2VHct2GVOI3xgiMrQ=
gopkg.in/go-playground/validator.v8 v8.18.2/go.mod h1:RX2a/7Ha8BgOhfk7j780h4/u/RRjR0eouCJSH80/M2Y=
gopkg.in/h2non/gock.v1 v1.0.15 h1:SzLqcIlb/fDfg7UvukMpNcWsu7sI5tWwL+KCATZqks0=
gopkg.in/h2non/gock.v1 v1.0.15/go.mod h1:sX4zAkdYX1TRGJ2JY156cFspQn4yRWn6p9EMdODlynE=
gopkg.in/inf.v0 v0.9.0 h1:3zYtXIO92bvsdS3ggAdA8Gb4Azj0YU+TVY1uGYNFA8o=
gopkg.in/inf.v0 v0.9.0/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.7 h1:VUgggvou5XRW9mHwD/yXxIYSMtY0zoKQf/v226p2nyo=
gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gotest.tools v2.2.0+incompatible h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo=
gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a h1:/8zB6iBfHCl1qAnEAWwGPNrUvapuy6CPla1VM0k8hQw=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
k8s.io/api v0.0.0-20190925180651-d58b53da08f5 h1:PEYuamj4laOODrvrh/KIKxihqE8kAnxFRZ6kKtrAS8c=
k8s.io/api v0.0.0-20190925180651-d58b53da08f5/go.mod h1:blPYY5r6fKug8SVOnjDtFAlzZzInCRL9NNls66SFhFI=

13
hack/install_docker.sh Executable file
View File

@@ -0,0 +1,13 @@
#!/bin/bash
set -e
# ++
# verified on Ubuntu 16.04 x64
# ++
apt-get remove -y docker docker-engine docker.io containerd runc
apt-get update -y
apt-get install -y apt-transport-https ca-certificates curl software-properties-common lsb-core
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
apt-get update -y
apt-get install -y docker-ce
docker run hello-world

View File

@@ -1,52 +1,11 @@
package handlers
import (
"io/ioutil"
"strings"
"github.com/apex/log"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
api "github.com/metrue/fx/container_runtimes/docker/http"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
"github.com/urfave/cli"
"github.com/metrue/fx/context"
)
// Call command handle
func Call(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) error {
params := strings.Join(ctx.Args()[1:], " ")
hosts, err := cfg.ListActiveMachines()
if err != nil {
log.Fatalf("list active machines failed: %v", err)
}
file := ctx.Args().First()
src, err := ioutil.ReadFile(file)
if err != nil {
log.Fatalf("Read Source: %v", err)
return err
}
log.Info("Read Source: \u2713")
lang := utils.GetLangFromFileName(file)
fn := types.Func{
Language: lang,
Source: string(src),
}
project, err := packer.Pack(file, fn)
if err != nil {
panic(err)
}
for name, host := range hosts {
if err := api.MustCreate(host.Host, constants.AgentPort).
Call(file, params, project); err != nil {
log.Fatalf("call functions on machine %s with %v failed: %v", name, params, err)
}
}
return nil
}
func Call(ctx context.Contexter) error {
// TODO not supported
return nil
}

View File

@@ -1,28 +1,26 @@
package handlers
import (
"os"
"github.com/apex/log"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/context"
"github.com/metrue/fx/doctor"
"github.com/urfave/cli"
)
// Doctor command handle
func Doctor(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) error {
hosts, err := cfg.ListMachines()
if err != nil {
log.Fatalf("list machines failed %v", err)
return nil
}
for name, h := range hosts {
if err := doctor.New(h).Start(); err != nil {
log.Warnf("machine %s is in dirty state: %v", name, err)
} else {
log.Infof("machine %s is in healthy state: %s", name, constants.CheckedSymbol)
}
}
return nil
func Doctor(ctx context.Contexter) error {
host := os.Getenv("DOCKER_REMOTE_HOST_ADDR")
user := os.Getenv("DOCKER_REMOTE_HOST_USER")
password := os.Getenv("DOCKER_REMOTE_HOST_PASSWORD")
if host == "" {
host = "localhost"
}
if err := doctor.New(host, user, password).Start(); err != nil {
log.Warnf("machine %s is in dirty state: %v", host, err)
} else {
log.Infof("machine %s is in healthy state: %s", host, constants.CheckedSymbol)
}
return nil
}

View File

@@ -1,38 +1,18 @@
package handlers
import (
"context"
"os"
"github.com/metrue/fx/config"
"github.com/metrue/fx/deploy"
dockerDeployer "github.com/metrue/fx/deploy/docker"
k8sDeployer "github.com/metrue/fx/deploy/kubernetes"
"github.com/urfave/cli"
"github.com/metrue/fx/context"
"github.com/metrue/fx/infra"
)
// Down command handle
func Down(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) (err error) {
services := ctx.Args()
c := context.Background()
var runner deploy.Deployer
if os.Getenv("KUBECONFIG") != "" {
runner, err = k8sDeployer.Create()
if err != nil {
return err
}
} else {
runner, err = dockerDeployer.CreateClient(c)
if err != nil {
return err
}
func Down(ctx context.Contexter) (err error) {
services := ctx.Get("services").([]string)
runner := ctx.Get("deployer").(infra.Deployer)
for _, svc := range services {
if err := runner.Destroy(ctx.GetContext(), svc); err != nil {
return err
}
for _, svc := range services {
if err := runner.Destroy(c, svc); err != nil {
return err
}
}
return nil
}
return nil
}

27
handlers/down_test.go Normal file
View File

@@ -0,0 +1,27 @@
package handlers
import (
"context"
"testing"
"github.com/golang/mock/gomock"
mockCtx "github.com/metrue/fx/context/mocks"
mockDeployer "github.com/metrue/fx/infra/mocks"
)
func TestDown(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
ctx := mockCtx.NewMockContexter(ctrl)
deployer := mockDeployer.NewMockDeployer(ctrl)
services := []string{"sample-name"}
ctx.EXPECT().Get("services").Return(services)
ctx.EXPECT().Get("deployer").Return(deployer)
ctx.EXPECT().GetContext().Return(context.Background())
deployer.EXPECT().Destroy(gomock.Any(), services[0]).Return(nil)
if err := Down(ctx); err != nil {
t.Fatal(err)
}
}

View File

@@ -1,6 +0,0 @@
package handlers
import "github.com/urfave/cli"
// HandleFunc command handle function
type HandleFunc func(ctx *cli.Context) error

View File

@@ -1,56 +0,0 @@
package handlers
import (
"log"
"github.com/metrue/fx/config"
"github.com/metrue/fx/utils"
"github.com/urfave/cli"
)
// AddHost add a host
func AddHost(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) error {
name := ctx.String("name")
addr := ctx.String("host")
user := ctx.String("user")
password := ctx.String("password")
host := config.NewHost(addr, user, password)
if !host.Valid() {
log.Fatalf("invaid host %v", host)
return nil
}
if host.IsRemote() {
if host.User == "" || host.Password == "" {
log.Fatalf("the host to add is a remote, user and password for SSH login is required")
return nil
}
}
return cfg.AddMachine(name, host)
}
}
// RemoveHost remove a host
func RemoveHost(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) error {
name := ctx.Args().First()
if name == "" {
log.Fatalf("no name given: fx infra remove <name>")
return nil
}
return cfg.RemoveHost(name)
}
}
// ListHosts list hosts
func ListHosts(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) error {
hosts, err := cfg.ListMachines()
if err != nil {
return err
}
return utils.OutputJSON(hosts)
}
}

View File

@@ -2,105 +2,74 @@ package handlers
import (
"fmt"
"io/ioutil"
"os"
"time"
"github.com/apex/log"
"github.com/google/uuid"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
api "github.com/metrue/fx/container_runtimes/docker/http"
containerruntimes "github.com/metrue/fx/container_runtimes"
"github.com/metrue/fx/context"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/provision"
"github.com/metrue/fx/types"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/utils"
"github.com/pkg/errors"
"github.com/urfave/cli"
"github.com/otiai10/copy"
)
// BuildImage build image
func BuildImage(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) error {
funcFile := ctx.Args().First()
tag := ctx.String("tag")
if tag == "" {
tag = uuid.New().String()
}
func BuildImage(ctx context.Contexter) (err error) {
spinner.Start("building")
defer func() {
spinner.Stop("building", err)
}()
workdir := fmt.Sprintf("/tmp/fx-%d", time.Now().Unix())
defer os.RemoveAll(workdir)
body, err := ioutil.ReadFile(funcFile)
if err != nil {
log.Fatalf("function code load failed: %v", err)
return err
}
log.Infof("function code loaded: %v", constants.CheckedSymbol)
lang := utils.GetLangFromFileName(funcFile)
pwd, err := os.Getwd()
if err != nil {
log.Fatalf("could not get current work directory: %v", err)
return err
}
tarFile := fmt.Sprintf("%s.%s.tar", pwd, tag)
defer os.RemoveAll(tarFile)
sources := ctx.Get("sources").([]string)
if err := packer.PackIntoTar(types.Func{Language: lang, Source: string(body)}, tarFile); err != nil {
log.Fatalf("could not pack function: %v", err)
return err
}
log.Infof("function packed: %v", constants.CheckedSymbol)
hosts, err := cfg.ListActiveMachines()
if err != nil {
log.Fatalf("could not list active machine: %v", err)
return errors.Wrap(err, "list active machines failed")
}
if len(hosts) == 0 {
log.Warnf("no active machines")
return nil
}
for n, host := range hosts {
if !host.Provisioned {
provisionor := provision.New(host)
if err := provisionor.Start(); err != nil {
return errors.Wrapf(err, "could not provision %s", n)
}
log.Infof("provision machine %v: %s", n, constants.CheckedSymbol)
if err := cfg.UpdateProvisionedStatus(n, true); err != nil {
return errors.Wrap(err, "update machine provision status failed")
}
}
if err := api.MustCreate(host.Host, constants.AgentPort).
BuildImage(tarFile, tag, map[string]string{}); err != nil {
return err
}
log.Infof("image built on machine %s: %v", n, constants.CheckedSymbol)
}
return nil
if len(sources) == 0 {
return fmt.Errorf("source file/directory of function required")
}
if len(sources) == 1 &&
utils.IsDir(sources[0]) &&
utils.HasDockerfile(sources[0]) {
if err := copy.Copy(sources[0], workdir); err != nil {
return err
}
} else {
if err := packer.Pack(workdir, sources...); err != nil {
return err
}
}
docker := ctx.Get("docker").(containerruntimes.ContainerRuntime)
nameWithTag := ctx.Get("tag").(string) + ":latest"
if err := docker.BuildImage(ctx.GetContext(), workdir, nameWithTag); err != nil {
return err
}
log.Infof("image built: %s %v", nameWithTag, constants.CheckedSymbol)
return nil
}
// ExportImage export service's code into a directory
func ExportImage() HandleFunc {
return func(ctx *cli.Context) (err error) {
funcFile := ctx.Args().First()
outputDir := ctx.String("output")
if outputDir == "" {
log.Fatalf("output directory required")
return nil
}
func ExportImage(ctx context.Contexter) (err error) {
outputDir := ctx.Get("output").(string)
sources := ctx.Get("sources").([]string)
body, err := ioutil.ReadFile(funcFile)
if err != nil {
return errors.Wrap(err, "read source failed")
}
lang := utils.GetLangFromFileName(funcFile)
if err := packer.PackIntoDir(types.Func{Language: lang, Source: string(body)}, outputDir); err != nil {
log.Fatalf("write source code to file failed: %v", constants.UncheckedSymbol)
if len(sources) == 0 {
return fmt.Errorf("source file/directory of function required")
}
if len(sources) == 1 &&
utils.IsDir(sources[0]) &&
utils.HasDockerfile(sources[0]) {
if err := copy.Copy(sources[0], outputDir); err != nil {
return err
}
} else {
if err := packer.Pack(outputDir, sources...); err != nil {
return err
}
log.Infof("exported to %v: %v", outputDir, constants.CheckedSymbol)
return nil
}
log.Infof("exported to %v: %v", outputDir, constants.CheckedSymbol)
return nil
}

107
handlers/infra.go Normal file
View File

@@ -0,0 +1,107 @@
package handlers
import (
"fmt"
"strings"
"github.com/metrue/fx/config"
"github.com/metrue/fx/context"
dockerInfra "github.com/metrue/fx/infra/docker"
k8sInfra "github.com/metrue/fx/infra/k8s"
"github.com/metrue/fx/pkg/spinner"
)
func setupK8S(masterInfo string, agentsInfo string) ([]byte, error) {
info := strings.Split(masterInfo, "@")
if len(info) != 2 {
return nil, fmt.Errorf("incorrect master info, should be <user>@<ip> format")
}
master, err := k8sInfra.CreateNode(info[1], info[0], "k3s_master", "master")
if err != nil {
return nil, err
}
nodes := []k8sInfra.Noder{master}
if agentsInfo != "" {
agentsInfoList := strings.Split(agentsInfo, ",")
for idx, agent := range agentsInfoList {
info := strings.Split(agent, "@")
if len(info) != 2 {
return nil, fmt.Errorf("incorrect agent info, should be <user>@<ip> format")
}
node, err := k8sInfra.CreateNode(info[1], info[0], "k3s_agent", fmt.Sprintf("agent-%d", idx))
if err != nil {
return nil, err
}
nodes = append(nodes, node)
}
}
cloud := k8sInfra.NewCloud(nodes...)
if err := cloud.Provision(); err != nil {
return nil, err
}
return cloud.Dump()
}
func setupDocker(hostInfo string, name string) ([]byte, error) {
info := strings.Split(hostInfo, "@")
if len(info) != 2 {
return nil, fmt.Errorf("incorrect master info, should be <user>@<ip> format")
}
user := info[0]
host := info[1]
cloud, err := dockerInfra.Create(host, user, name)
if err != nil {
return nil, err
}
if err := cloud.Provision(); err != nil {
return nil, err
}
return cloud.Dump()
}
// Setup infra
func Setup(ctx context.Contexter) (err error) {
const task = "setup infra"
spinner.Start(task)
defer func() {
spinner.Stop(task, err)
}()
cli := ctx.GetCliContext()
typ := cli.String("type")
name := cli.String("name")
if name == "" {
return fmt.Errorf("name required")
}
if typ == "docker" {
if cli.String("host") == "" {
return fmt.Errorf("host required, eg. 'root@123.1.2.12'")
}
} else if typ == "k8s" {
if cli.String("master") == "" {
return fmt.Errorf("master required, eg. 'root@123.1.2.12'")
}
} else {
return fmt.Errorf("invalid type, 'docker' and 'k8s' support")
}
fxConfig := ctx.Get("config").(*config.Config)
switch strings.ToLower(typ) {
case "k8s":
kubeconf, err := setupK8S(cli.String("master"), cli.String("agents"))
if err != nil {
return err
}
return fxConfig.AddCloud(name, kubeconf)
case "docker":
config, err := setupDocker(cli.String("host"), name)
if err != nil {
return err
}
return fxConfig.AddCloud(name, config)
}
return nil
}

View File

@@ -1,46 +0,0 @@
package handlers
import (
"github.com/apex/log"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/provision"
"github.com/urfave/cli"
)
// Activate a machine to be fx server
func Activate(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) error {
name := ctx.Args().First()
if name == "" {
log.Fatalf("name required for: fx infra activate <name>")
return nil
}
host, err := cfg.GetMachine(name)
if err != nil {
log.Fatalf("could get host %v, make sure you add it first", err)
log.Info("You can add a machine by: \n fx infra add -Name <name> -H <ip or hostname> -U <user> -P <password>")
return nil
}
if !host.Provisioned {
provisionor := provision.New(host)
if err := provisionor.Start(); err != nil {
log.Fatalf("could not provision %s: %v", name, err)
return nil
}
log.Infof("provision machine %v: %s", name, constants.CheckedSymbol)
if err := cfg.UpdateProvisionedStatus(name, true); err != nil {
log.Fatalf("update machine provision status failed: %v", err)
}
}
if err := cfg.EnableMachine(name); err != nil {
log.Fatalf("could not enable %s: %v", name, err)
return nil
}
log.Infof("enble machine %v: %s", name, constants.CheckedSymbol)
return nil
}
}

View File

@@ -1,25 +0,0 @@
package handlers
import (
"github.com/apex/log"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
"github.com/urfave/cli"
)
// Deactivate a machine
func Deactivate(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) error {
name := ctx.Args().First()
if name == "" {
log.Fatalf("name required for: fx infra activate <name>")
return nil
}
if err := cfg.DisableMachine(name); err != nil {
log.Fatalf("could not disable %s: %v", name, err)
return nil
}
log.Infof("machine %s deactive: %v", name, constants.CheckedSymbol)
return nil
}
}

View File

@@ -1,25 +1,21 @@
package handlers
import (
"github.com/apex/log"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
api "github.com/metrue/fx/container_runtimes/docker/http"
"github.com/urfave/cli"
"github.com/metrue/fx/context"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/pkg/render"
)
// List command handle
func List(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) error {
hosts, err := cfg.ListActiveMachines()
if err != nil {
log.Fatalf("list active machines failed: %v", err)
}
for name, host := range hosts {
if err := api.MustCreate(host.Host, constants.AgentPort).List(ctx.Args().First()); err != nil {
log.Fatalf("list functions on machine %s failed: %v", name, err)
}
}
return nil
func List(ctx context.Contexter) (err error) {
cli := ctx.GetCliContext()
deployer := ctx.Get("deployer").(infra.Deployer)
services, err := deployer.List(ctx.GetContext(), cli.Args().First())
if err != nil {
return err
}
render.Table(services)
return nil
}

19
handlers/list_infra.go Normal file
View File

@@ -0,0 +1,19 @@
package handlers
import (
"fmt"
"github.com/metrue/fx/config"
"github.com/metrue/fx/context"
)
// ListInfra list infra
func ListInfra(ctx context.Contexter) (err error) {
fxConfig := ctx.Get("config").(*config.Config)
conf, err := fxConfig.View()
if err != nil {
return err
}
fmt.Println(string(conf))
return nil
}

View File

@@ -1,94 +1,40 @@
package handlers
import (
"context"
"fmt"
"io/ioutil"
"os"
"github.com/apex/log"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/deploy"
dockerDeployer "github.com/metrue/fx/deploy/docker"
k8sDeployer "github.com/metrue/fx/deploy/kubernetes"
"github.com/metrue/fx/context"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/pkg/render"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
"github.com/pkg/errors"
"github.com/urfave/cli"
)
// PortRange usable port range https: //en.wikipedia.org/wiki/Ephemeral_port
var PortRange = struct {
min int
max int
}{
min: 1023,
max: 65535,
}
// Up command handle
func Up(cfg config.Configer) HandleFunc {
return func(ctx *cli.Context) (err error) {
funcFile := ctx.Args().First()
name := ctx.String("name")
port := ctx.Int("port")
defer func() {
if r := recover(); r != nil {
log.Fatalf("fatal error happened: %v", r)
}
if err != nil {
log.Fatalf("deploy function %s (%s) failed: %v", err)
}
log.Infof("function %s (%s) deployed successfully", name, funcFile)
}()
if port < PortRange.min || port > PortRange.max {
return fmt.Errorf("invalid port number: %d, port number should in range of %d - %d", port, PortRange.min, PortRange.max)
}
body, err := ioutil.ReadFile(funcFile)
if err != nil {
return errors.Wrap(err, "read source failed")
}
lang := utils.GetLangFromFileName(funcFile)
var deployer deploy.Deployer
var bindings []types.PortBinding
if os.Getenv("KUBECONFIG") != "" {
deployer, err = k8sDeployer.Create()
if err != nil {
return err
}
bindings = []types.PortBinding{
types.PortBinding{
ServiceBindingPort: 80,
ContainerExposePort: constants.FxContainerExposePort,
},
types.PortBinding{
ServiceBindingPort: 443,
ContainerExposePort: constants.FxContainerExposePort,
},
}
} else {
bctx := context.Background()
deployer, err = dockerDeployer.CreateClient(bctx)
if err != nil {
return err
}
bindings = []types.PortBinding{
types.PortBinding{
ServiceBindingPort: int32(port),
ContainerExposePort: constants.FxContainerExposePort,
},
}
}
return deployer.Deploy(
context.Background(),
types.Func{Language: lang, Source: string(body)},
name,
bindings,
)
func Up(ctx context.Contexter) (err error) {
fn, ok := ctx.Get("data").(string)
if !ok {
fn = ""
}
image, ok := ctx.Get("image").(string)
if !ok {
image = ""
}
name := ctx.Get("name").(string)
deployer := ctx.Get("deployer").(infra.Deployer)
bindings := ctx.Get("bindings").([]types.PortBinding)
if err := deployer.Deploy(
ctx.GetContext(),
fn,
name,
image,
bindings,
); err != nil {
return err
}
service, err := deployer.GetStatus(ctx.GetContext(), name)
if err != nil {
return err
}
render.Table([]types.Service{service})
return nil
}

View File

@@ -1,12 +1,40 @@
package handlers
import (
"context"
"testing"
"github.com/golang/mock/gomock"
mockCtx "github.com/metrue/fx/context/mocks"
mockDeployer "github.com/metrue/fx/infra/mocks"
"github.com/metrue/fx/types"
)
func TestUp(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
ctx := mockCtx.NewMockContexter(ctrl)
deployer := mockDeployer.NewMockDeployer(ctrl)
bindings := []types.PortBinding{}
name := "sample-name"
image := "sample-image"
data := "sample-data"
ctx.EXPECT().Get("name").Return(name)
ctx.EXPECT().Get("image").Return(image)
ctx.EXPECT().Get("deployer").Return(deployer)
ctx.EXPECT().Get("bindings").Return(bindings)
ctx.EXPECT().Get("data").Return(data)
ctx.EXPECT().GetContext().Return(context.Background()).Times(2)
deployer.EXPECT().Deploy(gomock.Any(), data, name, image, bindings).Return(nil)
deployer.EXPECT().GetStatus(gomock.Any(), name).Return(types.Service{
ID: "id-1",
Name: name,
Host: "127.0.0.1",
Port: 2100,
}, nil)
if err := Up(ctx); err != nil {
t.Fatal(err)
}
}

13
handlers/use_infra.go Normal file
View File

@@ -0,0 +1,13 @@
package handlers
import (
"github.com/metrue/fx/config"
"github.com/metrue/fx/context"
)
// UseInfra use infra
func UseInfra(ctx context.Contexter) error {
fxConfig := ctx.Get("config").(*config.Config)
cli := ctx.GetCliContext()
return fxConfig.UseCloud(cli.Args().First())
}

140
infra/docker/cloud.go Normal file
View File

@@ -0,0 +1,140 @@
package docker
import (
"encoding/json"
"os"
"path/filepath"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/types"
"github.com/metrue/go-ssh-client"
"github.com/mitchellh/go-homedir"
)
// Cloud define a docker host
type Cloud struct {
IP string `json:"ip"`
User string `json:"user"`
Name string `json:"name"`
Type string `json:"type"`
sshClient ssh.Clienter
}
// New new a docker cloud
func New(ip string, user string, name string) *Cloud {
return &Cloud{
IP: ip,
User: user,
Name: name,
Type: types.CloudTypeDocker,
}
}
// Create a docker node
func Create(ip string, user string, name string) (*Cloud, error) {
key, err := sshkey()
if err != nil {
return nil, err
}
port := sshport()
sshClient := ssh.New(ip).WithUser(user).WithKey(key).WithPort(port)
return &Cloud{
IP: ip,
User: user,
Name: name,
Type: types.CloudTypeDocker,
sshClient: sshClient,
}, nil
}
// Load a docker node from meta
func Load(meta []byte) (*Cloud, error) {
var cloud Cloud
if err := json.Unmarshal(meta, &cloud); err != nil {
return nil, err
}
key, err := sshkey()
if err != nil {
return nil, err
}
port := sshport()
sshClient := ssh.New(cloud.IP).WithUser(cloud.User).WithKey(key).WithPort(port)
cloud.sshClient = sshClient
return &cloud, nil
}
// Provision a host
func (c *Cloud) Provision() error {
if err := c.sshClient.RunCommand(infra.Scripts["docker_version"].(string), ssh.CommandOptions{}); err != nil {
if err := c.sshClient.RunCommand(infra.Scripts["install_docker"].(string), ssh.CommandOptions{}); err != nil {
return err
}
if err := c.sshClient.RunCommand(infra.Scripts["start_dockerd"].(string), ssh.CommandOptions{}); err != nil {
return err
}
}
if err := c.sshClient.RunCommand(infra.Scripts["check_fx_agent"].(string), ssh.CommandOptions{}); err != nil {
if err := c.sshClient.RunCommand(infra.Scripts["start_fx_agent"].(string), ssh.CommandOptions{}); err != nil {
return err
}
}
return nil
}
// GetType cloud type
func (c *Cloud) GetType() string {
return c.Type
}
func (c *Cloud) GetConfig() (string, error) {
data, err := json.Marshal(c)
if err != nil {
return "", err
}
return string(data), nil
}
func (c *Cloud) Dump() ([]byte, error) {
return json.Marshal(c)
}
// NOTE only using for unit testing
func (c *Cloud) setsshClient(client ssh.Clienter) {
c.sshClient = client
}
// NOTE the reason putting sshkey() and sshport here inside node.go is because
// ssh key and ssh port is related to node it self, we may extend this in future
func sshkey() (string, error) {
path := os.Getenv("SSH_KEY_FILE")
if path != "" {
absPath, err := filepath.Abs(path)
if err != nil {
return "", err
}
return absPath, nil
}
key, err := homedir.Expand("~/.ssh/id_rsa")
if err != nil {
return "", err
}
return key, nil
}
func sshport() string {
port := os.Getenv("SSH_PORT")
if port != "" {
return port
}
return "22"
}
var (
_ infra.Clouder = &Cloud{}
)

103
infra/docker/cloud_test.go Normal file
View File

@@ -0,0 +1,103 @@
package docker
import (
"fmt"
"os"
"testing"
"github.com/golang/mock/gomock"
"github.com/metrue/fx/infra"
"github.com/metrue/go-ssh-client"
sshMocks "github.com/metrue/go-ssh-client/mocks"
"github.com/mitchellh/go-homedir"
)
func TestCloud(t *testing.T) {
t.Run("fx agent started", func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
n := New("127.0.0.1", "fx", "master")
sshClient := sshMocks.NewMockClienter(ctrl)
n.setsshClient(sshClient)
sshClient.EXPECT().RunCommand(infra.Scripts["docker_version"].(string), ssh.CommandOptions{}).Return(nil)
sshClient.EXPECT().RunCommand(infra.Scripts["check_fx_agent"].(string), ssh.CommandOptions{}).Return(nil)
if err := n.Provision(); err != nil {
t.Fatal(err)
}
})
t.Run("fx agent not started", func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
n := New("127.0.0.1", "fx", "master")
sshClient := sshMocks.NewMockClienter(ctrl)
n.setsshClient(sshClient)
sshClient.EXPECT().RunCommand(infra.Scripts["docker_version"].(string), ssh.CommandOptions{}).Return(nil)
sshClient.EXPECT().RunCommand(infra.Scripts["check_fx_agent"].(string), ssh.CommandOptions{}).Return(fmt.Errorf("no such container"))
sshClient.EXPECT().RunCommand(infra.Scripts["start_fx_agent"].(string), ssh.CommandOptions{}).Return(nil)
if err := n.Provision(); err != nil {
t.Fatal(err)
}
})
t.Run("docker not installed and fx agent not started", func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
n := New("127.0.0.1", "fx", "master")
sshClient := sshMocks.NewMockClienter(ctrl)
n.setsshClient(sshClient)
sshClient.EXPECT().RunCommand(infra.Scripts["docker_version"].(string), ssh.CommandOptions{}).Return(fmt.Errorf("no such command"))
sshClient.EXPECT().RunCommand(infra.Scripts["install_docker"].(string), ssh.CommandOptions{}).Return(nil)
sshClient.EXPECT().RunCommand(infra.Scripts["start_dockerd"].(string), ssh.CommandOptions{}).Return(nil)
sshClient.EXPECT().RunCommand(infra.Scripts["check_fx_agent"].(string), ssh.CommandOptions{}).Return(fmt.Errorf("no such container"))
sshClient.EXPECT().RunCommand(infra.Scripts["start_fx_agent"].(string), ssh.CommandOptions{}).Return(nil)
if err := n.Provision(); err != nil {
t.Fatal(err)
}
})
}
func TestGetSSHKeyFile(t *testing.T) {
t.Run("defaut", func(t *testing.T) {
defau, err := sshkey()
if err != nil {
t.Fatal(err)
}
defaultPath, _ := homedir.Expand("~/.ssh/id_rsa")
if defau != defaultPath {
t.Fatalf("should get %s but got %s", defaultPath, defau)
}
})
t.Run("override from env", func(t *testing.T) {
os.Setenv("SSH_KEY_FILE", "/tmp/id_rsa")
keyFile, err := sshkey()
if err != nil {
t.Fatal(err)
}
if keyFile != "/tmp/id_rsa" {
t.Fatalf("should get %s but got %s", "/tmp/id_rsa", keyFile)
}
})
}
func TestGetSSHPort(t *testing.T) {
t.Run("defaut", func(t *testing.T) {
defau := sshport()
if defau != "22" {
t.Fatalf("should get %s but got %s", "22", defau)
}
})
t.Run("override from env", func(t *testing.T) {
os.Setenv("SSH_PORT", "2222")
defau := sshport()
if defau != "2222" {
t.Fatalf("should get %s but got %s", "2222", defau)
}
})
}

101
infra/docker/deployer.go Normal file
View File

@@ -0,0 +1,101 @@
package docker
import (
"context"
"strconv"
dockerTypes "github.com/docker/docker/api/types"
containerruntimes "github.com/metrue/fx/container_runtimes"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/types"
)
// Deployer manage container
type Deployer struct {
cli containerruntimes.ContainerRuntime
}
// CreateClient create a docker instance
func CreateClient(client containerruntimes.ContainerRuntime) (d *Deployer, err error) {
return &Deployer{cli: client}, nil
}
// Deploy create a Docker container from given image, and bind the constants.FxContainerExposePort to given port
func (d *Deployer) Deploy(ctx context.Context, fn string, name string, image string, ports []types.PortBinding) (err error) {
spinner.Start("deploying " + name)
defer func() {
spinner.Stop("deploying "+name, err)
}()
return d.cli.StartContainer(ctx, name, image, ports)
}
// Update a container
func (d *Deployer) Update(ctx context.Context, name string) error {
return nil
}
// Destroy stop and remove container
func (d *Deployer) Destroy(ctx context.Context, name string) (err error) {
spinner.Start("destroying " + name)
defer func() {
spinner.Stop("destroying "+name, err)
}()
return d.cli.StopContainer(ctx, name)
}
// GetStatus get a service status
func (d *Deployer) GetStatus(ctx context.Context, name string) (types.Service, error) {
var container dockerTypes.ContainerJSON
if err := d.cli.InspectContainer(ctx, name, &container); err != nil {
return types.Service{}, err
}
service := types.Service{
ID: container.ID,
Name: container.Name,
}
for _, bindings := range container.NetworkSettings.Ports {
if len(bindings) > 0 {
binding := bindings[0]
port, err := strconv.Atoi(binding.HostPort)
if err != nil {
return service, err
}
service.Port = port
service.Host = binding.HostIP
service.State = container.State.Status
service.Image = container.Image
break
}
if service.Port != 0 && service.Host != "" {
break
}
}
return service, nil
}
// Ping check healty status of infra
func (d *Deployer) Ping(ctx context.Context) error {
if _, err := d.cli.Version(ctx); err != nil {
return err
}
return nil
}
// List services
func (d *Deployer) List(ctx context.Context, name string) (svcs []types.Service, err error) {
const task = "listing"
spinner.Start(task)
defer func() {
spinner.Stop(task, err)
}()
// FIXME support remote host
return d.cli.ListContainer(ctx, name)
}
var (
_ infra.Deployer = &Deployer{}
)

View File

@@ -0,0 +1,43 @@
package docker
import (
"testing"
)
func TestDocker(t *testing.T) {
// ctx := context.Background()
// cli, err := CreateClient(ctx)
// if err != nil {
// t.Fatal(err)
// }
//
// name := "helloworld"
// bindings := []types.PortBinding{
// types.PortBinding{
// ServiceBindingPort: 80,
// ContainerExposePort: 3000,
// },
// types.PortBinding{
// ServiceBindingPort: 443,
// ContainerExposePort: 3000,
// },
// }
//
// fn := types.Func{
// Language: "node",
// Source: `
// module.exports = (ctx) => {
// ctx.body = 'hello world'
// }
// `,
// }
// if err := cli.Deploy(ctx, fn, name, name, bindings); err != nil {
// t.Fatal(err)
// }
//
// time.Sleep(1 * time.Second)
//
// if err := cli.Destroy(ctx, name); err != nil {
// t.Fatal(err)
// }
}

8
infra/docker/docker.go Normal file
View File

@@ -0,0 +1,8 @@
package docker
import containerruntimes "github.com/metrue/fx/container_runtimes"
// CreateDeployer create a deployer
func CreateDeployer(client containerruntimes.ContainerRuntime) (*Deployer, error) {
return &Deployer{cli: client}, nil
}

30
infra/infra.go Normal file
View File

@@ -0,0 +1,30 @@
package infra
import (
"context"
"github.com/metrue/fx/types"
)
// Clouder cloud interface
type Clouder interface {
Provision() error
GetConfig() (string, error)
GetType() string
Dump() ([]byte, error)
}
// Deployer deploy interface
type Deployer interface {
Deploy(ctx context.Context, fn string, name string, image string, bindings []types.PortBinding) error
Destroy(ctx context.Context, name string) error
Update(ctx context.Context, name string) error
GetStatus(ctx context.Context, name string) (types.Service, error)
List(ctx context.Context, name string) ([]types.Service, error)
Ping(ctx context.Context) error
}
// Infra infrastructure provision interface
type Infra interface {
Deployer
}

262
infra/k8s/cloud.go Normal file
View File

@@ -0,0 +1,262 @@
package k8s
import (
"encoding/json"
"fmt"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/types"
)
// Cloud define a cloud
type Cloud struct {
Config string `json:"config"`
URL string `json:"url"`
Token string `json:"token"`
Type string `json:"type"`
Nodes map[string]Noder `json:"nodes"`
}
// Load a cloud from config
func Load(meta []byte, messup ...func(n Noder) (Noder, error)) (*Cloud, error) {
var cloud Cloud
if err := json.Unmarshal(meta, &cloud); err != nil {
return nil, err
}
for name, n := range cloud.Nodes {
// NOTE messup function is just for unit testing
// we use it to replace the real not with mock node
if len(messup) > 0 {
node, err := messup[0](n)
if err != nil {
return nil, err
}
cloud.Nodes[name] = node
}
}
return &cloud, nil
}
// NewCloud new a cloud
func NewCloud(node ...Noder) *Cloud {
nodes := map[string]Noder{}
for _, n := range node {
nodes[n.GetName()] = n
}
return &Cloud{
Type: types.CloudTypeK8S,
Nodes: nodes,
}
}
// Provision provision cloud
func (c *Cloud) Provision() error {
var master Noder
agents := []Noder{}
for _, n := range c.Nodes {
if n.GetType() == NodeTypeMaster {
master = n
} else {
agents = append(agents, n)
}
}
// when it's k3s cluster
if master != nil {
c.URL = fmt.Sprintf("https://%s:6443", master.GetIP())
if err := master.Provision(map[string]string{}); err != nil {
return err
}
tok, err := master.GetToken()
if err != nil {
return err
}
c.Token = tok
config, err := master.GetConfig()
if err != nil {
return err
}
c.Config = config
}
// when it's a docker agent
if len(agents) == 1 && agents[0].GetType() == NodeTypeDocker {
config, err := agents[0].GetConfig()
if err != nil {
return err
}
c.Config = config
}
if len(agents) > 0 {
errCh := make(chan error, len(agents))
defer close(errCh)
for _, agent := range agents {
go func(node Noder) {
errCh <- node.Provision(map[string]string{
"url": c.URL,
"token": c.Token,
})
}(agent)
}
for range agents {
err := <-errCh
if err != nil {
return err
}
}
}
return nil
}
// AddNode a node
func (c *Cloud) AddNode(n Noder, skipProvision bool) error {
if !skipProvision {
if err := n.Provision(map[string]string{
"url": c.URL,
"token": c.Token,
}); err != nil {
return err
}
}
c.Nodes[n.GetName()] = n
return nil
}
// DeleteNode a node
func (c *Cloud) DeleteNode(name string) error {
node, ok := c.Nodes[name]
if ok {
delete(c.Nodes, name)
}
if node.GetType() == NodeTypeMaster && len(c.Nodes) > 0 {
return fmt.Errorf("could not delete master node since there is still agent node running")
}
return nil
}
// State get cloud state
func (c *Cloud) State() {}
// UnmarshalJSON unmarsha json
func (c *Cloud) UnmarshalJSON(data []byte) error {
var m map[string]interface{}
if err := json.Unmarshal(data, &m); err != nil {
return err
}
c.Nodes = make(map[string]Noder)
for k, v := range m {
if k == "nodes" {
nodes, ok := v.(map[string]interface{})
if !ok {
return fmt.Errorf("invalid nodes data")
}
for name, n := range nodes {
node, ok := n.(map[string]interface{})
if !ok {
return fmt.Errorf("invalid node data")
}
n, err := CreateNode(node["ip"].(string), node["user"].(string), node["type"].(string), node["name"].(string))
if err != nil {
return err
}
c.Nodes[name] = n
}
} else if k == "token" {
tok, ok := v.(string)
if ok {
c.Token = tok
} else {
c.Token = ""
}
} else if k == "config" {
config, ok := v.(string)
if ok {
c.Config = config
} else {
c.Config = ""
}
} else if k == "type" {
typ, ok := v.(string)
if ok {
c.Type = typ
} else {
c.Type = ""
}
} else if k == "url" {
url, ok := v.(string)
if ok {
c.URL = url
} else {
c.URL = ""
}
}
}
return nil
}
// MarshalJSON cloud information
func (c *Cloud) MarshalJSON() ([]byte, error) {
nodes := map[string]Node{}
for name, node := range c.Nodes {
nodes[name] = Node{
IP: node.GetIP(),
Type: node.GetType(),
User: node.GetUser(),
Name: node.GetName(),
}
}
body, err := json.Marshal(struct {
URL string `json:"url"`
Config string `json:"config"`
Type string `json:"type"`
Token string `json:"token"`
Nodes map[string]Node `json:"nodes"`
}{
URL: c.URL,
Config: c.Config,
Type: c.Type,
Token: c.Token,
Nodes: nodes,
})
if err != nil {
return nil, err
}
return body, nil
}
// GetType get type of cloud
func (c *Cloud) GetType() string {
return c.Type
}
// Dump cloud data
func (c *Cloud) Dump() ([]byte, error) {
return json.Marshal(c)
}
// GetConfig get config
func (c *Cloud) GetConfig() (string, error) {
if c.Config != "" {
return c.Config, nil
}
if err := c.Provision(); err != nil {
return "", err
}
return c.Config, nil
}
var (
_ infra.Clouder = &Cloud{}
)

137
infra/k8s/cloud_test.go Normal file
View File

@@ -0,0 +1,137 @@
package k8s
import (
"encoding/json"
"fmt"
"testing"
"github.com/golang/mock/gomock"
mock_infra "github.com/metrue/fx/infra/k8s/mocks"
)
func TestLoad(t *testing.T) {
t.Run("empty meta", func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
var createNodeFn = func(n Noder) (Noder, error) {
return nil, nil
}
_, err := Load([]byte{}, createNodeFn)
if err == nil {
t.Fatalf("should load with error")
}
})
t.Run("only master node", func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
master := mock_infra.NewMockNoder(ctrl)
var createNodeFn = func(n Noder) (Noder, error) {
return master, nil
}
typ := NodeTypeMaster
name := "master"
ip := "127.0.0.1"
user := "testuser"
master.EXPECT().GetName().Return(name)
master.EXPECT().GetType().Return(typ).Times(2)
master.EXPECT().GetIP().Return(ip).Times(2)
master.EXPECT().GetUser().Return(user)
master.EXPECT().GetConfig().Return("sample-config", nil)
claud := &Cloud{
Config: "",
URL: "",
Token: "",
Type: "k8s",
Nodes: map[string]Noder{"master-node": master},
}
meta, err := json.Marshal(claud)
if err != nil {
t.Fatal(err)
}
cloud, err := Load(meta, createNodeFn)
if err != nil {
t.Fatal(err)
}
if len(cloud.Nodes) != 1 {
t.Fatalf("should get %d but got %d", 1, len(cloud.Nodes))
}
master.EXPECT().Provision(map[string]string{}).Return(nil)
master.EXPECT().GetToken().Return("tok-1", nil)
if err := cloud.Provision(); err != nil {
t.Fatal(err)
}
})
t.Run("one master node and one agent", func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
master := mock_infra.NewMockNoder(ctrl)
node := mock_infra.NewMockNoder(ctrl)
var createNodeFn = func(n Noder) (Noder, error) {
if n.GetType() == NodeTypeMaster {
return master, nil
}
return node, nil
}
typ := NodeTypeMaster
name := "master"
ip := "127.0.0.1"
user := "testuser"
master.EXPECT().GetName().Return(name)
master.EXPECT().GetType().Return(typ).Times(2)
master.EXPECT().GetIP().Return(ip).Times(3)
master.EXPECT().GetConfig().Return("sample-config", nil)
master.EXPECT().GetUser().Return(user)
nodeType := NodeTypeAgent
nodeName := "agent_name"
nodeIP := "12.12.12.12"
nodeUser := "testuser"
node.EXPECT().GetName().Return(nodeName)
node.EXPECT().GetType().Return(nodeType).Times(3)
node.EXPECT().GetIP().Return(nodeIP)
node.EXPECT().GetUser().Return(nodeUser)
url := fmt.Sprintf("https://%s:6443", master.GetIP())
tok := "tok-1"
claud := &Cloud{
Config: "",
URL: url,
Token: tok,
Type: "k8s",
Nodes: map[string]Noder{"master-node": master, "agent-node": node},
}
meta, err := json.Marshal(claud)
if err != nil {
t.Fatal(err)
}
cloud, err := Load(meta, createNodeFn)
if err != nil {
t.Fatal(err)
}
if len(cloud.Nodes) != 2 {
t.Fatalf("should get %d but got %d", 2, len(cloud.Nodes))
}
master.EXPECT().Provision(map[string]string{}).Return(nil)
master.EXPECT().GetToken().Return(tok, nil)
node.EXPECT().Provision(map[string]string{
"url": cloud.URL,
"token": cloud.Token,
}).Return(nil)
if err := cloud.Provision(); err != nil {
t.Fatal(err)
}
})
}
func TestProvision(t *testing.T) {}

View File

@@ -1,4 +1,4 @@
package kubernetes
package k8s
import (
apiv1 "k8s.io/api/core/v1"

View File

@@ -1,4 +1,4 @@
package kubernetes
package k8s
import (
"os"
@@ -11,7 +11,7 @@ func TestConfigMap(t *testing.T) {
t.Skip("skip test since no KUBECONFIG given in environment variable")
}
k8s, err := Create()
k8s, err := Create("")
if err != nil {
t.Fatal(err)
}

View File

@@ -1,4 +1,4 @@
package kubernetes
package k8s
// ConfigMap is the key to function docker project source code in configmap
var ConfigMap = struct {

View File

@@ -1,10 +1,11 @@
package kubernetes
package k8s
import (
"context"
"os"
"testing"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/types"
)
@@ -26,21 +27,17 @@ func TestK8SDeployer(t *testing.T) {
if kubeconfig == "" || username == "" || password == "" {
t.Skip("skip test since no KUBECONFIG, DOCKER_USERNAME and DOCKER_PASSWORD given in environment variable")
}
k8s, err := Create()
k8s, err := Create("")
if err != nil {
t.Fatal(err)
}
fn := types.Func{
Language: "node",
Source: `
module.exports = (ctx) => {
ctx.body = 'hello world'
}
`,
data, err := packer.PackIntoK8SConfigMapFile("./fixture")
if err != nil {
t.Fatal(err)
}
ctx := context.Background()
if err := k8s.Deploy(ctx, fn, name, bindings); err != nil {
if err := k8s.Deploy(ctx, data, name, name, bindings); err != nil {
t.Fatal(err)
}

View File

@@ -1,10 +1,12 @@
package kubernetes
package k8s
import (
"context"
"fmt"
"os"
"github.com/metrue/fx/deploy"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/types"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
@@ -18,8 +20,12 @@ type K8S struct {
const namespace = "default"
// Create a k8s cluster client
func Create() (*K8S, error) {
config, err := clientcmd.BuildConfigFromKubeconfigGetter("", clientcmd.NewDefaultClientConfigLoadingRules().Load)
func Create(kubeconfig string) (*K8S, error) {
if os.Getenv("KUBECONFIG") != "" {
kubeconfig = os.Getenv("KUBECONFIG")
}
config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
if err != nil {
return nil, err
}
@@ -34,17 +40,13 @@ func Create() (*K8S, error) {
// Deploy a image to be a service
func (k *K8S) Deploy(
ctx context.Context,
fn types.Func,
fn string,
name string,
image string,
ports []types.PortBinding,
) error {
// put source code of function docker project into k8s config map
tree, err := packer.PackIntoK8SConfigMapFile(fn)
if err != nil {
return err
}
data := map[string]string{}
data[ConfigMap.AppMetaEnvName] = tree
data[ConfigMap.AppMetaEnvName] = fn
if _, err := k.CreateOrUpdateConfigMap(namespace, name, data); err != nil {
return err
}
@@ -56,14 +58,28 @@ func (k *K8S) Deploy(
const replicas = int32(3)
if _, err := k.GetDeployment(namespace, name); err != nil {
// TODO enable passing replica from fx CLI
if _, err := k.CreateDeploymentWithInitContainer(
namespace,
name,
ports,
replicas,
selector,
); err != nil {
return err
if os.Getenv("K3S") != "" {
// NOTE Doing docker build in initial container will fail when cluster is created by K3S
if _, err := k.CreateDeployment(
namespace,
name,
image,
ports,
replicas,
selector,
); err != nil {
return err
}
} else {
if _, err := k.CreateDeploymentWithInitContainer(
namespace,
name,
ports,
replicas,
selector,
); err != nil {
return err
}
}
} else {
if _, err := k.UpdateDeployment(
@@ -80,10 +96,9 @@ func (k *K8S) Deploy(
// TODO fx should be able to know what's the target Kubernetes service platform
// it's going to deploy to
const isOnPublicCloud = true
typ := "LoadBalancer"
if !isOnPublicCloud {
typ = "NodePort"
if os.Getenv("SERVICE_TYPE") != "" {
typ = os.Getenv("SERVICE_TYPE")
}
if _, err := k.GetService(namespace, name); err != nil {
@@ -127,10 +142,49 @@ func (k *K8S) Destroy(ctx context.Context, name string) error {
}
// GetStatus get status of a service
func (k *K8S) GetStatus(ctx context.Context, name string) error {
func (k *K8S) GetStatus(ctx context.Context, name string) (types.Service, error) {
svc, err := k.GetService(namespace, name)
service := types.Service{}
if err != nil {
return service, err
}
service.Host = svc.Spec.ClusterIP
if len(svc.Spec.ExternalIPs) > 0 {
service.Host = svc.Spec.ExternalIPs[0]
}
for _, port := range svc.Spec.Ports {
// TODO should clearify which port (target port, node port) should use
service.Port = int(port.Port)
break
}
return service, nil
}
// List services
func (k *K8S) List(ctx context.Context, name string) (svcs []types.Service, err error) {
const task = "listing"
spinner.Start(task)
defer func() {
spinner.Stop(task, err)
}()
return []types.Service{}, nil
}
// Ping health check of infra
func (k *K8S) Ping(ctx context.Context) error {
// Does not find any ping method for k8s
nodes, err := k.ListNodes()
if err != nil {
return err
}
if len(nodes.Items) <= 0 {
return fmt.Errorf("no available nodes")
}
return nil
}
var (
_ deploy.Deployer = &K8S{}
_ infra.Deployer = &K8S{}
)

View File

@@ -1,4 +1,4 @@
package kubernetes
package k8s
import (
"fmt"
@@ -29,7 +29,7 @@ func generateDeploymentSpec(
Name: "fx-placeholder-container-name",
Image: image,
Ports: ports,
ImagePullPolicy: v1.PullNever,
ImagePullPolicy: v1.PullIfNotPresent,
}
return &appsv1.Deployment{
ObjectMeta: metav1.ObjectMeta{
@@ -98,6 +98,5 @@ func (k *K8S) CreateDeploymentWithInitContainer(
) (*appsv1.Deployment, error) {
deployment := generateDeploymentSpec(name, name, ports, replicas, selector)
updatedDeployment := injectInitContainer(name, deployment)
fmt.Println(updatedDeployment)
return k.AppsV1().Deployments(namespace).Create(updatedDeployment)
}

View File

@@ -1,4 +1,4 @@
package kubernetes
package k8s
import (
"os"
@@ -20,7 +20,7 @@ func TestDeployment(t *testing.T) {
t.Skip("skip test since no KUBECONFIG given in environment variable")
}
k8s, err := Create()
k8s, err := Create("")
if err != nil {
t.Fatal(err)
}

5
infra/k8s/doc.go Normal file
View File

@@ -0,0 +1,5 @@
/*
*/
package k8s

View File

@@ -1,4 +1,4 @@
package kubernetes
package k8s
import (
appsv1 "k8s.io/api/apps/v1"

6
infra/k8s/k8s.go Normal file
View File

@@ -0,0 +1,6 @@
package k8s
// CreateDeployer create a deployer
func CreateDeployer(kubeconfig string) (*K8S, error) {
return Create(kubeconfig)
}

15
infra/k8s/k8s_node.go Normal file
View File

@@ -0,0 +1,15 @@
package k8s
import (
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// ListNodes list node
func (k *K8S) ListNodes() (*v1.NodeList, error) {
nodes, err := k.CoreV1().Nodes().List(metav1.ListOptions{})
if err != nil {
return nil, err
}
return nodes, nil
}

147
infra/k8s/mocks/node.go Normal file
View File

@@ -0,0 +1,147 @@
// Code generated by MockGen. DO NOT EDIT.
// Source: node.go
// Package mock_k8s is a generated GoMock package.
package mock_k8s
import (
gomock "github.com/golang/mock/gomock"
reflect "reflect"
)
// MockNoder is a mock of Noder interface
type MockNoder struct {
ctrl *gomock.Controller
recorder *MockNoderMockRecorder
}
// MockNoderMockRecorder is the mock recorder for MockNoder
type MockNoderMockRecorder struct {
mock *MockNoder
}
// NewMockNoder creates a new mock instance
func NewMockNoder(ctrl *gomock.Controller) *MockNoder {
mock := &MockNoder{ctrl: ctrl}
mock.recorder = &MockNoderMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockNoder) EXPECT() *MockNoderMockRecorder {
return m.recorder
}
// Provision mocks base method
func (m *MockNoder) Provision(meta map[string]string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Provision", meta)
ret0, _ := ret[0].(error)
return ret0
}
// Provision indicates an expected call of Provision
func (mr *MockNoderMockRecorder) Provision(meta interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Provision", reflect.TypeOf((*MockNoder)(nil).Provision), meta)
}
// GetConfig mocks base method
func (m *MockNoder) GetConfig() (string, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetConfig")
ret0, _ := ret[0].(string)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetConfig indicates an expected call of GetConfig
func (mr *MockNoderMockRecorder) GetConfig() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetConfig", reflect.TypeOf((*MockNoder)(nil).GetConfig))
}
// GetType mocks base method
func (m *MockNoder) GetType() string {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetType")
ret0, _ := ret[0].(string)
return ret0
}
// GetType indicates an expected call of GetType
func (mr *MockNoderMockRecorder) GetType() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetType", reflect.TypeOf((*MockNoder)(nil).GetType))
}
// GetName mocks base method
func (m *MockNoder) GetName() string {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetName")
ret0, _ := ret[0].(string)
return ret0
}
// GetName indicates an expected call of GetName
func (mr *MockNoderMockRecorder) GetName() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetName", reflect.TypeOf((*MockNoder)(nil).GetName))
}
// GetUser mocks base method
func (m *MockNoder) GetUser() string {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetUser")
ret0, _ := ret[0].(string)
return ret0
}
// GetUser indicates an expected call of GetUser
func (mr *MockNoderMockRecorder) GetUser() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetUser", reflect.TypeOf((*MockNoder)(nil).GetUser))
}
// GetToken mocks base method
func (m *MockNoder) GetToken() (string, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetToken")
ret0, _ := ret[0].(string)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetToken indicates an expected call of GetToken
func (mr *MockNoderMockRecorder) GetToken() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetToken", reflect.TypeOf((*MockNoder)(nil).GetToken))
}
// GetIP mocks base method
func (m *MockNoder) GetIP() string {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetIP")
ret0, _ := ret[0].(string)
return ret0
}
// GetIP indicates an expected call of GetIP
func (mr *MockNoderMockRecorder) GetIP() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetIP", reflect.TypeOf((*MockNoder)(nil).GetIP))
}
// Dump mocks base method
func (m *MockNoder) Dump() map[string]string {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Dump")
ret0, _ := ret[0].(map[string]string)
return ret0
}
// Dump indicates an expected call of Dump
func (mr *MockNoderMockRecorder) Dump() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Dump", reflect.TypeOf((*MockNoder)(nil).Dump))
}

216
infra/k8s/node.go Normal file
View File

@@ -0,0 +1,216 @@
package k8s
import (
"bufio"
"bytes"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/metrue/fx/infra"
"github.com/metrue/go-ssh-client"
"github.com/mitchellh/go-homedir"
)
const NodeTypeMaster = "k3s_master"
const NodeTypeAgent = "k3s_agent"
const NodeTypeDocker = "docker_agent"
// Noder node interface
type Noder interface {
Provision(meta map[string]string) error
GetConfig() (string, error)
GetType() string
GetName() string
GetUser() string
GetToken() (string, error)
GetIP() string
Dump() map[string]string
}
// Node define a node
type Node struct {
IP string `json:"ip"`
User string `json:"user"`
Type string `json:"type"`
Name string `json:"name"`
sshClient ssh.Clienter
}
// CreateNode create a node
func CreateNode(ip string, user string, typ string, name string) (*Node, error) {
key, err := sshkey()
if err != nil {
return nil, err
}
port := sshport()
sshClient := ssh.New(ip).WithUser(user).WithKey(key).WithPort(port)
return &Node{
IP: ip,
User: user,
Type: typ,
Name: name,
sshClient: sshClient,
}, nil
}
func (n *Node) runCmd(script string) error {
return n.sshClient.RunCommand(script, ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
})
}
// Provision provision node
func (n *Node) Provision(meta map[string]string) error {
if err := n.runCmd(infra.Scripts["docker_version"].(string)); err != nil {
if err := n.runCmd(infra.Scripts["install_docker"].(string)); err != nil {
return err
}
if err := n.runCmd(infra.Scripts["start_dockerd"].(string)); err != nil {
return err
}
}
if n.Type == NodeTypeMaster {
if err := n.runCmd(infra.Scripts["check_k3s_server"].(string)); err != nil {
cmd := infra.Scripts["setup_k3s_master"].(func(ip string) string)(n.IP)
if err := n.runCmd(cmd); err != nil {
return err
}
}
} else if n.Type == NodeTypeAgent {
if err := n.runCmd(infra.Scripts["check_k3s_agent"].(string)); err != nil {
cmd := infra.Scripts["setup_k3s_agent"].(func(url string, tok string) string)(meta["url"], meta["token"])
if err := n.runCmd(cmd); err != nil {
return err
}
}
}
return nil
}
// GetToken get token from master node
func (n *Node) GetToken() (string, error) {
if n.Type != NodeTypeMaster {
return "", fmt.Errorf("could not get token from a non-master node")
}
var outPipe bytes.Buffer
if err := n.sshClient.RunCommand(infra.Scripts["get_k3s_token"].(string), ssh.CommandOptions{Stdout: bufio.NewWriter(&outPipe)}); err != nil {
return "", err
}
return outPipe.String(), nil
}
// State get node state
func (n *Node) State() {}
// Dump node information to json
func (n *Node) Dump() map[string]string {
return map[string]string{
"ip": n.IP,
"name": n.Name,
"user": n.User,
"type": n.Type,
}
}
// GetType get node type
func (n *Node) GetType() string {
return n.Type
}
// GetName get node type
func (n *Node) GetName() string {
return n.Name
}
// GetIP get node type
func (n *Node) GetIP() string {
return n.IP
}
// GetUser get user
func (n *Node) GetUser() string {
return n.User
}
// GetConfig get config
func (n *Node) GetConfig() (string, error) {
if n.Type == NodeTypeMaster {
var outPipe bytes.Buffer
if err := n.sshClient.RunCommand(infra.Scripts["get_k3s_kubeconfig"].(string), ssh.CommandOptions{
Stdout: bufio.NewWriter(&outPipe),
}); err != nil {
return "", err
}
return string(rewriteKubeconfig(outPipe.String(), n.IP, "default")), nil
} else if n.Type == NodeTypeDocker {
data, err := json.Marshal(n.Dump())
if err != nil {
return "", err
}
return string(data), nil
}
return "", fmt.Errorf("no config for node type of %s", n.Type)
}
// NOTE only using for unit testing
func (n *Node) setsshClient(client ssh.Clienter) {
n.sshClient = client
}
// NOTE the reason putting sshkey() and sshport here inside node.go is because
// ssh key and ssh port is related to node it self, we may extend this in future
func sshkey() (string, error) {
path := os.Getenv("SSH_KEY_FILE")
if path != "" {
absPath, err := filepath.Abs(path)
if err != nil {
return "", err
}
return absPath, nil
}
key, err := homedir.Expand("~/.ssh/id_rsa")
if err != nil {
return "", err
}
return key, nil
}
func sshport() string {
port := os.Getenv("SSH_PORT")
if port != "" {
return port
}
return "22"
}
func rewriteKubeconfig(kubeconfig string, ip string, context string) []byte {
if context == "" {
// nolint
context = "default"
}
kubeconfigReplacer := strings.NewReplacer(
"127.0.0.1", ip,
"localhost", ip,
"default", context,
)
return []byte(kubeconfigReplacer.Replace(kubeconfig))
}
var (
_ Noder = &Node{}
)

211
infra/k8s/node_test.go Normal file
View File

@@ -0,0 +1,211 @@
package k8s
import (
"fmt"
"os"
"testing"
"github.com/golang/mock/gomock"
"github.com/metrue/fx/infra"
"github.com/metrue/go-ssh-client"
sshMocks "github.com/metrue/go-ssh-client/mocks"
"github.com/mitchellh/go-homedir"
)
func TestGetSSHKeyFile(t *testing.T) {
t.Run("defaut", func(t *testing.T) {
defau, err := sshkey()
if err != nil {
t.Fatal(err)
}
defaultPath, _ := homedir.Expand("~/.ssh/id_rsa")
if defau != defaultPath {
t.Fatalf("should get %s but got %s", defaultPath, defau)
}
})
t.Run("override from env", func(t *testing.T) {
os.Setenv("SSH_KEY_FILE", "/tmp/id_rsa")
keyFile, err := sshkey()
if err != nil {
t.Fatal(err)
}
if keyFile != "/tmp/id_rsa" {
t.Fatalf("should get %s but got %s", "/tmp/id_rsa", keyFile)
}
})
}
func TestGetSSHPort(t *testing.T) {
t.Run("defaut", func(t *testing.T) {
defau := sshport()
if defau != "22" {
t.Fatalf("should get %s but got %s", "22", defau)
}
})
t.Run("override from env", func(t *testing.T) {
os.Setenv("SSH_PORT", "2222")
defau := sshport()
if defau != "2222" {
t.Fatalf("should get %s but got %s", "2222", defau)
}
})
}
func TestNode(t *testing.T) {
t.Run("master node already has docker and k3s server", func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
n, err := CreateNode("127.0.0.1", "fx", NodeTypeMaster, "master")
if err != nil {
t.Fatal(err)
}
if n.sshClient == nil {
t.Fatal("ssh client should not be nil")
}
sshClient := sshMocks.NewMockClienter(ctrl)
n.setsshClient(sshClient)
sshClient.EXPECT().RunCommand(infra.Scripts["docker_version"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(nil)
sshClient.EXPECT().RunCommand(infra.Scripts["check_k3s_server"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(nil)
if err := n.Provision(map[string]string{}); err != nil {
t.Fatal(err)
}
})
t.Run("master node no docker and k3s server", func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
n, err := CreateNode("127.0.0.1", "fx", NodeTypeMaster, "master")
if err != nil {
t.Fatal(err)
}
if n.sshClient == nil {
t.Fatal("ssh client should not be nil")
}
sshClient := sshMocks.NewMockClienter(ctrl)
n.setsshClient(sshClient)
sshClient.EXPECT().RunCommand(infra.Scripts["docker_version"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(fmt.Errorf("no such command"))
sshClient.EXPECT().RunCommand(infra.Scripts["install_docker"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(nil)
sshClient.EXPECT().RunCommand(infra.Scripts["start_dockerd"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(nil)
sshClient.EXPECT().RunCommand(infra.Scripts["check_k3s_server"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(fmt.Errorf("no such progress"))
cmd := infra.Scripts["setup_k3s_master"].(func(ip string) string)(n.IP)
sshClient.EXPECT().RunCommand(cmd, ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(nil)
if err := n.Provision(map[string]string{}); err != nil {
t.Fatal(err)
}
})
t.Run("agent node already has docker and k3s agent", func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
n, err := CreateNode("127.0.0.1", "fx", NodeTypeAgent, "agent")
if err != nil {
t.Fatal(err)
}
if n.sshClient == nil {
t.Fatal("ssh client should not be nil")
}
sshClient := sshMocks.NewMockClienter(ctrl)
n.setsshClient(sshClient)
sshClient.EXPECT().RunCommand(infra.Scripts["docker_version"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(nil)
sshClient.EXPECT().RunCommand(infra.Scripts["check_k3s_agent"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(nil)
if err := n.Provision(map[string]string{}); err != nil {
t.Fatal(err)
}
})
t.Run("agent node no docker and k3s agent", func(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
n, err := CreateNode("127.0.0.1", "fx", NodeTypeAgent, "agent")
if err != nil {
t.Fatal(err)
}
if n.sshClient == nil {
t.Fatal("ssh client should not be nil")
}
sshClient := sshMocks.NewMockClienter(ctrl)
n.setsshClient(sshClient)
sshClient.EXPECT().RunCommand(infra.Scripts["docker_version"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(fmt.Errorf("no such command"))
sshClient.EXPECT().RunCommand(infra.Scripts["install_docker"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(nil)
sshClient.EXPECT().RunCommand(infra.Scripts["start_dockerd"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(nil)
sshClient.EXPECT().RunCommand(infra.Scripts["check_k3s_agent"].(string), ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(fmt.Errorf("no such progress"))
url := "url-1"
token := "token-1"
cmd := infra.Scripts["setup_k3s_agent"].(func(url string, ip string) string)(url, token)
sshClient.EXPECT().RunCommand(cmd, ssh.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}).Return(nil)
if err := n.Provision(map[string]string{"url": url, "token": token}); err != nil {
t.Fatal(err)
}
})
}

View File

@@ -1,4 +1,4 @@
package kubernetes
package k8s
import (
"github.com/metrue/fx/constants"

View File

@@ -1,4 +1,4 @@
package kubernetes
package k8s
import (
"strconv"

View File

@@ -1,4 +1,4 @@
package kubernetes
package k8s
import (
"os"
@@ -27,7 +27,7 @@ func TestK8S(t *testing.T) {
if kubeconfig == "" {
t.Skip("skip test since no KUBECONFIG given in environment variable")
}
k8s, err := Create()
k8s, err := Create("")
if err != nil {
t.Fatal(err)
}

313
infra/mocks/infra.go Normal file
View File

@@ -0,0 +1,313 @@
// Code generated by MockGen. DO NOT EDIT.
// Source: infra.go
// Package mock_infra is a generated GoMock package.
package mock_infra
import (
context "context"
gomock "github.com/golang/mock/gomock"
types "github.com/metrue/fx/types"
reflect "reflect"
)
// MockProvisioner is a mock of Provisioner interface
type MockProvisioner struct {
ctrl *gomock.Controller
recorder *MockProvisionerMockRecorder
}
// MockProvisionerMockRecorder is the mock recorder for MockProvisioner
type MockProvisionerMockRecorder struct {
mock *MockProvisioner
}
// NewMockProvisioner creates a new mock instance
func NewMockProvisioner(ctrl *gomock.Controller) *MockProvisioner {
mock := &MockProvisioner{ctrl: ctrl}
mock.recorder = &MockProvisionerMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockProvisioner) EXPECT() *MockProvisionerMockRecorder {
return m.recorder
}
// Provision mocks base method
func (m *MockProvisioner) Provision() ([]byte, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Provision")
ret0, _ := ret[0].([]byte)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// Provision indicates an expected call of Provision
func (mr *MockProvisionerMockRecorder) Provision() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Provision", reflect.TypeOf((*MockProvisioner)(nil).Provision))
}
// HealthCheck mocks base method
func (m *MockProvisioner) HealthCheck() (bool, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "HealthCheck")
ret0, _ := ret[0].(bool)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// HealthCheck indicates an expected call of HealthCheck
func (mr *MockProvisionerMockRecorder) HealthCheck() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "HealthCheck", reflect.TypeOf((*MockProvisioner)(nil).HealthCheck))
}
// MockDeployer is a mock of Deployer interface
type MockDeployer struct {
ctrl *gomock.Controller
recorder *MockDeployerMockRecorder
}
// MockDeployerMockRecorder is the mock recorder for MockDeployer
type MockDeployerMockRecorder struct {
mock *MockDeployer
}
// NewMockDeployer creates a new mock instance
func NewMockDeployer(ctrl *gomock.Controller) *MockDeployer {
mock := &MockDeployer{ctrl: ctrl}
mock.recorder = &MockDeployerMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockDeployer) EXPECT() *MockDeployerMockRecorder {
return m.recorder
}
// Deploy mocks base method
func (m *MockDeployer) Deploy(ctx context.Context, fn, name, image string, bindings []types.PortBinding) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Deploy", ctx, fn, name, image, bindings)
ret0, _ := ret[0].(error)
return ret0
}
// Deploy indicates an expected call of Deploy
func (mr *MockDeployerMockRecorder) Deploy(ctx, fn, name, image, bindings interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Deploy", reflect.TypeOf((*MockDeployer)(nil).Deploy), ctx, fn, name, image, bindings)
}
// Destroy mocks base method
func (m *MockDeployer) Destroy(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Destroy", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// Destroy indicates an expected call of Destroy
func (mr *MockDeployerMockRecorder) Destroy(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Destroy", reflect.TypeOf((*MockDeployer)(nil).Destroy), ctx, name)
}
// Update mocks base method
func (m *MockDeployer) Update(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Update", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// Update indicates an expected call of Update
func (mr *MockDeployerMockRecorder) Update(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Update", reflect.TypeOf((*MockDeployer)(nil).Update), ctx, name)
}
// GetStatus mocks base method
func (m *MockDeployer) GetStatus(ctx context.Context, name string) (types.Service, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetStatus", ctx, name)
ret0, _ := ret[0].(types.Service)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetStatus indicates an expected call of GetStatus
func (mr *MockDeployerMockRecorder) GetStatus(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetStatus", reflect.TypeOf((*MockDeployer)(nil).GetStatus), ctx, name)
}
// List mocks base method
func (m *MockDeployer) List(ctx context.Context, name string) ([]types.Service, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "List", ctx, name)
ret0, _ := ret[0].([]types.Service)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// List indicates an expected call of List
func (mr *MockDeployerMockRecorder) List(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "List", reflect.TypeOf((*MockDeployer)(nil).List), ctx, name)
}
// Ping mocks base method
func (m *MockDeployer) Ping(ctx context.Context) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Ping", ctx)
ret0, _ := ret[0].(error)
return ret0
}
// Ping indicates an expected call of Ping
func (mr *MockDeployerMockRecorder) Ping(ctx interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Ping", reflect.TypeOf((*MockDeployer)(nil).Ping), ctx)
}
// MockInfra is a mock of Infra interface
type MockInfra struct {
ctrl *gomock.Controller
recorder *MockInfraMockRecorder
}
// MockInfraMockRecorder is the mock recorder for MockInfra
type MockInfraMockRecorder struct {
mock *MockInfra
}
// NewMockInfra creates a new mock instance
func NewMockInfra(ctrl *gomock.Controller) *MockInfra {
mock := &MockInfra{ctrl: ctrl}
mock.recorder = &MockInfraMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockInfra) EXPECT() *MockInfraMockRecorder {
return m.recorder
}
// Provision mocks base method
func (m *MockInfra) Provision() ([]byte, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Provision")
ret0, _ := ret[0].([]byte)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// Provision indicates an expected call of Provision
func (mr *MockInfraMockRecorder) Provision() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Provision", reflect.TypeOf((*MockInfra)(nil).Provision))
}
// HealthCheck mocks base method
func (m *MockInfra) HealthCheck() (bool, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "HealthCheck")
ret0, _ := ret[0].(bool)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// HealthCheck indicates an expected call of HealthCheck
func (mr *MockInfraMockRecorder) HealthCheck() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "HealthCheck", reflect.TypeOf((*MockInfra)(nil).HealthCheck))
}
// Deploy mocks base method
func (m *MockInfra) Deploy(ctx context.Context, fn, name, image string, bindings []types.PortBinding) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Deploy", ctx, fn, name, image, bindings)
ret0, _ := ret[0].(error)
return ret0
}
// Deploy indicates an expected call of Deploy
func (mr *MockInfraMockRecorder) Deploy(ctx, fn, name, image, bindings interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Deploy", reflect.TypeOf((*MockInfra)(nil).Deploy), ctx, fn, name, image, bindings)
}
// Destroy mocks base method
func (m *MockInfra) Destroy(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Destroy", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// Destroy indicates an expected call of Destroy
func (mr *MockInfraMockRecorder) Destroy(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Destroy", reflect.TypeOf((*MockInfra)(nil).Destroy), ctx, name)
}
// Update mocks base method
func (m *MockInfra) Update(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Update", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// Update indicates an expected call of Update
func (mr *MockInfraMockRecorder) Update(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Update", reflect.TypeOf((*MockInfra)(nil).Update), ctx, name)
}
// GetStatus mocks base method
func (m *MockInfra) GetStatus(ctx context.Context, name string) (types.Service, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetStatus", ctx, name)
ret0, _ := ret[0].(types.Service)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetStatus indicates an expected call of GetStatus
func (mr *MockInfraMockRecorder) GetStatus(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetStatus", reflect.TypeOf((*MockInfra)(nil).GetStatus), ctx, name)
}
// List mocks base method
func (m *MockInfra) List(ctx context.Context, name string) ([]types.Service, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "List", ctx, name)
ret0, _ := ret[0].([]types.Service)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// List indicates an expected call of List
func (mr *MockInfraMockRecorder) List(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "List", reflect.TypeOf((*MockInfra)(nil).List), ctx, name)
}
// Ping mocks base method
func (m *MockInfra) Ping(ctx context.Context) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Ping", ctx)
ret0, _ := ret[0].(error)
return ret0
}
// Ping indicates an expected call of Ping
func (mr *MockInfraMockRecorder) Ping(ctx interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Ping", reflect.TypeOf((*MockInfra)(nil).Ping), ctx)
}

28
infra/scripts.go Normal file
View File

@@ -0,0 +1,28 @@
package infra
import (
"fmt"
)
// TODO upgrade to latest when k3s fix the tls scan issue
// https://github.com/rancher/k3s/issues/556
const k3sVersion = "v0.9.1"
// Scripts to provision host
var Scripts = map[string]interface{}{
"docker_version": "docker version",
"install_docker": "curl -fsSL https://download.docker.com/linux/static/stable/x86_64/docker-18.06.3-ce.tgz -o docker.tgz && tar zxvf docker.tgz && mv docker/* /usr/bin && rm -rf docker docker.tgz",
"start_dockerd": "dockerd >/dev/null 2>&1 & sleep 2",
"check_fx_agent": "docker inspect fx-agent",
"start_fx_agent": "docker run -d --name=fx-agent --rm -v /var/run/docker.sock:/var/run/docker.sock -p 0.0.0.0:8866:1234 bobrik/socat TCP-LISTEN:1234,fork UNIX-CONNECT:/var/run/docker.sock",
"check_k3s_server": "ps aux | grep 'k3s server --docker'",
"setup_k3s_master": func(ip string) string {
return fmt.Sprintf("curl -sLS https://get.k3s.io | INSTALL_K3S_EXEC='server --docker --tls-san %s' INSTALL_K3S_VERSION='%s' sh -", ip, k3sVersion)
},
"check_k3s_agent": "ps aux | grep 'k3s agent --docker'",
"setup_k3s_agent": func(masterURL string, tok string) string {
return fmt.Sprintf("curl -fL https://get.k3s.io/ | K3S_URL='%s' K3S_TOKEN='%s' INSTALL_K3S_VERSION='%s' sh -s - --docker", masterURL, tok, k3sVersion)
},
"get_k3s_token": "cat /var/lib/rancher/k3s/server/node-token",
"get_k3s_kubeconfig": "cat /etc/rancher/k3s/k3s.yaml",
}

9
infra/sudo.go Normal file
View File

@@ -0,0 +1,9 @@
package infra
// Sudo append sudo when user is not root
func Sudo(cmd string, user string) string {
if user == "root" {
return cmd
}
return "sudo " + cmd
}

5
k3s.cluster.json Normal file
View File

@@ -0,0 +1,5 @@
{
"master": "52.78.196.250",
"agents": [
"13.125.243.192",
],

61
middlewares/binding.go Normal file
View File

@@ -0,0 +1,61 @@
package middlewares
import (
"fmt"
"os"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/context"
"github.com/metrue/fx/types"
"github.com/phayes/freeport"
)
// PortRange usable port range https: //en.wikipedia.org/wiki/Ephemeral_port
var PortRange = struct {
min int
max int
}{
min: 1023,
max: 65535,
}
// Binding create bindings
func Binding(ctx context.Contexter) (err error) {
port := ctx.Get("port").(int)
if port == 0 {
port, err = freeport.GetFreePort()
if err != nil {
return err
}
}
if port < PortRange.min || port > PortRange.max {
return fmt.Errorf("invalid port number: %d, port number should in range of %d - %d", port, PortRange.min, PortRange.max)
}
var bindings []types.PortBinding
if os.Getenv("KUBECONFIG") != "" {
bindings = []types.PortBinding{
types.PortBinding{
ServiceBindingPort: 80,
ContainerExposePort: constants.FxContainerExposePort,
},
types.PortBinding{
ServiceBindingPort: 443,
ContainerExposePort: constants.FxContainerExposePort,
},
types.PortBinding{
ServiceBindingPort: int32(port),
ContainerExposePort: constants.FxContainerExposePort,
},
}
} else {
bindings = []types.PortBinding{
types.PortBinding{
ServiceBindingPort: int32(port),
ContainerExposePort: constants.FxContainerExposePort,
},
}
}
ctx.Set("bindings", bindings)
return nil
}

View File

@@ -0,0 +1,20 @@
package middlewares
import (
"testing"
"github.com/golang/mock/gomock"
mockCtx "github.com/metrue/fx/context/mocks"
)
func TestBinding(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
ctx := mockCtx.NewMockContexter(ctrl)
ctx.EXPECT().Get("port").Return(0)
ctx.EXPECT().Set("bindings", gomock.Any())
if err := Binding(ctx); err != nil {
t.Fatal(err)
}
}

78
middlewares/build.go Normal file
View File

@@ -0,0 +1,78 @@
package middlewares
import (
"fmt"
"os"
"time"
containerruntimes "github.com/metrue/fx/container_runtimes"
"github.com/metrue/fx/context"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
"github.com/otiai10/copy"
)
// Build image
func Build(ctx context.Contexter) (err error) {
const task = "building"
spinner.Start(task)
defer func() {
spinner.Stop(task, err)
}()
workdir := fmt.Sprintf("/tmp/fx-%d", time.Now().Unix())
defer os.RemoveAll(workdir)
// Cases supports
// 1. a single file function
// fx up func.js
// 2. a directory with Docker in it
// fx up ./func/
// 3. a directory without Dockerfile in it, but has fx handle function file
// 4. a fx handlefunction file and its dependencies files or/and directory
// fx up func.js helper.js ./lib/
// When only one directory given and there is a Dockerfile in given directory, treat it as a containerized project and skip packing
sources := ctx.Get("sources").([]string)
if len(sources) == 0 {
return fmt.Errorf("source file/directory of function required")
}
if len(sources) == 1 &&
utils.IsDir(sources[0]) &&
utils.HasDockerfile(sources[0]) {
if err := copy.Copy(sources[0], workdir); err != nil {
return err
}
} else {
if err := packer.Pack(workdir, sources...); err != nil {
return err
}
}
cloudType := ctx.Get("cloud_type").(string)
name := ctx.Get("name").(string)
if cloudType == types.CloudTypeK8S {
data, err := packer.PackIntoK8SConfigMapFile(workdir)
if err != nil {
return err
}
ctx.Set("data", data)
} else {
docker := ctx.Get("docker").(containerruntimes.ContainerRuntime)
if err := docker.BuildImage(ctx.GetContext(), workdir, name); err != nil {
return err
}
nameWithTag := name + ":latest"
if err := docker.TagImage(ctx.GetContext(), name, nameWithTag); err != nil {
return err
}
ctx.Set("image", nameWithTag)
}
return nil
}

View File

@@ -0,0 +1,16 @@
package middlewares
import (
"github.com/metrue/fx/config"
"github.com/metrue/fx/context"
)
// LoadConfig load default config
func LoadConfig(ctx context.Contexter) error {
config, err := config.LoadDefault()
if err != nil {
return err
}
ctx.Set("config", config)
return nil
}

64
middlewares/parse.go Normal file
View File

@@ -0,0 +1,64 @@
package middlewares
import (
"fmt"
"github.com/google/uuid"
"github.com/metrue/fx/context"
)
// Parse parse input
func Parse(action string) func(ctx context.Contexter) (err error) {
return func(ctx context.Contexter) error {
cli := ctx.GetCliContext()
switch action {
case "up":
sources := []string{}
for _, s := range cli.Args() {
sources = append(sources, s)
}
ctx.Set("sources", sources)
name := cli.String("name")
ctx.Set("name", name)
port := cli.Int("port")
ctx.Set("port", port)
case "down":
services := cli.Args()
if len(services) == 0 {
return fmt.Errorf("service name required")
}
svc := []string{}
for _, service := range services {
svc = append(svc, service)
}
ctx.Set("services", svc)
case "list":
name := cli.Args().First()
ctx.Set("filter", name)
case "image_build":
sources := []string{}
for _, s := range cli.Args() {
sources = append(sources, s)
}
ctx.Set("sources", sources)
tag := cli.String("tag")
if tag == "" {
tag = uuid.New().String()
}
ctx.Set("tag", tag)
case "image_export":
sources := []string{}
for _, s := range cli.Args() {
sources = append(sources, s)
}
ctx.Set("sources", sources)
outputDir := cli.String("output")
if outputDir == "" {
return fmt.Errorf("output directory required")
}
ctx.Set("output", outputDir)
}
return nil
}
}

87
middlewares/provision.go Normal file
View File

@@ -0,0 +1,87 @@
package middlewares
import (
"encoding/json"
"fmt"
"os"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
dockerHTTP "github.com/metrue/fx/container_runtimes/docker/http"
"github.com/metrue/fx/context"
"github.com/metrue/fx/infra"
dockerInfra "github.com/metrue/fx/infra/docker"
k8sInfra "github.com/metrue/fx/infra/k8s"
"github.com/metrue/fx/types"
"github.com/pkg/errors"
)
// Provision make sure infrastructure is healthy
func Provision(ctx context.Contexter) (err error) {
fxConfig := ctx.Get("config").(*config.Config)
meta, err := fxConfig.GetCurrentCloud()
if err != nil {
return err
}
cloudType, err := fxConfig.GetCurrentCloudType()
if err != nil {
return err
}
ctx.Set("cloud_type", cloudType)
var cloud infra.Clouder
switch cloudType {
case types.CloudTypeK8S:
cloud, err = k8sInfra.Load(meta)
case types.CloudTypeDocker:
cloud, err = dockerInfra.Load(meta)
}
if err != nil {
return err
}
ctx.Set("cloud", cloud)
conf, err := cloud.GetConfig()
if err != nil {
return err
}
var deployer infra.Deployer
if os.Getenv("KUBECONFIG") != "" {
deployer, err = k8sInfra.CreateDeployer(os.Getenv("KUBECONFIG"))
if err != nil {
return err
}
ctx.Set("cloud_type", types.CloudTypeK8S)
} else if cloud.GetType() == types.CloudTypeDocker {
var meta map[string]string
if err := json.Unmarshal([]byte(conf), &meta); err != nil {
return err
}
docker, err := dockerHTTP.Create(meta["ip"], constants.AgentPort)
if err != nil {
return errors.Wrapf(err, "please make sure docker is installed and running on your host")
}
// TODO should clean up, but it needed in middlewares.Build
ctx.Set("docker", docker)
deployer, err = dockerInfra.CreateDeployer(docker)
if err != nil {
return err
}
} else if cloud.GetType() == types.CloudTypeK8S {
kubeconfig, err := fxConfig.GetKubeConfig()
if err != nil {
return err
}
deployer, err = k8sInfra.CreateDeployer(kubeconfig)
if err != nil {
return err
}
} else {
return fmt.Errorf("unsupport cloud type %s, please make sure you config is correct", cloud.GetType())
}
ctx.Set("deployer", deployer)
return nil
}

File diff suppressed because one or more lines are too long

14
packer/doc.go Normal file
View File

@@ -0,0 +1,14 @@
/*
Packer takes source codes of a function, and pack them into a containerized service, that means there is Dockerfile generated in the output directory
e.g.
Pack(output, "hello.js") # a single file function
Pack(output, "hello.js", "helper.js") # multiple files function
Pack(output, "./func/") # a directory of function
Pack(output, "hello.js", "./func/") # a directory and files of function
*/
package packer

View File

@@ -1,63 +0,0 @@
package packer
import (
"fmt"
"path/filepath"
"strings"
"github.com/gobuffalo/packr"
"github.com/metrue/fx/types"
)
// DockerPacker pack a function source code to a Docker build-able project
type DockerPacker struct {
box packr.Box
}
func isHandler(name string) bool {
basename := filepath.Base(name)
nameWithoutExt := strings.TrimSuffix(basename, filepath.Ext(basename))
return nameWithoutExt == "fx" ||
nameWithoutExt == "Fx" || // Fx is for Java
nameWithoutExt == "mod" // mod.rs is for Rust
}
// NewDockerPacker new a Docker packer
func NewDockerPacker(box packr.Box) *DockerPacker {
return &DockerPacker{box: box}
}
// Pack pack a single function source code to be project
func (p *DockerPacker) Pack(serviceName string, fn types.Func) (types.Project, error) {
var files []types.ProjectSourceFile
for _, name := range p.box.List() {
prefix := fmt.Sprintf("%s/", fn.Language)
if strings.HasPrefix(name, prefix) {
content, err := p.box.FindString(name)
if err != nil {
return types.Project{}, err
}
// if preset's file is handler function of project, replace it with give one
if isHandler(name) {
files = append(files, types.ProjectSourceFile{
Path: strings.Replace(name, prefix, "", 1),
Body: fn.Source,
IsHandler: true,
})
} else {
files = append(files, types.ProjectSourceFile{
Path: strings.Replace(name, prefix, "", 1),
Body: content,
IsHandler: false,
})
}
}
}
return types.Project{
Name: serviceName,
Files: files,
Language: fn.Language,
}, nil
}

View File

@@ -1,64 +0,0 @@
package packer
import (
"testing"
"github.com/gobuffalo/packr"
"github.com/golang/mock/gomock"
"github.com/metrue/fx/types"
)
func TestPacker(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
box := packr.NewBox("./images")
p := NewDockerPacker(box)
mockSource := `
module.exports = ({a, b}) => {
return a + b
}
`
fn := types.ServiceFunctionSource{
Language: "node",
Source: mockSource,
}
serviceName := "service-mock"
project, err := p.Pack(serviceName, fn)
if err != nil {
t.Fatal(err)
}
if project.Name != serviceName {
t.Fatalf("should get %s but got %s", serviceName, project.Name)
}
if project.Language != "node" {
t.Fatal("incorrect Language")
}
if len(project.Files) != 3 {
t.Fatal("node project should have 3 files")
}
for _, file := range project.Files {
if file.Path == "fx.js" {
if file.IsHandler == false {
t.Fatal("fx.js should be handler")
}
if file.Body != mockSource {
t.Fatalf("should get %s but got %v", mockSource, file.Body)
}
} else if file.Path == "Dockerfile" {
if file.IsHandler == true {
t.Fatalf("should get %v but got %v", false, file.IsHandler)
}
} else {
if file.IsHandler == true {
t.Fatalf("should get %v but %v", false, file.IsHandler)
}
}
}
}

View File

@@ -0,0 +1,5 @@
FROM metrue/fx-node-base
COPY . .
EXPOSE 3000
CMD ["node", "app.js"]

9
packer/fixture/p1/app.js Normal file
View File

@@ -0,0 +1,9 @@
const Koa = require('koa');
const bodyParser = require('koa-bodyparser');
const fx = require('./fx');
const app = new Koa();
app.use(bodyParser());
app.use(fx);
app.listen(3000);

Some files were not shown because too many files have changed in this diff Show More