Compare commits

...

8 Commits

Author SHA1 Message Date
Minghe
06f87c4d8e fix deploy to aks issue, and update docs (#396)
* fix deploy to aks issue, and update docs
* update docs
2019-12-06 11:54:27 +08:00
Minghe
35262de828 release v0.8.7 (#394) 2019-12-06 10:04:40 +08:00
dependabot-preview[bot]
34a495984c Bump github.com/olekukonko/tablewriter from 0.0.3 to 0.0.4 (#393)
Bumps [github.com/olekukonko/tablewriter](https://github.com/olekukonko/tablewriter) from 0.0.3 to 0.0.4.
- [Release notes](https://github.com/olekukonko/tablewriter/releases)
- [Commits](https://github.com/olekukonko/tablewriter/compare/v0.0.3...v0.0.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-06 09:45:27 +08:00
Minghe
d7130c4e28 support a project with multiple files (#392)
* support a project with multiple files
* fix lint issue
* fix test
2019-12-05 17:57:39 +08:00
Minghe
c9630a53c3 should treat CICD code block seperately (#391)
* should treat CICD code block seperately

* auto provision when host is not health

* fix lint
2019-12-05 11:38:54 +08:00
Minghe
0522690472 merge provisioner and deployer into infra (#390) 2019-12-04 19:09:14 +08:00
Minghe
a8a0fbed32 add ping method to check infra is healthy or not (#388)
* add ping method to check infra is healthy or not
* merge k3s and k8s deployer (#389)
* fix typo
2019-12-04 17:28:03 +08:00
Minghe
26ae9585f6 move cli args parsing into middleware level (#387) 2019-12-04 11:56:07 +08:00
63 changed files with 1163 additions and 1140 deletions

View File

@@ -1,5 +1,6 @@
fx
------
Poor man's function as a service.
<br/>
![CI](https://github.com/metrue/fx/workflows/ci/badge.svg)
@@ -13,6 +14,7 @@ Poor man's function as a service.
- [Introduction](#introduction)
- [Installation](#installation)
- [Usage](#usage)
- [Manage Infrastructure](#manage-infrastructure)
- [Contribute](#contribute)
@@ -79,7 +81,7 @@ USAGE:
fx [global options] command [command options] [arguments...]
VERSION:
0.8.4
0.8.7
COMMANDS:
infra manage infrastructure
@@ -96,44 +98,36 @@ GLOBAL OPTIONS:
--version, -v print the version
```
1. Write a function
### Deploy your function to Docker
You can check out [examples](https://github.com/metrue/fx/tree/master/examples/functions) for reference. Let's write a function as an example, it calculates the sum of two numbers then returns:
```js
module.exports = (ctx) => {
ctx.body = 'hello world'
}
```
Then save it to a file `func.js`.
$ fx up --name hello-fx ./examples/functions/JavaScript/func.js
2. Deploy your function as a service
Give your service a port with `--port`, and name with `--name`, heath checking with `--healthcheck` if you want.
```shell
$ fx up -name fx_service_name -p 10001 --healthcheck func.js
2019/08/10 13:26:37 info Pack Service: ✓
2019/08/10 13:26:39 info Build Service: ✓
2019/08/10 13:26:39 info Run Service: ✓
2019/08/10 13:26:39 info Service (fx_service_name) is running on: 0.0.0.0:10001
2019/08/10 13:26:39 info up function fx_service_name(func.js) to machine localhost: ✓
+------------------------------------------------------------------+-----------+---------------+
| ID | NAME | ENDPOINT |
+------------------------------------------------------------------+-----------+---------------+
| 5b24d36608ee392c937a61a530805f74551ddec304aea3aca2ffa0fabcf98cf3 | /hello-fx | 0.0.0.0:58328 |
+------------------------------------------------------------------+-----------+---------------+
```
if you want see what the source code of your service looks like, you can export it into a dirctory,
### Deploy your function to Kubernetes
```shell
$ fx image export -o <path of dir> func.js
2019/09/25 19:31:19 info exported to <path of dir>: ✓
```
$ KUBECONFIG=~/.kube/config ./build/fx up examples/functions/JavaScript/func.js --name hello-fx
+-------------------------------+------+----------------+
| ID | NAME | ENDPOINT |
+----+--------------------------+-----------------------+
| 5b24d36608ee392c937a | hello-fx | 10.0.242.75:80 |
+------------------------+-------------+----------------+
```
3. Test your service
### Test your service
then you can test your service:
```shell
$ curl -v 0.0.0.0:10001
$ curl -v 0.0.0.0:58328
GET / HTTP/1.1
@@ -155,39 +149,32 @@ hello world
```
## Docker
## Manage Infrastructure
**fx** is originally designed to turn a function into a runnable Docker container in a easiest way, on a host with Docker running, you can just deploy your function with `fx up` command,
**fx** is originally designed to turn a function into a runnable Docker container in a easiest way, on a host with Docker running, you can just deploy your function with `fx up` command, and now **fx** supports deploy function to be a service onto Kubernetes cluster infrasture, and we encourage you to do that other than on bare Docker environment, there are lots of advantage to run your function on Kubernetes like self-healing, load balancing, easy horizontal scaling, etc. It's pretty simple to deploy your function onto Kubernetes with **fx**, you just set KUBECONFIG in your enviroment.
By default. **fx** use localhost as target infrastructure to run your service, and you can also setup your remote virtual machines as **fx**'s infrastructure and deploy your functions onto it.
### `fx infra create`
You can create types (docker and k8s) of infrastructures for **fx** to deploy functions
```shell
fx up --name hello-svc --port 7777 hello.js # onto localhost
DOCKER_REMOTE_HOST_ADDR=xx.xx.xx.xx DOCKER_REMOTE_HOST_USER=xxxx DOCKER_REMOTE_HOST_PASSWORD=xxxx fx up --name hello-svc --port 7777 hello.js # onto remote host
$ fx infra create --name infra_us --type docker --host <user>@<ip> ## create docker type infrasture on <ip>
$ fx infra create --name infra_bj --type k8s --master <user>@<ip> --agents '<user1>@<ip1>,<user2>@<ip2>' ## create k8s type infrasture use <ip> as master node, and <ip1> and <ip2> as agents nodes
```
## Kubernetes
### `fx infra use`
**fx** supports deploy function to be a service onto Kubernetes cluster infrasture, and we encourage you to do that other than on bare Docker environment, there are lots of advantage to run your function on Kubernetes like self-healing, load balancing, easy horizontal scaling, etc. It's pretty simple to deploy your function onto Kubernetes with **fx**, you just set KUBECONFIG in your enviroment.
To use a infrastructure, you can use `fx infra use` command to activate it.
```shell
KUBECONFIG=<Your KUBECONFIG> fx deploy -n fx-service-abc_js -p 12349 examples/functions/JavaScript/func.js # function will be deploy to your Kubernetes cluster and expose a IP address of your loadbalencer
fx infra use <infrastructure name>
```
or
and you can list your infrastructure with `fx infra list`
```shell
$ export KUBECONFIG=<Your KUBECONFIG>
$ fx deploy -n fx-service-abc_js -p 12349 examples/functions/JavaScript/func.js # function will be deploy to your Kubernetes cluster and expose a IP address of your loadbalencer
```
* Local Kubernetes Cluster
Docker for Mac and Docker for Windows already support Kubernetes with single node cluster, we can use it directly, and the default `KUBECONFIG` is `~/.kube/config`.
```shell
$ export KUBECONFIG=~/.kube/config # then fx will take the config to deloy function
```
if you have multiple Kubernetes clusters configured, you have to set context correctly. FYI [configure-access-multiple-clusters](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/)
## Use Public Cloud Kubernetes Service as infrastructure to run your functions
* Azure Kubernetes Service (AKS)

View File

@@ -129,6 +129,41 @@ func (api *API) post(path string, body []byte, expectStatus int, v interface{})
return nil
}
// Version get version of docker engine
func (api *API) Version(ctx context.Context) (string, error) {
path := api.endpoint + "/version"
if !strings.HasPrefix(path, "http") {
path = "http://" + path
}
req, err := http.NewRequest("GET", path, nil)
if err != nil {
return "", err
}
client := &http.Client{Timeout: 20 * time.Second}
resp, err := client.Do(req)
if err != nil {
return "", err
}
if resp.StatusCode != 200 {
return "", fmt.Errorf("request %s failed: %d - %s", path, resp.StatusCode, resp.Status)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return "", err
}
var res dockerTypes.Version
err = json.Unmarshal(body, &res)
if err != nil {
return "", err
}
return res.APIVersion, nil
}
// ListContainer list service
func (api *API) ListContainer(ctx context.Context, name string) ([]types.Service, error) {
if name != "" {

View File

@@ -235,6 +235,15 @@ func (d *Docker) ListContainer(ctx context.Context, name string) ([]types.Servic
return services, nil
}
// Version get version of docker engine
func (d *Docker) Version(ctx context.Context) (string, error) {
ping, err := d.Ping(ctx)
if err != nil {
return "", err
}
return ping.APIVersion, nil
}
var (
_ containerruntimes.ContainerRuntime = &Docker{}
)

View File

@@ -16,4 +16,5 @@ type ContainerRuntime interface {
StopContainer(ctx context.Context, name string) error
InspectContainer(ctx context.Context, name string, container interface{}) error
ListContainer(ctx context.Context, filter string) ([]types.Service, error)
Version(ctx context.Context) (string, error)
}

View File

@@ -1,16 +0,0 @@
package deploy
import (
"context"
types "github.com/metrue/fx/types"
)
// Deployer make a image a service
type Deployer interface {
Deploy(ctx context.Context, fn types.Func, name string, image string, bindings []types.PortBinding) error
Destroy(ctx context.Context, name string) error
Update(ctx context.Context, name string) error
GetStatus(ctx context.Context, name string) (types.Service, error)
List(ctx context.Context, name string) ([]types.Service, error)
}

View File

@@ -1,8 +0,0 @@
package k3s
// ConfigMap is the key to function docker project source code in configmap
var ConfigMap = struct {
AppMetaEnvName string
}{
AppMetaEnvName: "APP_META",
}

View File

@@ -1,102 +0,0 @@
package k3s
import (
"fmt"
"github.com/metrue/fx/types"
appsv1 "k8s.io/api/apps/v1"
apiv1 "k8s.io/api/core/v1"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func generateDeploymentSpec(
name string,
image string,
bindPorts []types.PortBinding,
replicas int32,
selector map[string]string,
) *appsv1.Deployment {
ports := []apiv1.ContainerPort{}
for index, binding := range bindPorts {
ports = append(ports, apiv1.ContainerPort{
Name: fmt.Sprintf("fx-container-%d", index),
ContainerPort: binding.ContainerExposePort,
})
}
container := apiv1.Container{
Name: "fx-placeholder-container-name",
Image: image,
Ports: ports,
ImagePullPolicy: v1.PullIfNotPresent,
}
return &appsv1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Name: name,
},
Spec: appsv1.DeploymentSpec{
Replicas: &replicas,
Selector: &metav1.LabelSelector{
MatchLabels: selector,
},
Template: apiv1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: selector,
},
Spec: apiv1.PodSpec{
Containers: []apiv1.Container{container},
},
},
},
}
}
// GetDeployment get a deployment
func (k *K3S) GetDeployment(namespace string, name string) (*appsv1.Deployment, error) {
return k.AppsV1().Deployments(namespace).Get(name, metav1.GetOptions{})
}
// CreateDeployment create a deployment
func (k *K3S) CreateDeployment(
namespace string,
name string,
image string,
ports []types.PortBinding,
replicas int32,
selector map[string]string,
) (*appsv1.Deployment, error) {
deployment := generateDeploymentSpec(name, image, ports, replicas, selector)
return k.AppsV1().Deployments(namespace).Create(deployment)
}
// UpdateDeployment update a deployment
func (k *K3S) UpdateDeployment(
namespace string,
name string,
image string,
ports []types.PortBinding,
replicas int32,
selector map[string]string,
) (*appsv1.Deployment, error) {
deployment := generateDeploymentSpec(name, image, ports, replicas, selector)
return k.AppsV1().Deployments(namespace).Update(deployment)
}
// DeleteDeployment delete a deployment
func (k *K3S) DeleteDeployment(namespace string, name string) error {
return k.AppsV1().Deployments(namespace).Delete(name, &metav1.DeleteOptions{})
}
// CreateDeploymentWithInitContainer create a deployment which will wait InitContainer to do the image build before function container start
func (k *K3S) CreateDeploymentWithInitContainer(
namespace string,
name string,
ports []types.PortBinding,
replicas int32,
selector map[string]string,
) (*appsv1.Deployment, error) {
deployment := generateDeploymentSpec(name, name, ports, replicas, selector)
updatedDeployment := injectInitContainer(name, deployment)
return k.AppsV1().Deployments(namespace).Create(updatedDeployment)
}

View File

@@ -1,61 +0,0 @@
package k3s
import (
"os"
"testing"
"github.com/metrue/fx/types"
)
func TestDeployment(t *testing.T) {
namespace := "default"
name := "fx-hello-world"
image := "metrue/kube-hello"
selector := map[string]string{
"app": "fx-app",
}
kubeconfig := os.Getenv("KUBECONFIG")
if kubeconfig == "" {
t.Skip("skip test since no KUBECONFIG given in environment variable")
}
k8s, err := Create("")
if err != nil {
t.Fatal(err)
}
if _, err := k8s.GetDeployment(namespace, name); err == nil {
t.Fatalf("should get not found error")
}
replicas := int32(2)
bindings := []types.PortBinding{
types.PortBinding{
ServiceBindingPort: 80,
ContainerExposePort: 3000,
},
types.PortBinding{
ServiceBindingPort: 443,
ContainerExposePort: 3000,
},
}
deployment, err := k8s.CreateDeployment(namespace, name, image, bindings, replicas, selector)
if err != nil {
t.Fatal(err)
}
if deployment == nil {
t.Fatalf("deploymetn should not be %v", nil)
}
if deployment.Name != name {
t.Fatalf("should get %s but got %s", name, deployment.Name)
}
if *deployment.Spec.Replicas != replicas {
t.Fatalf("should get %v but got %v", replicas, deployment.Spec.Replicas)
}
if err := k8s.DeleteDeployment(namespace, name); err != nil {
t.Fatal(err)
}
}

View File

@@ -1,70 +0,0 @@
package k3s
import (
appsv1 "k8s.io/api/apps/v1"
apiv1 "k8s.io/api/core/v1"
v1 "k8s.io/api/core/v1"
)
// This is docker image provided by fx/contrib/docker_packer
// it can build a Docker image with give Docker project source codes encoded with base64
// check the detail fx/contrib/docker_packer/main.go
const image = "metrue/fx-docker"
func injectInitContainer(name string, deployment *appsv1.Deployment) *appsv1.Deployment {
configMapHasToBeReady := true
valueInConfigMapHasToBeReady := true
initContainer := v1.Container{
Name: "fx-docker-build-c",
Image: image,
ImagePullPolicy: v1.PullAlways,
Command: []string{
"/bin/sh",
"-c",
"/usr/bin/docker_packer $(APP_META) " + name,
}, // Maybe it can be passed by Binary data from config map
// Args: []string{"${APP_META}"}, // function source codes and name
VolumeMounts: []v1.VolumeMount{
v1.VolumeMount{
Name: "dockersock",
MountPath: "/var/run/docker.sock",
},
},
Env: []v1.EnvVar{
v1.EnvVar{
Name: ConfigMap.AppMetaEnvName,
ValueFrom: &v1.EnvVarSource{
ConfigMapKeyRef: &v1.ConfigMapKeySelector{
LocalObjectReference: v1.LocalObjectReference{Name: name},
Key: ConfigMap.AppMetaEnvName,
Optional: &valueInConfigMapHasToBeReady,
},
},
},
},
EnvFrom: []v1.EnvFromSource{
v1.EnvFromSource{
ConfigMapRef: &v1.ConfigMapEnvSource{
LocalObjectReference: v1.LocalObjectReference{
Name: name,
},
Optional: &configMapHasToBeReady,
},
},
},
}
volumes := []v1.Volume{
v1.Volume{
Name: "dockersock",
VolumeSource: v1.VolumeSource{
HostPath: &v1.HostPathVolumeSource{
Path: "/var/run/docker.sock",
},
},
},
}
deployment.Spec.Template.Spec.InitContainers = []apiv1.Container{initContainer}
deployment.Spec.Template.Spec.Volumes = volumes
return deployment
}

View File

@@ -1,151 +0,0 @@
package k3s
import (
"context"
"os"
"github.com/metrue/fx/deploy"
"github.com/metrue/fx/types"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
)
// K3S client
type K3S struct {
*kubernetes.Clientset
}
const namespace = "default"
// Create a k8s cluster client
func Create(kubeconfig string) (*K3S, error) {
if os.Getenv("KUBECONFIG") != "" {
kubeconfig = os.Getenv("KUBECONFIG")
}
config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
if err != nil {
return nil, err
}
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
return nil, err
}
return &K3S{clientset}, nil
}
// Deploy a image to be a service
func (k *K3S) Deploy(
ctx context.Context,
fn types.Func,
name string,
image string,
ports []types.PortBinding,
) error {
selector := map[string]string{
"app": "fx-app-" + name,
}
const replicas = int32(3)
if _, err := k.GetDeployment(namespace, name); err != nil {
// TODO enable passing replica from fx CLI
if _, err := k.CreateDeployment(
namespace,
name,
image,
ports,
replicas,
selector,
); err != nil {
return err
}
} else {
if _, err := k.UpdateDeployment(
namespace,
name,
image,
ports,
replicas,
selector,
); err != nil {
return err
}
}
// TODO fx should be able to know what's the target Kubernetes service platform
// it's going to deploy to
typ := "LoadBalancer"
if os.Getenv("SERVICE_TYPE") != "" {
typ = os.Getenv("SERVICE_TYPE")
}
if _, err := k.GetService(namespace, name); err != nil {
if _, err := k.CreateService(
namespace,
name,
typ,
ports,
selector,
); err != nil {
return err
}
} else {
if _, err := k.UpdateService(
namespace,
name,
typ,
ports,
selector,
); err != nil {
return err
}
}
return nil
}
// Update a service
func (k *K3S) Update(ctx context.Context, name string) error {
return nil
}
// Destroy a service
func (k *K3S) Destroy(ctx context.Context, name string) error {
if err := k.DeleteService(namespace, name); err != nil {
return err
}
if err := k.DeleteDeployment(namespace, name); err != nil {
return err
}
return nil
}
// GetStatus get status of a service
func (k *K3S) GetStatus(ctx context.Context, name string) (types.Service, error) {
svc, err := k.GetService(namespace, name)
service := types.Service{}
if err != nil {
return service, err
}
service.Host = svc.Spec.ClusterIP
if len(svc.Spec.ExternalIPs) > 0 {
service.Host = svc.Spec.ExternalIPs[0]
}
for _, port := range svc.Spec.Ports {
// TODO should clearify which port (target port, node port) should use
service.Port = int(port.Port)
break
}
return service, nil
}
// List services
func (k *K3S) List(ctx context.Context, name string) ([]types.Service, error) {
return []types.Service{}, nil
}
var (
_ deploy.Deployer = &K3S{}
)

View File

@@ -1,87 +0,0 @@
package k3s
import (
"strconv"
"github.com/metrue/fx/types"
apiv1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
intstr "k8s.io/apimachinery/pkg/util/intstr"
)
func generateServiceSpec(
namespace string,
name string,
typ string,
bindings []types.PortBinding,
selector map[string]string,
) *apiv1.Service {
servicePorts := []apiv1.ServicePort{}
for index, binding := range bindings {
servicePorts = append(servicePorts, apiv1.ServicePort{
Name: "port-" + strconv.Itoa(index),
Protocol: apiv1.ProtocolTCP,
Port: binding.ServiceBindingPort,
TargetPort: intstr.FromInt(int(binding.ContainerExposePort)),
})
}
return &apiv1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: name,
ClusterName: namespace,
},
Spec: apiv1.ServiceSpec{
Ports: servicePorts,
Type: apiv1.ServiceType(typ),
Selector: selector,
},
}
}
// CreateService create a service
func (k *K3S) CreateService(
namespace string,
name string,
typ string,
bindings []types.PortBinding,
selector map[string]string,
) (*apiv1.Service, error) {
service := generateServiceSpec(namespace, name, typ, bindings, selector)
createdService, err := k.CoreV1().Services(namespace).Create(service)
if err != nil {
return nil, err
}
return createdService, nil
}
// UpdateService update a service
// TODO this method is not perfect yet, should refactor later
func (k *K3S) UpdateService(
namespace string,
name string,
typ string,
bindings []types.PortBinding,
selector map[string]string,
) (*apiv1.Service, error) {
svc, err := k.GetService(namespace, name)
if err != nil {
return nil, err
}
svc.Spec.Selector = selector
svc.Spec.Type = apiv1.ServiceType(typ)
return k.CoreV1().Services(namespace).Update(svc)
}
// DeleteService a service
func (k *K3S) DeleteService(namespace string, name string) error {
// TODO figure out the elegant way to delete a service
options := &metav1.DeleteOptions{}
return k.CoreV1().Services(namespace).Delete(name, options)
}
// GetService get a service
func (k *K3S) GetService(namespace string, name string) (*apiv1.Service, error) {
return k.CoreV1().Services(namespace).Get(name, metav1.GetOptions{})
}

View File

@@ -1,107 +0,0 @@
// Code generated by MockGen. DO NOT EDIT.
// Source: deployer.go
// Package mock_deploy is a generated GoMock package.
package mock_deploy
import (
context "context"
gomock "github.com/golang/mock/gomock"
types "github.com/metrue/fx/types"
reflect "reflect"
)
// MockDeployer is a mock of Deployer interface
type MockDeployer struct {
ctrl *gomock.Controller
recorder *MockDeployerMockRecorder
}
// MockDeployerMockRecorder is the mock recorder for MockDeployer
type MockDeployerMockRecorder struct {
mock *MockDeployer
}
// NewMockDeployer creates a new mock instance
func NewMockDeployer(ctrl *gomock.Controller) *MockDeployer {
mock := &MockDeployer{ctrl: ctrl}
mock.recorder = &MockDeployerMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockDeployer) EXPECT() *MockDeployerMockRecorder {
return m.recorder
}
// Deploy mocks base method
func (m *MockDeployer) Deploy(ctx context.Context, fn types.Func, name, image string, bindings []types.PortBinding) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Deploy", ctx, fn, name, image, bindings)
ret0, _ := ret[0].(error)
return ret0
}
// Deploy indicates an expected call of Deploy
func (mr *MockDeployerMockRecorder) Deploy(ctx, fn, name, image, bindings interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Deploy", reflect.TypeOf((*MockDeployer)(nil).Deploy), ctx, fn, name, image, bindings)
}
// Destroy mocks base method
func (m *MockDeployer) Destroy(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Destroy", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// Destroy indicates an expected call of Destroy
func (mr *MockDeployerMockRecorder) Destroy(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Destroy", reflect.TypeOf((*MockDeployer)(nil).Destroy), ctx, name)
}
// Update mocks base method
func (m *MockDeployer) Update(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Update", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// Update indicates an expected call of Update
func (mr *MockDeployerMockRecorder) Update(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Update", reflect.TypeOf((*MockDeployer)(nil).Update), ctx, name)
}
// GetStatus mocks base method
func (m *MockDeployer) GetStatus(ctx context.Context, name string) (types.Service, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetStatus", ctx, name)
ret0, _ := ret[0].(types.Service)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetStatus indicates an expected call of GetStatus
func (mr *MockDeployerMockRecorder) GetStatus(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetStatus", reflect.TypeOf((*MockDeployer)(nil).GetStatus), ctx, name)
}
// List mocks base method
func (m *MockDeployer) List(ctx context.Context, name string) ([]types.Service, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "List", ctx, name)
ret0, _ := ret[0].([]types.Service)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// List indicates an expected call of List
func (mr *MockDeployerMockRecorder) List(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "List", reflect.TypeOf((*MockDeployer)(nil).List), ctx, name)
}

16
fx.go
View File

@@ -16,7 +16,7 @@ import (
"github.com/urfave/cli"
)
const version = "0.8.6"
const version = "0.8.71"
func init() {
go checkForUpdate()
@@ -156,8 +156,8 @@ func main() {
},
Action: handle(
middlewares.LoadConfig,
middlewares.Setup,
middlewares.Parse,
middlewares.Provision,
middlewares.Parse("up"),
middlewares.Binding,
middlewares.Build,
handlers.Up,
@@ -168,8 +168,9 @@ func main() {
Usage: "destroy a service",
ArgsUsage: "[service 1, service 2, ....]",
Action: handle(
middlewares.Parse("down"),
middlewares.LoadConfig,
middlewares.Setup,
middlewares.Provision,
handlers.Down,
),
},
@@ -178,8 +179,9 @@ func main() {
Aliases: []string{"ls"},
Usage: "list deployed services",
Action: handle(
middlewares.Parse("list"),
middlewares.LoadConfig,
middlewares.Setup,
middlewares.Provision,
handlers.List,
),
},
@@ -209,7 +211,7 @@ func main() {
},
Action: handle(
middlewares.LoadConfig,
middlewares.Setup,
middlewares.Provision,
handlers.BuildImage,
),
},
@@ -224,7 +226,7 @@ func main() {
},
Action: handle(
middlewares.LoadConfig,
middlewares.Setup,
middlewares.Provision,
handlers.ExportImage,
),
},

3
go.mod
View File

@@ -27,9 +27,10 @@ require (
github.com/mitchellh/go-homedir v1.1.0
github.com/morikuni/aec v1.0.0 // indirect
github.com/nwaples/rardecode v1.0.0 // indirect
github.com/olekukonko/tablewriter v0.0.3
github.com/olekukonko/tablewriter v0.0.4
github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
github.com/opencontainers/image-spec v1.0.1 // indirect
github.com/otiai10/copy v1.0.2
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
github.com/pierrec/lz4 v0.0.0-20190222153722-062282ea0dcf // indirect
github.com/pkg/errors v0.8.1

8
go.sum
View File

@@ -163,6 +163,8 @@ github.com/mattn/go-isatty v0.0.8 h1:HLtExJ+uU2HOZ+wI0Tt5DtUDrx8yhUqDcp7fYERX4CE
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-runewidth v0.0.6 h1:V2iyH+aX9C5fsYCpK60U8BYIvmhqxuOL3JZcqc1NB7k=
github.com/mattn/go-runewidth v0.0.6/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/mattn/go-runewidth v0.0.7 h1:Ei8KR0497xHyKJPAv59M1dkC+rOZCMBJ+t3fZ+twI54=
github.com/mattn/go-runewidth v0.0.7/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/metrue/go-ssh-client v0.0.0-20191125030649-4ac058ee958b h1:JGD0sJ44XzhsT1voOg00zji4ubuMNcVNK3m7d9GI88k=
github.com/metrue/go-ssh-client v0.0.0-20191125030649-4ac058ee958b/go.mod h1:ERHOEBrDy6+8vfoJjjmhdmBpOzdvvP7bLtwYTTK6LOs=
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
@@ -187,6 +189,8 @@ github.com/nwaples/rardecode v1.0.0 h1:r7vGuS5akxOnR4JQSkko62RJ1ReCMXxQRPtxsiFMB
github.com/nwaples/rardecode v1.0.0/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0=
github.com/olekukonko/tablewriter v0.0.3 h1:i0LBnzgiChAWHJYTQAZJDOgf8MNxAVYZJ2m63SIDimI=
github.com/olekukonko/tablewriter v0.0.3/go.mod h1:YZeBtGzYYEsCHp2LST/u/0NDwGkRoBtmn1cIWCJiS6M=
github.com/olekukonko/tablewriter v0.0.4 h1:vHD/YYe1Wolo78koG299f7V/VAS08c6IpCLn+Ejf/w8=
github.com/olekukonko/tablewriter v0.0.4/go.mod h1:zq6QwlOf5SlnkVbMSr5EoBv3636FWnp+qbPhuoO21uA=
github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
@@ -196,6 +200,10 @@ github.com/opencontainers/go-digest v1.0.0-rc1 h1:WzifXhOVOEOuFYOJAW6aQqW0TooG2i
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVojFA6h/TRcI=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/otiai10/copy v1.0.2 h1:DDNipYy6RkIkjMwy+AWzgKiNTyj2RUI9yEMeETEpVyc=
github.com/otiai10/copy v1.0.2/go.mod h1:c7RpqBkwMom4bYTSkLSym4VSJz/XtncWRAj/J4PEIMY=
github.com/otiai10/curr v0.0.0-20150429015615-9b4961190c95/go.mod h1:9qAhocn7zKJG+0mI8eUu6xqkFDYS2kb2saOteoSB3cE=
github.com/otiai10/mint v1.3.0/go.mod h1:F5AjcsTsWUqX+Na9fpHb52P8pcRX2CI6A3ctIT91xUo=
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2 h1:JhzVVoYvbOACxoUmOs6V/G4D5nPVUW73rKvXxP4XUJc=

View File

@@ -1,43 +1,11 @@
package handlers
import (
"io/ioutil"
"strings"
"github.com/apex/log"
"github.com/metrue/fx/context"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
)
// Call command handle
func Call(ctx context.Contexter) error {
cli := ctx.GetCliContext()
_ = strings.Join(cli.Args()[1:], " ")
file := cli.Args().First()
src, err := ioutil.ReadFile(file)
if err != nil {
log.Fatalf("Read Source: %v", err)
return err
}
log.Info("Read Source: \u2713")
lang := utils.GetLangFromFileName(file)
fn := types.Func{
Language: lang,
Source: string(src),
}
if _, err := packer.Pack(file, fn); err != nil {
panic(err)
}
// TODO not supported
// if err := api.MustCreate(host.Host, constants.AgentPort).
// Call(file, params, project); err != nil {
// log.Fatalf("call functions on machine %s with %v failed: %v", name, params, err)
// }
return nil
}

View File

@@ -2,21 +2,13 @@ package handlers
import (
"github.com/metrue/fx/context"
"github.com/metrue/fx/deploy"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/infra"
)
// Down command handle
func Down(ctx context.Contexter) (err error) {
const task = "destroying"
spinner.Start(task)
defer func() {
spinner.Stop(task, err)
}()
cli := ctx.GetCliContext()
services := cli.Args()
runner := ctx.Get("deployer").(deploy.Deployer)
services := ctx.Get("services").([]string)
runner := ctx.Get("deployer").(infra.Deployer)
for _, svc := range services {
if err := runner.Destroy(ctx.GetContext(), svc); err != nil {
return err

27
handlers/down_test.go Normal file
View File

@@ -0,0 +1,27 @@
package handlers
import (
"context"
"testing"
"github.com/golang/mock/gomock"
mockCtx "github.com/metrue/fx/context/mocks"
mockDeployer "github.com/metrue/fx/infra/mocks"
)
func TestDown(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()
ctx := mockCtx.NewMockContexter(ctrl)
deployer := mockDeployer.NewMockDeployer(ctrl)
services := []string{"sample-name"}
ctx.EXPECT().Get("services").Return(services)
ctx.EXPECT().Get("deployer").Return(deployer)
ctx.EXPECT().GetContext().Return(context.Background())
deployer.EXPECT().Destroy(gomock.Any(), services[0]).Return(nil)
if err := Down(ctx); err != nil {
t.Fatal(err)
}
}

View File

@@ -6,21 +6,21 @@ import (
"github.com/metrue/fx/config"
"github.com/metrue/fx/context"
"github.com/metrue/fx/infra/docker"
"github.com/metrue/fx/infra/k3s"
dockerInfra "github.com/metrue/fx/infra/docker"
"github.com/metrue/fx/infra/k8s"
"github.com/metrue/fx/pkg/spinner"
)
func setupK3S(masterInfo string, agentsInfo string) ([]byte, error) {
func setupK8S(masterInfo string, agentsInfo string) ([]byte, error) {
info := strings.Split(masterInfo, "@")
if len(info) != 2 {
return nil, fmt.Errorf("incorrect master info, should be <user>@<ip> format")
}
master := k3s.MasterNode{
master := k8s.MasterNode{
User: info[0],
IP: info[1],
}
agents := []k3s.AgentNode{}
agents := []k8s.AgentNode{}
if agentsInfo != "" {
agentsInfoList := strings.Split(agentsInfo, ",")
for _, agent := range agentsInfoList {
@@ -28,7 +28,7 @@ func setupK3S(masterInfo string, agentsInfo string) ([]byte, error) {
if len(info) != 2 {
return nil, fmt.Errorf("incorrect agent info, should be <user>@<ip> format")
}
agents = append(agents, k3s.AgentNode{
agents = append(agents, k8s.AgentNode{
User: info[0],
IP: info[1],
})
@@ -36,8 +36,8 @@ func setupK3S(masterInfo string, agentsInfo string) ([]byte, error) {
}
fmt.Println(master, agents, len(agents))
k3sOperator := k3s.New(master, agents)
return k3sOperator.Provision()
k8sOperator := k8s.New(master, agents)
return k8sOperator.Provision()
}
func setupDocker(hostInfo string) ([]byte, error) {
@@ -47,7 +47,7 @@ func setupDocker(hostInfo string) ([]byte, error) {
}
user := info[1]
host := info[0]
dockr := docker.New(user, host)
dockr := dockerInfra.CreateProvisioner(user, host)
return dockr.Provision()
}
@@ -69,20 +69,19 @@ func Setup(ctx context.Contexter) (err error) {
if cli.String("host") == "" {
return fmt.Errorf("host required, eg. 'root@123.1.2.12'")
}
} else if typ == "k3s" {
} else if typ == "k8s" {
if cli.String("master") == "" {
return fmt.Errorf("master required, eg. 'root@123.1.2.12'")
}
} else if typ == "k8s" {
} else {
return fmt.Errorf("invalid type, 'docker', 'k3s' and 'k8s' support")
return fmt.Errorf("invalid type, 'docker' and 'k8s' support")
}
fxConfig := ctx.Get("config").(*config.Config)
switch strings.ToLower(typ) {
case "k3s":
kubeconf, err := setupK3S(cli.String("master"), cli.String("agents"))
case "k8s":
kubeconf, err := setupK8S(cli.String("master"), cli.String("agents"))
if err != nil {
return err
}
@@ -93,8 +92,6 @@ func Setup(ctx context.Contexter) (err error) {
return err
}
return fxConfig.AddDockerCloud(name, config)
case "k8s":
fmt.Println("WIP")
}
return nil
}

View File

@@ -2,21 +2,14 @@ package handlers
import (
"github.com/metrue/fx/context"
"github.com/metrue/fx/deploy"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/pkg/render"
"github.com/metrue/fx/pkg/spinner"
)
// List command handle
func List(ctx context.Contexter) (err error) {
const task = "deploying"
spinner.Start(task)
defer func() {
spinner.Stop(task, err)
}()
cli := ctx.GetCliContext()
deployer := ctx.Get("deployer").(deploy.Deployer)
deployer := ctx.Get("deployer").(infra.Deployer)
services, err := deployer.List(ctx.GetContext(), cli.Args().First())
if err != nil {

View File

@@ -2,17 +2,23 @@ package handlers
import (
"github.com/metrue/fx/context"
"github.com/metrue/fx/deploy"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/pkg/render"
"github.com/metrue/fx/types"
)
// Up command handle
func Up(ctx context.Contexter) (err error) {
fn := ctx.Get("fn").(types.Func)
image := ctx.Get("image").(string)
fn, ok := ctx.Get("data").(string)
if !ok {
fn = ""
}
image, ok := ctx.Get("image").(string)
if !ok {
image = ""
}
name := ctx.Get("name").(string)
deployer := ctx.Get("deployer").(deploy.Deployer)
deployer := ctx.Get("deployer").(infra.Deployer)
bindings := ctx.Get("bindings").([]types.PortBinding)
if err := deployer.Deploy(

View File

@@ -6,9 +6,8 @@ import (
"github.com/golang/mock/gomock"
mockCtx "github.com/metrue/fx/context/mocks"
mockDeployer "github.com/metrue/fx/deploy/mocks"
mockDeployer "github.com/metrue/fx/infra/mocks"
"github.com/metrue/fx/types"
fxTypes "github.com/metrue/fx/types"
)
func TestUp(t *testing.T) {
@@ -18,17 +17,17 @@ func TestUp(t *testing.T) {
ctx := mockCtx.NewMockContexter(ctrl)
deployer := mockDeployer.NewMockDeployer(ctrl)
fn := fxTypes.Func{}
bindings := []types.PortBinding{}
name := "sample-name"
image := "sample-image"
ctx.EXPECT().Get("fn").Return(fn)
data := "sample-data"
ctx.EXPECT().Get("name").Return(name)
ctx.EXPECT().Get("image").Return(image)
ctx.EXPECT().Get("deployer").Return(deployer)
ctx.EXPECT().Get("bindings").Return(bindings)
ctx.EXPECT().Get("data").Return(data)
ctx.EXPECT().GetContext().Return(context.Background()).Times(2)
deployer.EXPECT().Deploy(gomock.Any(), fn, name, image, bindings).Return(nil)
deployer.EXPECT().Deploy(gomock.Any(), data, name, image, bindings).Return(nil)
deployer.EXPECT().GetStatus(gomock.Any(), name).Return(types.Service{
ID: "id-1",
Name: name,

View File

@@ -6,42 +6,46 @@ import (
dockerTypes "github.com/docker/docker/api/types"
containerruntimes "github.com/metrue/fx/container_runtimes"
"github.com/metrue/fx/deploy"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/types"
)
// Docker manage container
type Docker struct {
// Deployer manage container
type Deployer struct {
cli containerruntimes.ContainerRuntime
}
// CreateClient create a docker instance
func CreateClient(client containerruntimes.ContainerRuntime) (d *Docker, err error) {
return &Docker{cli: client}, nil
func CreateClient(client containerruntimes.ContainerRuntime) (d *Deployer, err error) {
return &Deployer{cli: client}, nil
}
// Deploy create a Docker container from given image, and bind the constants.FxContainerExposePort to given port
func (d *Docker) Deploy(ctx context.Context, fn types.Func, name string, image string, ports []types.PortBinding) (err error) {
spinner.Start("deploying")
func (d *Deployer) Deploy(ctx context.Context, fn string, name string, image string, ports []types.PortBinding) (err error) {
spinner.Start("deploying " + name)
defer func() {
spinner.Stop("deploying", err)
spinner.Stop("deploying "+name, err)
}()
return d.cli.StartContainer(ctx, name, image, ports)
}
// Update a container
func (d *Docker) Update(ctx context.Context, name string) error {
func (d *Deployer) Update(ctx context.Context, name string) error {
return nil
}
// Destroy stop and remove container
func (d *Docker) Destroy(ctx context.Context, name string) error {
func (d *Deployer) Destroy(ctx context.Context, name string) (err error) {
spinner.Start("destroying " + name)
defer func() {
spinner.Stop("destroying "+name, err)
}()
return d.cli.StopContainer(ctx, name)
}
// GetStatus get a service status
func (d *Docker) GetStatus(ctx context.Context, name string) (types.Service, error) {
func (d *Deployer) GetStatus(ctx context.Context, name string) (types.Service, error) {
var container dockerTypes.ContainerJSON
if err := d.cli.InspectContainer(ctx, name, &container); err != nil {
return types.Service{}, err
@@ -72,12 +76,26 @@ func (d *Docker) GetStatus(ctx context.Context, name string) (types.Service, err
return service, nil
}
// Ping check healty status of infra
func (d *Deployer) Ping(ctx context.Context) error {
if _, err := d.cli.Version(ctx); err != nil {
return err
}
return nil
}
// List services
func (d *Docker) List(ctx context.Context, name string) ([]types.Service, error) {
func (d *Deployer) List(ctx context.Context, name string) (svcs []types.Service, err error) {
const task = "listing"
spinner.Start(task)
defer func() {
spinner.Stop(task, err)
}()
// FIXME support remote host
return d.cli.ListContainer(ctx, name)
}
var (
_ deploy.Deployer = &Docker{}
_ infra.Deployer = &Deployer{}
)

View File

@@ -1,171 +1,13 @@
package docker
import (
"encoding/json"
"fmt"
"os"
"os/exec"
"strings"
import containerruntimes "github.com/metrue/fx/container_runtimes"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/infra"
sshOperator "github.com/metrue/go-ssh-client"
)
// Docker docker host
type Docker struct {
IP string
User string
// CreateProvisioner create a provisioner
func CreateProvisioner(ip string, user string) *Provisioner {
return NewProvisioner(ip, user)
}
// New new a docker object
func New(ip string, user string) *Docker {
return &Docker{
IP: ip,
User: user,
}
// CreateDeployer create a deployer
func CreateDeployer(client containerruntimes.ContainerRuntime) (*Deployer, error) {
return &Deployer{cli: client}, nil
}
// Provision provision a host, install docker and start dockerd
func (d *Docker) Provision() ([]byte, error) {
// TODO clean up, skip check localhost or not if in CICD env
if d.isLocalHost() {
if os.Getenv("CICD") == "" {
if !d.hasDocker() {
return nil, fmt.Errorf("please make sure docker installed and running")
}
if err := d.StartFxAgentLocally(); err != nil {
return nil, err
}
config, _ := json.Marshal(map[string]string{
"ip": d.IP,
"user": d.User,
})
return config, nil
}
}
if err := d.Install(); err != nil {
return nil, err
}
if err := d.StartDockerd(); err != nil {
return nil, err
}
if err := d.StartFxAgent(); err != nil {
return nil, err
}
config, _ := json.Marshal(map[string]string{
"ip": d.IP,
"user": d.User,
})
return config, nil
}
func (d *Docker) isLocalHost() bool {
return strings.ToLower(d.IP) == "localhost" || d.IP == "127.0.0.1"
}
func (d *Docker) hasDocker() bool {
cmd := exec.Command("docker", "version")
if err := cmd.Run(); err != nil {
return false
}
return true
}
// HealthCheck check healthy status of host
func (d *Docker) HealthCheck() (bool, error) {
// TODO
return true, nil
}
// Install docker on host
func (d *Docker) Install() error {
sudo := ""
if d.User != "root" {
sudo = "sudo"
}
installCmd := fmt.Sprintf("curl -fsSL https://download.docker.com/linux/static/stable/x86_64/docker-18.06.3-ce.tgz -o docker.tgz && tar zxvf docker.tgz && %s mv docker/* /usr/bin && rm -rf docker docker.tgz", sudo)
sshKeyFile, _ := infra.GetSSHKeyFile()
sshPort := infra.GetSSHPort()
ssh := sshOperator.New(d.IP).WithUser(d.User).WithKey(sshKeyFile).WithPort(sshPort)
if err := ssh.RunCommand(installCmd, sshOperator.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}); err != nil {
fmt.Println("install docker failed \n================")
fmt.Println(err)
fmt.Println("================")
return err
}
return nil
}
// StartDockerd start dockerd
func (d *Docker) StartDockerd() error {
sudo := ""
if d.User != "root" {
sudo = "sudo"
}
installCmd := fmt.Sprintf("%s dockerd >/dev/null 2>&1 & sleep 2", sudo)
sshKeyFile, _ := infra.GetSSHKeyFile()
sshPort := infra.GetSSHPort()
ssh := sshOperator.New(d.IP).WithUser(d.User).WithKey(sshKeyFile).WithPort(sshPort)
if err := ssh.RunCommand(installCmd, sshOperator.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}); err != nil {
fmt.Println("start dockerd failed \n================")
fmt.Println(err)
fmt.Println("================")
return err
}
return nil
}
// StartFxAgent start fx agent
func (d *Docker) StartFxAgent() error {
startCmd := fmt.Sprintf("sleep 3 && docker stop %s || true && docker run -d --name=%s --rm -v /var/run/docker.sock:/var/run/docker.sock -p 0.0.0.0:%s:1234 bobrik/socat TCP-LISTEN:1234,fork UNIX-CONNECT:/var/run/docker.sock", constants.AgentContainerName, constants.AgentContainerName, constants.AgentPort)
sshKeyFile, _ := infra.GetSSHKeyFile()
sshPort := infra.GetSSHPort()
ssh := sshOperator.New(d.IP).WithUser(d.User).WithKey(sshKeyFile).WithPort(sshPort)
if err := ssh.RunCommand(startCmd, sshOperator.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}); err != nil {
fmt.Println("start fx agent failed \n================")
fmt.Println(err)
fmt.Println("================")
return err
}
return nil
}
// StartFxAgentLocally start fx agent
func (d *Docker) StartFxAgentLocally() error {
startCmd := fmt.Sprintf("docker run -d --name=%s --rm -v /var/run/docker.sock:/var/run/docker.sock -p 0.0.0.0:%s:1234 bobrik/socat TCP-LISTEN:1234,fork UNIX-CONNECT:/var/run/docker.sock", constants.AgentContainerName, constants.AgentPort)
params := strings.Split(startCmd, " ")
fmt.Println(params)
var cmd *exec.Cmd
if len(params) > 1 {
// nolint: gosec
cmd = exec.Command(params[0], params[1:]...)
} else {
// nolint: gosec
cmd = exec.Command(params[0])
}
if out, err := cmd.CombinedOutput(); err != nil {
fmt.Println(string(out))
return err
}
return nil
}
var _ infra.Infra = &Docker{}

217
infra/docker/provisioner.go Normal file
View File

@@ -0,0 +1,217 @@
package docker
import (
"encoding/json"
"fmt"
"os"
"os/exec"
"strings"
"github.com/metrue/fx/constants"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/pkg/spinner"
sshOperator "github.com/metrue/go-ssh-client"
)
// Provisioner docker host
type Provisioner struct {
IP string
User string
}
// NewProvisioner new a docker object
func NewProvisioner(ip string, user string) *Provisioner {
return &Provisioner{
IP: ip,
User: user,
}
}
// Provision provision a host, install docker and start dockerd
func (d *Provisioner) Provision() (config []byte, err error) {
spinner.Start("provisioning")
defer func() {
spinner.Stop("provisioning", err)
}()
// TODO clean up, skip check localhost or not if in CICD env
if os.Getenv("CICD") != "" {
if err := d.Install(); err != nil {
return nil, err
}
if err := d.StartDockerd(); err != nil {
return nil, err
}
if err := d.StartFxAgent(); err != nil {
return nil, err
}
config, _ := json.Marshal(map[string]string{
"ip": d.IP,
"user": d.User,
})
return config, nil
}
if d.isLocalHost() {
if !d.hasDocker() {
return nil, fmt.Errorf("please make sure docker installed and running")
}
if err := d.StartFxAgentLocally(); err != nil {
return nil, err
}
config, _ := json.Marshal(map[string]string{
"ip": d.IP,
"user": d.User,
})
return config, nil
}
if err := d.Install(); err != nil {
return nil, err
}
if err := d.StartDockerd(); err != nil {
return nil, err
}
if err := d.StartFxAgent(); err != nil {
return nil, err
}
return json.Marshal(map[string]string{
"ip": d.IP,
"user": d.User,
})
}
func (d *Provisioner) isLocalHost() bool {
return strings.ToLower(d.IP) == "localhost" || d.IP == "127.0.0.1"
}
func (d *Provisioner) hasDocker() bool {
cmd := exec.Command("docker", "version")
if err := cmd.Run(); err != nil {
return false
}
return true
}
// HealthCheck check healthy status of host
func (d *Provisioner) HealthCheck() (bool, error) {
if d.isLocalHost() {
return d.IfFxAgentRunningLocally(), nil
}
return d.IfFxAgentRunning(), nil
}
// Install docker on host
func (d *Provisioner) Install() error {
sudo := ""
if d.User != "root" {
sudo = "sudo"
}
installCmd := fmt.Sprintf("curl -fsSL https://download.docker.com/linux/static/stable/x86_64/docker-18.06.3-ce.tgz -o docker.tgz && tar zxvf docker.tgz && %s mv docker/* /usr/bin && rm -rf docker docker.tgz", sudo)
sshKeyFile, _ := infra.GetSSHKeyFile()
sshPort := infra.GetSSHPort()
ssh := sshOperator.New(d.IP).WithUser(d.User).WithKey(sshKeyFile).WithPort(sshPort)
if err := ssh.RunCommand(installCmd, sshOperator.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}); err != nil {
fmt.Println("install docker failed \n================")
fmt.Println(err)
fmt.Println("================")
return err
}
return nil
}
// StartDockerd start dockerd
func (d *Provisioner) StartDockerd() error {
sudo := ""
if d.User != "root" {
sudo = "sudo"
}
installCmd := fmt.Sprintf("%s dockerd >/dev/null 2>&1 & sleep 2", sudo)
sshKeyFile, _ := infra.GetSSHKeyFile()
sshPort := infra.GetSSHPort()
ssh := sshOperator.New(d.IP).WithUser(d.User).WithKey(sshKeyFile).WithPort(sshPort)
if err := ssh.RunCommand(installCmd, sshOperator.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}); err != nil {
fmt.Println("start dockerd failed \n================")
fmt.Println(err)
fmt.Println("================")
return err
}
return nil
}
// StartFxAgent start fx agent
func (d *Provisioner) StartFxAgent() error {
startCmd := fmt.Sprintf("sleep 3 && docker stop %s || true && docker run -d --name=%s --rm -v /var/run/docker.sock:/var/run/docker.sock -p 0.0.0.0:%s:1234 bobrik/socat TCP-LISTEN:1234,fork UNIX-CONNECT:/var/run/docker.sock", constants.AgentContainerName, constants.AgentContainerName, constants.AgentPort)
sshKeyFile, _ := infra.GetSSHKeyFile()
sshPort := infra.GetSSHPort()
ssh := sshOperator.New(d.IP).WithUser(d.User).WithKey(sshKeyFile).WithPort(sshPort)
if err := ssh.RunCommand(startCmd, sshOperator.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}); err != nil {
fmt.Println("start fx agent failed \n================")
fmt.Println(err)
fmt.Println("================")
return err
}
return nil
}
// StartFxAgentLocally start fx agent
func (d *Provisioner) StartFxAgentLocally() error {
startCmd := fmt.Sprintf("docker run -d --name=%s --rm -v /var/run/docker.sock:/var/run/docker.sock -p 0.0.0.0:%s:1234 bobrik/socat TCP-LISTEN:1234,fork UNIX-CONNECT:/var/run/docker.sock", constants.AgentContainerName, constants.AgentPort)
params := strings.Split(startCmd, " ")
var cmd *exec.Cmd
if len(params) > 1 {
// nolint: gosec
cmd = exec.Command(params[0], params[1:]...)
} else {
// nolint: gosec
cmd = exec.Command(params[0])
}
if out, err := cmd.CombinedOutput(); err != nil {
fmt.Println(string(out))
return err
}
return nil
}
// IfFxAgentRunningLocally check if fx agent is running
func (d *Provisioner) IfFxAgentRunningLocally() bool {
cmd := exec.Command("docker", "inspect", "fx-agent")
if err := cmd.Run(); err != nil {
return false
}
return true
}
// IfFxAgentRunning check if fx agent is running
func (d *Provisioner) IfFxAgentRunning() bool {
inspectCmd := infra.Sudo("docker inspect fx-agent", d.User)
sshKeyFile, _ := infra.GetSSHKeyFile()
sshPort := infra.GetSSHPort()
ssh := sshOperator.New(d.IP).WithUser(d.User).WithKey(sshKeyFile).WithPort(sshPort)
if err := ssh.RunCommand(inspectCmd, sshOperator.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
}); err != nil {
return false
}
return true
}
var _ infra.Provisioner = &Provisioner{}

View File

@@ -5,12 +5,12 @@ import (
"testing"
)
func TestDocker(t *testing.T) {
func TestProvisioner(t *testing.T) {
if os.Getenv("DOCKER_HOST") == "" ||
os.Getenv("DOCKER_USER") == "" {
t.Skip("skip test since DOCKER_HOST and DOCKER_USER not ready")
}
d := New(os.Getenv("DOCKER_HOST"), os.Getenv("DOCKER_USER"))
d := NewProvisioner(os.Getenv("DOCKER_HOST"), os.Getenv("DOCKER_USER"))
if err := d.Install(); err != nil {
t.Fatal(err)
}

View File

@@ -1,7 +1,29 @@
package infra
// Infra infrastructure provision interface
type Infra interface {
import (
"context"
"github.com/metrue/fx/types"
)
// Provisioner provision interface
type Provisioner interface {
Provision() (config []byte, err error)
HealthCheck() (bool, error)
}
// Deployer deploy interface
type Deployer interface {
Deploy(ctx context.Context, fn string, name string, image string, bindings []types.PortBinding) error
Destroy(ctx context.Context, name string) error
Update(ctx context.Context, name string) error
GetStatus(ctx context.Context, name string) (types.Service, error)
List(ctx context.Context, name string) ([]types.Service, error)
Ping(ctx context.Context) error
}
// Infra infrastructure provision interface
type Infra interface {
Provisioner
Deployer
}

View File

@@ -5,6 +5,7 @@ import (
"os"
"testing"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/types"
)
@@ -31,16 +32,12 @@ func TestK8SDeployer(t *testing.T) {
t.Fatal(err)
}
fn := types.Func{
Language: "node",
Source: `
module.exports = (ctx) => {
ctx.body = 'hello world'
}
`,
data, err := packer.PackIntoK8SConfigMapFile("./fixture")
if err != nil {
t.Fatal(err)
}
ctx := context.Background()
if err := k8s.Deploy(ctx, fn, name, name, bindings); err != nil {
if err := k8s.Deploy(ctx, data, name, name, bindings); err != nil {
t.Fatal(err)
}

View File

@@ -2,10 +2,11 @@ package k8s
import (
"context"
"fmt"
"os"
"github.com/metrue/fx/deploy"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/types"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"
@@ -39,18 +40,13 @@ func Create(kubeconfig string) (*K8S, error) {
// Deploy a image to be a service
func (k *K8S) Deploy(
ctx context.Context,
fn types.Func,
fn string,
name string,
image string,
ports []types.PortBinding,
) error {
// put source code of function docker project into k8s config map
tree, err := packer.PackIntoK8SConfigMapFile(fn)
if err != nil {
return err
}
data := map[string]string{}
data[ConfigMap.AppMetaEnvName] = tree
data[ConfigMap.AppMetaEnvName] = fn
if _, err := k.CreateOrUpdateConfigMap(namespace, name, data); err != nil {
return err
}
@@ -62,14 +58,28 @@ func (k *K8S) Deploy(
const replicas = int32(3)
if _, err := k.GetDeployment(namespace, name); err != nil {
// TODO enable passing replica from fx CLI
if _, err := k.CreateDeploymentWithInitContainer(
namespace,
name,
ports,
replicas,
selector,
); err != nil {
return err
if os.Getenv("K3S") != "" {
// NOTE Doing docker build in initial container will fail when cluster is created by K3S
if _, err := k.CreateDeployment(
namespace,
name,
image,
ports,
replicas,
selector,
); err != nil {
return err
}
} else {
if _, err := k.CreateDeploymentWithInitContainer(
namespace,
name,
ports,
replicas,
selector,
); err != nil {
return err
}
}
} else {
if _, err := k.UpdateDeployment(
@@ -153,10 +163,28 @@ func (k *K8S) GetStatus(ctx context.Context, name string) (types.Service, error)
}
// List services
func (k *K8S) List(ctx context.Context, name string) ([]types.Service, error) {
func (k *K8S) List(ctx context.Context, name string) (svcs []types.Service, err error) {
const task = "listing"
spinner.Start(task)
defer func() {
spinner.Stop(task, err)
}()
return []types.Service{}, nil
}
// Ping health check of infra
func (k *K8S) Ping(ctx context.Context) error {
// Does not find any ping method for k8s
nodes, err := k.ListNodes()
if err != nil {
return err
}
if len(nodes.Items) <= 0 {
return fmt.Errorf("no available nodes")
}
return nil
}
var (
_ deploy.Deployer = &K8S{}
_ infra.Deployer = &K8S{}
)

11
infra/k8s/k8s.go Normal file
View File

@@ -0,0 +1,11 @@
package k8s
// CreateProvisioner create a provisioner
func CreateProvisioner(master MasterNode, agents []AgentNode) *Provisioner {
return New(master, agents)
}
// CreateDeployer create a deployer
func CreateDeployer(kubeconfig string) (*K8S, error) {
return Create(kubeconfig)
}

15
infra/k8s/node.go Normal file
View File

@@ -0,0 +1,15 @@
package k8s
import (
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// ListNodes list node
func (k *K8S) ListNodes() (*v1.NodeList, error) {
nodes, err := k.CoreV1().Nodes().List(metav1.ListOptions{})
if err != nil {
return nil, err
}
return nodes, nil
}

View File

@@ -1,4 +1,4 @@
package k3s
package k8s
import (
"bufio"
@@ -23,8 +23,8 @@ type AgentNode struct {
User string
}
// K3S k3s operator
type K3S struct {
// Provisioner k3s operator
type Provisioner struct {
master MasterNode
agents []AgentNode
}
@@ -34,15 +34,15 @@ type K3S struct {
const version = "v0.9.1"
// New new a operator
func New(master MasterNode, agents []AgentNode) *K3S {
return &K3S{
func New(master MasterNode, agents []AgentNode) *Provisioner {
return &Provisioner{
master: master,
agents: agents,
}
}
// Provision provision k3s cluster
func (k *K3S) Provision() ([]byte, error) {
func (k *Provisioner) Provision() ([]byte, error) {
if err := k.SetupMaster(); err != nil {
return nil, err
}
@@ -53,17 +53,17 @@ func (k *K3S) Provision() ([]byte, error) {
}
// HealthCheck check healthy status of host
func (k *K3S) HealthCheck() (bool, error) {
func (k *Provisioner) HealthCheck() (bool, error) {
// TODO
return true, nil
}
// SetupMaster setup master node
func (k *K3S) SetupMaster() error {
func (k *Provisioner) SetupMaster() error {
sshKeyFile, _ := infra.GetSSHKeyFile()
ssh := sshOperator.New(k.master.IP).WithUser(k.master.User).WithKey(sshKeyFile)
installCmd := fmt.Sprintf("curl -sLS https://get.k3s.io | INSTALL_K3S_EXEC='server --tls-san %s' INSTALL_K3S_VERSION='%s' sh -", k.master.IP, version)
if err := ssh.RunCommand(infra.Sudo(installCmd, k.master.IP), sshOperator.CommandOptions{
if err := ssh.RunCommand(infra.Sudo(installCmd, k.master.User), sshOperator.CommandOptions{
Stdout: os.Stdout,
Stdin: os.Stdin,
Stderr: os.Stderr,
@@ -75,12 +75,12 @@ func (k *K3S) SetupMaster() error {
return nil
}
func (k *K3S) getToken() (string, error) {
func (k *Provisioner) getToken() (string, error) {
sshKeyFile, _ := infra.GetSSHKeyFile()
ssh := sshOperator.New(k.master.IP).WithUser(k.master.User).WithKey(sshKeyFile)
script := "cat /var/lib/rancher/k3s/server/node-token"
var outPipe bytes.Buffer
if err := ssh.RunCommand(infra.Sudo(script, k.master.IP), sshOperator.CommandOptions{
if err := ssh.RunCommand(infra.Sudo(script, k.master.User), sshOperator.CommandOptions{
Stdout: bufio.NewWriter(&outPipe),
Stdin: os.Stdin,
Stderr: os.Stderr,
@@ -91,7 +91,7 @@ func (k *K3S) getToken() (string, error) {
}
// SetupAgent set agent node
func (k *K3S) SetupAgent() error {
func (k *Provisioner) SetupAgent() error {
sshKeyFile, _ := infra.GetSSHKeyFile()
tok, err := k.getToken()
if err != nil {
@@ -117,13 +117,13 @@ func (k *K3S) SetupAgent() error {
}
// GetKubeConfig get kubeconfig of k3s cluster
func (k *K3S) GetKubeConfig() ([]byte, error) {
func (k *Provisioner) GetKubeConfig() ([]byte, error) {
sshKeyFile, _ := infra.GetSSHKeyFile()
var config []byte
getConfigCmd := "cat /etc/rancher/k3s/k3s.yaml\n"
ssh := sshOperator.New(k.master.IP).WithUser(k.master.User).WithKey(sshKeyFile)
var outPipe bytes.Buffer
if err := ssh.RunCommand(infra.Sudo(getConfigCmd, k.master.IP), sshOperator.CommandOptions{
if err := ssh.RunCommand(infra.Sudo(getConfigCmd, k.master.User), sshOperator.CommandOptions{
Stdout: bufio.NewWriter(&outPipe),
Stdin: os.Stdin,
Stderr: os.Stderr,
@@ -138,6 +138,7 @@ func (k *K3S) GetKubeConfig() ([]byte, error) {
func rewriteKubeconfig(kubeconfig string, ip string, context string) []byte {
if context == "" {
// nolint
context = "default"
}
@@ -150,4 +151,4 @@ func rewriteKubeconfig(kubeconfig string, ip string, context string) []byte {
return []byte(kubeconfigReplacer.Replace(kubeconfig))
}
var _ infra.Infra = &K3S{}
var _ infra.Provisioner = &Provisioner{}

View File

@@ -1,4 +1,4 @@
package k3s
package k8s
import (
"fmt"
@@ -6,7 +6,7 @@ import (
"testing"
)
func TestK3S(t *testing.T) {
func TestProvisioner(t *testing.T) {
if os.Getenv("K3S_MASTER_IP") == "" ||
os.Getenv("K3S_MASTER_USER") == "" ||
os.Getenv("K3S_AGENT_IP") == "" ||

View File

@@ -5,10 +5,174 @@
package mock_infra
import (
context "context"
gomock "github.com/golang/mock/gomock"
types "github.com/metrue/fx/types"
reflect "reflect"
)
// MockProvisioner is a mock of Provisioner interface
type MockProvisioner struct {
ctrl *gomock.Controller
recorder *MockProvisionerMockRecorder
}
// MockProvisionerMockRecorder is the mock recorder for MockProvisioner
type MockProvisionerMockRecorder struct {
mock *MockProvisioner
}
// NewMockProvisioner creates a new mock instance
func NewMockProvisioner(ctrl *gomock.Controller) *MockProvisioner {
mock := &MockProvisioner{ctrl: ctrl}
mock.recorder = &MockProvisionerMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockProvisioner) EXPECT() *MockProvisionerMockRecorder {
return m.recorder
}
// Provision mocks base method
func (m *MockProvisioner) Provision() ([]byte, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Provision")
ret0, _ := ret[0].([]byte)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// Provision indicates an expected call of Provision
func (mr *MockProvisionerMockRecorder) Provision() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Provision", reflect.TypeOf((*MockProvisioner)(nil).Provision))
}
// HealthCheck mocks base method
func (m *MockProvisioner) HealthCheck() (bool, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "HealthCheck")
ret0, _ := ret[0].(bool)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// HealthCheck indicates an expected call of HealthCheck
func (mr *MockProvisionerMockRecorder) HealthCheck() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "HealthCheck", reflect.TypeOf((*MockProvisioner)(nil).HealthCheck))
}
// MockDeployer is a mock of Deployer interface
type MockDeployer struct {
ctrl *gomock.Controller
recorder *MockDeployerMockRecorder
}
// MockDeployerMockRecorder is the mock recorder for MockDeployer
type MockDeployerMockRecorder struct {
mock *MockDeployer
}
// NewMockDeployer creates a new mock instance
func NewMockDeployer(ctrl *gomock.Controller) *MockDeployer {
mock := &MockDeployer{ctrl: ctrl}
mock.recorder = &MockDeployerMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockDeployer) EXPECT() *MockDeployerMockRecorder {
return m.recorder
}
// Deploy mocks base method
func (m *MockDeployer) Deploy(ctx context.Context, fn, name, image string, bindings []types.PortBinding) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Deploy", ctx, fn, name, image, bindings)
ret0, _ := ret[0].(error)
return ret0
}
// Deploy indicates an expected call of Deploy
func (mr *MockDeployerMockRecorder) Deploy(ctx, fn, name, image, bindings interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Deploy", reflect.TypeOf((*MockDeployer)(nil).Deploy), ctx, fn, name, image, bindings)
}
// Destroy mocks base method
func (m *MockDeployer) Destroy(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Destroy", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// Destroy indicates an expected call of Destroy
func (mr *MockDeployerMockRecorder) Destroy(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Destroy", reflect.TypeOf((*MockDeployer)(nil).Destroy), ctx, name)
}
// Update mocks base method
func (m *MockDeployer) Update(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Update", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// Update indicates an expected call of Update
func (mr *MockDeployerMockRecorder) Update(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Update", reflect.TypeOf((*MockDeployer)(nil).Update), ctx, name)
}
// GetStatus mocks base method
func (m *MockDeployer) GetStatus(ctx context.Context, name string) (types.Service, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetStatus", ctx, name)
ret0, _ := ret[0].(types.Service)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetStatus indicates an expected call of GetStatus
func (mr *MockDeployerMockRecorder) GetStatus(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetStatus", reflect.TypeOf((*MockDeployer)(nil).GetStatus), ctx, name)
}
// List mocks base method
func (m *MockDeployer) List(ctx context.Context, name string) ([]types.Service, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "List", ctx, name)
ret0, _ := ret[0].([]types.Service)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// List indicates an expected call of List
func (mr *MockDeployerMockRecorder) List(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "List", reflect.TypeOf((*MockDeployer)(nil).List), ctx, name)
}
// Ping mocks base method
func (m *MockDeployer) Ping(ctx context.Context) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Ping", ctx)
ret0, _ := ret[0].(error)
return ret0
}
// Ping indicates an expected call of Ping
func (mr *MockDeployerMockRecorder) Ping(ctx interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Ping", reflect.TypeOf((*MockDeployer)(nil).Ping), ctx)
}
// MockInfra is a mock of Infra interface
type MockInfra struct {
ctrl *gomock.Controller
@@ -61,3 +225,89 @@ func (mr *MockInfraMockRecorder) HealthCheck() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "HealthCheck", reflect.TypeOf((*MockInfra)(nil).HealthCheck))
}
// Deploy mocks base method
func (m *MockInfra) Deploy(ctx context.Context, fn, name, image string, bindings []types.PortBinding) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Deploy", ctx, fn, name, image, bindings)
ret0, _ := ret[0].(error)
return ret0
}
// Deploy indicates an expected call of Deploy
func (mr *MockInfraMockRecorder) Deploy(ctx, fn, name, image, bindings interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Deploy", reflect.TypeOf((*MockInfra)(nil).Deploy), ctx, fn, name, image, bindings)
}
// Destroy mocks base method
func (m *MockInfra) Destroy(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Destroy", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// Destroy indicates an expected call of Destroy
func (mr *MockInfraMockRecorder) Destroy(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Destroy", reflect.TypeOf((*MockInfra)(nil).Destroy), ctx, name)
}
// Update mocks base method
func (m *MockInfra) Update(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Update", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// Update indicates an expected call of Update
func (mr *MockInfraMockRecorder) Update(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Update", reflect.TypeOf((*MockInfra)(nil).Update), ctx, name)
}
// GetStatus mocks base method
func (m *MockInfra) GetStatus(ctx context.Context, name string) (types.Service, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetStatus", ctx, name)
ret0, _ := ret[0].(types.Service)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetStatus indicates an expected call of GetStatus
func (mr *MockInfraMockRecorder) GetStatus(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetStatus", reflect.TypeOf((*MockInfra)(nil).GetStatus), ctx, name)
}
// List mocks base method
func (m *MockInfra) List(ctx context.Context, name string) ([]types.Service, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "List", ctx, name)
ret0, _ := ret[0].([]types.Service)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// List indicates an expected call of List
func (mr *MockInfraMockRecorder) List(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "List", reflect.TypeOf((*MockInfra)(nil).List), ctx, name)
}
// Ping mocks base method
func (m *MockInfra) Ping(ctx context.Context) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Ping", ctx)
ret0, _ := ret[0].(error)
return ret0
}
// Ping indicates an expected call of Ping
func (mr *MockInfraMockRecorder) Ping(ctx interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Ping", reflect.TypeOf((*MockInfra)(nil).Ping), ctx)
}

View File

@@ -5,11 +5,11 @@ import (
"os"
"time"
"github.com/metrue/fx/config"
containerruntimes "github.com/metrue/fx/container_runtimes"
"github.com/metrue/fx/context"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/types"
)
// Build image
@@ -20,36 +20,44 @@ func Build(ctx context.Contexter) (err error) {
spinner.Stop(task, err)
}()
cli := ctx.GetCliContext()
name := cli.String("name")
fn := ctx.Get("fn").(types.Func)
docker := ctx.Get("docker").(containerruntimes.ContainerRuntime)
workdir := fmt.Sprintf("/tmp/fx-%d", time.Now().Unix())
defer os.RemoveAll(workdir)
if err := packer.PackIntoDir(fn, workdir); err != nil {
return err
}
if err := docker.BuildImage(ctx.GetContext(), workdir, name); err != nil {
if err := packer.Pack(workdir, ctx.Get("sources").([]string)...); err != nil {
return err
}
nameWithTag := name + ":latest"
if err := docker.TagImage(ctx.GetContext(), name, nameWithTag); err != nil {
return err
}
ctx.Set("image", nameWithTag)
cloudType := ctx.Get("cloud_type").(string)
name := ctx.Get("name").(string)
if cloudType == config.CloudTypeK8S && os.Getenv("K3S") == "" {
data, err := packer.PackIntoK8SConfigMapFile(workdir)
if err != nil {
return err
}
ctx.Set("data", data)
} else {
docker := ctx.Get("docker").(containerruntimes.ContainerRuntime)
if err := docker.BuildImage(ctx.GetContext(), workdir, name); err != nil {
return err
}
if os.Getenv("K3S") != "" {
name := cli.String("name")
username := os.Getenv("DOCKER_USERNAME")
password := os.Getenv("DOCKER_PASSWORD")
if username != "" && password != "" {
if _, err := docker.PushImage(ctx.GetContext(), name); err != nil {
return err
nameWithTag := name + ":latest"
if err := docker.TagImage(ctx.GetContext(), name, nameWithTag); err != nil {
return err
}
ctx.Set("image", nameWithTag)
if os.Getenv("K3S") != "" {
username := os.Getenv("DOCKER_USERNAME")
password := os.Getenv("DOCKER_PASSWORD")
if username != "" && password != "" {
if _, err := docker.PushImage(ctx.GetContext(), name); err != nil {
return err
}
ctx.Set("image", username+"/"+name)
}
ctx.Set("image", username+"/"+name)
}
}
return nil
}

View File

@@ -1,40 +1,49 @@
package middlewares
import (
"io/ioutil"
"os"
"github.com/metrue/fx/context"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
"github.com/pkg/errors"
)
// Parse parse input
func Parse(ctx context.Contexter) (err error) {
const task = "parsing"
spinner.Start(task)
defer func() {
spinner.Stop(task, err)
}()
func Parse(action string) func(ctx context.Contexter) (err error) {
return func(ctx context.Contexter) error {
cli := ctx.GetCliContext()
switch action {
case "up":
sources := []string{}
for _, s := range cli.Args() {
sources = append(sources, s)
}
if len(sources) == 0 {
pwd, err := os.Getwd()
if err != nil {
return err
}
sources = append(sources, pwd)
}
ctx.Set("sources", sources)
name := cli.String("name")
ctx.Set("name", name)
port := cli.Int("port")
ctx.Set("port", port)
case "down":
services := cli.Args()
if len(services) == 0 {
return errors.New("service name required")
}
svc := []string{}
for _, service := range services {
svc = append(svc, service)
}
ctx.Set("services", svc)
case "list":
name := cli.Args().First()
ctx.Set("filter", name)
}
cli := ctx.GetCliContext()
funcFile := cli.Args().First()
lang := utils.GetLangFromFileName(funcFile)
body, err := ioutil.ReadFile(funcFile)
if err != nil {
return errors.Wrap(err, "read source failed")
return nil
}
fn := types.Func{
Language: lang,
Source: string(body),
}
ctx.Set("fn", fn)
name := cli.String("name")
ctx.Set("name", name)
port := cli.Int("port")
ctx.Set("port", port)
return nil
}

66
middlewares/provision.go Normal file
View File

@@ -0,0 +1,66 @@
package middlewares
import (
"fmt"
"os"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
dockerHTTP "github.com/metrue/fx/container_runtimes/docker/http"
"github.com/metrue/fx/context"
"github.com/metrue/fx/infra"
dockerInfra "github.com/metrue/fx/infra/docker"
k8sInfra "github.com/metrue/fx/infra/k8s"
"github.com/pkg/errors"
)
// Provision make sure infrastructure is healthy
func Provision(ctx context.Contexter) (err error) {
fxConfig := ctx.Get("config").(*config.Config)
cloud := fxConfig.Clouds[fxConfig.CurrentCloud]
var deployer infra.Deployer
if os.Getenv("KUBECONFIG") != "" {
deployer, err = k8sInfra.CreateDeployer(os.Getenv("KUBECONFIG"))
if err != nil {
return err
}
ctx.Set("cloud_type", config.CloudTypeK8S)
} else if cloud["type"] == config.CloudTypeDocker {
provisioner := dockerInfra.CreateProvisioner(cloud["host"], cloud["user"])
ok, err := provisioner.HealthCheck()
if err != nil {
return err
}
if !ok {
if _, err := provisioner.Provision(); err != nil {
return err
}
}
docker, err := dockerHTTP.Create(cloud["host"], constants.AgentPort)
if err != nil {
return errors.Wrapf(err, "please make sure docker is installed and running on your host")
}
// TODO should clean up, but it needed in middlewares.Build
ctx.Set("docker", docker)
deployer, err = dockerInfra.CreateDeployer(docker)
if err != nil {
return err
}
ctx.Set("cloud_type", config.CloudTypeDocker)
} else if cloud["type"] == config.CloudTypeK8S {
deployer, err = k8sInfra.CreateDeployer(cloud["kubeconfig"])
if err != nil {
return err
}
ctx.Set("cloud_type", config.CloudTypeK8S)
} else {
return fmt.Errorf("unsupport cloud type %s, please make sure you config is correct", cloud["type"])
}
ctx.Set("deployer", deployer)
return nil
}

View File

@@ -1,60 +0,0 @@
package middlewares
import (
"fmt"
"os"
"github.com/metrue/fx/config"
"github.com/metrue/fx/constants"
dockerHTTP "github.com/metrue/fx/container_runtimes/docker/http"
"github.com/metrue/fx/context"
"github.com/metrue/fx/deploy"
dockerDeployer "github.com/metrue/fx/deploy/docker"
k3sDeployer "github.com/metrue/fx/deploy/k3s"
k8sDeployer "github.com/metrue/fx/deploy/k8s"
"github.com/metrue/fx/pkg/spinner"
)
// Setup create k8s or docker cli
func Setup(ctx context.Contexter) (err error) {
const task = "setup"
spinner.Start(task)
defer func() {
spinner.Stop(task, err)
}()
fxConfig := ctx.Get("config").(*config.Config)
cloud := fxConfig.Clouds[fxConfig.CurrentCloud]
var deployer deploy.Deployer
if cloud["type"] == config.CloudTypeDocker {
docker, err := dockerHTTP.Create(cloud["host"], constants.AgentPort)
if err != nil {
return err
}
// TODO should clean up, but it needed in middlewares.Build
ctx.Set("docker", docker)
deployer, err = dockerDeployer.CreateClient(docker)
if err != nil {
return err
}
} else if cloud["type"] == config.CloudTypeK8S {
if os.Getenv("K3S") != "" {
deployer, err = k3sDeployer.Create(cloud["kubeconfig"])
if err != nil {
return err
}
} else if os.Getenv("KUBECONFIG") != "" {
deployer, err = k8sDeployer.Create(cloud["kubeconfig"])
if err != nil {
return err
}
}
} else {
return fmt.Errorf("unsupport cloud type %s, please make sure you config is correct", cloud["type"])
}
ctx.Set("deployer", deployer)
return nil
}

View File

@@ -8,7 +8,7 @@ import (
"github.com/metrue/fx/types"
)
func TestPacker(t *testing.T) {
func TestDockerPacker(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()

View File

@@ -0,0 +1,5 @@
FROM metrue/fx-node-base
COPY . .
EXPOSE 3000
CMD ["node", "app.js"]

9
packer/fixture/p1/app.js Normal file
View File

@@ -0,0 +1,9 @@
const Koa = require('koa');
const bodyParser = require('koa-bodyparser');
const fx = require('./fx');
const app = new Koa();
app.use(bodyParser());
app.use(fx);
app.listen(3000);

3
packer/fixture/p1/fx.js Normal file
View File

@@ -0,0 +1,3 @@
module.exports = (ctx) => {
ctx.body = 'hello world'
}

3
packer/fixture/p2/fx.js Normal file
View File

@@ -0,0 +1,3 @@
module.exports = (ctx) => {
ctx.body = 'hello world'
}

6
packer/fixture/p3/fx.js Normal file
View File

@@ -0,0 +1,6 @@
const say = require('./helper')
module.exports = (ctx) => {
say("hi")
ctx.body = 'hello world'
}

View File

@@ -0,0 +1,5 @@
const say = (msg) => {
console.log(msg)
}
module.exports = say

View File

@@ -1,9 +1,12 @@
package packer
import (
"fmt"
"io/ioutil"
"os"
"path/filepath"
"strings"
"time"
"encoding/base64"
"encoding/json"
@@ -11,15 +14,106 @@ import (
"github.com/gobuffalo/packr"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
"github.com/otiai10/copy"
"github.com/pkg/errors"
)
// Packer interface
type Packer interface {
Pack(serviceName string, fn types.Func) (types.Project, error)
// Pack pack a file or directory into a Docker project
func Pack(output string, input ...string) error {
if len(input) == 0 {
return fmt.Errorf("source file or directory required")
}
if len(input) == 1 {
file := input[0]
stat, err := os.Stat(file)
if err != nil {
return err
}
if !stat.IsDir() {
lang := utils.GetLangFromFileName(file)
body, err := ioutil.ReadFile(file)
if err != nil {
return errors.Wrap(err, "read source failed")
}
fn := types.Func{
Language: lang,
Source: string(body),
}
if err := PackIntoDir(fn, output); err != nil {
return err
}
return nil
}
}
workdir := fmt.Sprintf("./fx-%d", time.Now().Unix())
defer os.RemoveAll(workdir)
for _, f := range input {
if err := copy.Copy(f, filepath.Join(workdir, f)); err != nil {
return err
}
}
if dockerfile, has := hasDockerfileInDir(workdir); has {
return copy.Copy(filepath.Dir(dockerfile), output)
}
if f, has := hasFxHandleFileInDir(workdir); has {
lang := utils.GetLangFromFileName(f)
body, err := ioutil.ReadFile(f)
if err != nil {
return errors.Wrap(err, "read source failed")
}
fn := types.Func{
Language: lang,
Source: string(body),
}
if err := PackIntoDir(fn, output); err != nil {
return err
}
return copy.Copy(filepath.Dir(f), output)
}
return fmt.Errorf("input directories or files has no Dockerfile or file with fx as name, e.g. fx.js")
}
func hasDockerfileInDir(dir string) (string, bool) {
var dockerfile string
if err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
// nolint
if !info.IsDir() && info.Name() == "Dockerfile" {
dockerfile = path
}
return nil
}); err != nil {
return "", false
}
if dockerfile == "" {
return "", false
}
return dockerfile, true
}
func hasFxHandleFileInDir(dir string) (string, bool) {
var handleFile string
if err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if !info.IsDir() && isHandler(info.Name()) {
handleFile = path
}
return nil
}); err != nil {
return "", false
}
if handleFile == "" {
return "", false
}
return handleFile, true
}
// Pack a function to be a docker project which is web service, handle the imcome request with given function
func Pack(svcName string, fn types.Func) (types.Project, error) {
func pack(svcName string, fn types.Func) (types.Project, error) {
box := packr.NewBox("./images")
pkr := NewDockerPacker(box)
return pkr.Pack(svcName, fn)
@@ -27,7 +121,7 @@ func Pack(svcName string, fn types.Func) (types.Project, error) {
// PackIntoDir pack service code into directory
func PackIntoDir(fn types.Func, outputDir string) error {
project, err := Pack("", fn)
project, err := pack("", fn)
if err != nil {
return err
}
@@ -44,16 +138,24 @@ func PackIntoDir(fn types.Func, outputDir string) error {
}
// PackIntoK8SConfigMapFile pack function a K8S config map file
func PackIntoK8SConfigMapFile(fn types.Func) (string, error) {
project, err := Pack("", fn)
if err != nil {
return "", err
}
func PackIntoK8SConfigMapFile(dir string) (string, error) {
tree := map[string]string{}
for _, file := range project.Files {
tree[file.Path] = file.Body
if err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if !info.IsDir() {
relpath := strings.Replace(path, dir, "", 1)
body, err := ioutil.ReadFile(path)
if err != nil {
return err
}
tree[relpath] = string(body)
}
return nil
}); err != nil {
return "", nil
}
data, err := json.Marshal(tree)
if err != nil {
return "", err

View File

@@ -1,79 +1,122 @@
package packer
import (
"os"
"testing"
"github.com/metrue/fx/types"
)
func TestPack(t *testing.T) {
mockSource := `
func TestPacker(t *testing.T) {
t.Run("Pack directory with Dockerfile in it", func(t *testing.T) {
input := "./fixture/p1"
output := "output-1"
defer func() {
os.RemoveAll(output)
}()
if err := Pack(output, input); err != nil {
t.Fatal(err)
}
})
t.Run("Pack directory only fx.js in it", func(t *testing.T) {
input := "./fixture/p2"
output := "output-2"
defer func() {
os.RemoveAll(output)
}()
if err := Pack(output, input); err != nil {
t.Fatal(err)
}
})
t.Run("Pack directory with fx.js and helper in it", func(t *testing.T) {
input := "./fixture/p3"
output := "output-3"
defer func() {
os.RemoveAll(output)
}()
if err := Pack(output, input); err != nil {
t.Fatal(err)
}
})
t.Run("Pack files list with fx.js in it", func(t *testing.T) {
handleFile := "./fixture/p3/fx.js"
helperFile := "./fixture/p3/helper.js"
output := "output-4"
defer func() {
os.RemoveAll(output)
}()
if err := Pack(output, handleFile, helperFile); err != nil {
t.Fatal(err)
}
})
t.Run("Pack files list without fx.js in it", func(t *testing.T) {
f1 := "./fixture/p3/helper.js"
f2 := "./fixture/p3/helper.js"
output := "output-5"
defer func() {
os.RemoveAll(output)
}()
if err := Pack(output, f1, f2); err == nil {
t.Fatalf("should report error when there is not Dockerfile or fx.[ext] in it")
}
})
t.Run("pack", func(t *testing.T) {
mockSource := `
module.exports = ({a, b}) => {
return a + b
}
`
fn := types.Func{
Language: "node",
Source: mockSource,
}
fn := types.Func{
Language: "node",
Source: mockSource,
}
serviceName := "service-mock"
project, err := Pack(serviceName, fn)
if err != nil {
t.Fatal(err)
}
serviceName := "service-mock"
project, err := pack(serviceName, fn)
if err != nil {
t.Fatal(err)
}
if project.Name != serviceName {
t.Fatalf("should get %s but got %s", serviceName, project.Name)
}
if project.Name != serviceName {
t.Fatalf("should get %s but got %s", serviceName, project.Name)
}
if project.Language != "node" {
t.Fatal("incorrect Language")
}
if project.Language != "node" {
t.Fatal("incorrect Language")
}
if len(project.Files) != 3 {
t.Fatal("node project should have 3 files")
}
if len(project.Files) != 3 {
t.Fatal("node project should have 3 files")
}
for _, file := range project.Files {
if file.Path == "fx.js" {
if file.IsHandler == false {
t.Fatal("fx.js should be handler")
}
if file.Body != mockSource {
t.Fatalf("should get %s but got %v", mockSource, file.Body)
}
} else if file.Path == "Dockerfile" {
if file.IsHandler == true {
t.Fatalf("should get %v but got %v", false, file.IsHandler)
}
} else {
if file.IsHandler == true {
t.Fatalf("should get %v but %v", false, file.IsHandler)
for _, file := range project.Files {
if file.Path == "fx.js" {
if file.IsHandler == false {
t.Fatal("fx.js should be handler")
}
if file.Body != mockSource {
t.Fatalf("should get %s but got %v", mockSource, file.Body)
}
} else if file.Path == "Dockerfile" {
if file.IsHandler == true {
t.Fatalf("should get %v but got %v", false, file.IsHandler)
}
} else {
if file.IsHandler == true {
t.Fatalf("should get %v but %v", false, file.IsHandler)
}
}
}
}
})
}
func TestTreeAndUnTree(t *testing.T) {
mockSource := `
package fx;
import org.json.JSONObject;
public class Fx {
public int handle(JSONObject input) {
String a = input.get("a").toString();
String b = input.get("b").toString();
return Integer.parseInt(a) + Integer.parseInt(b);
}
}
`
fn := types.Func{
Language: "java",
Source: mockSource,
}
_, err := PackIntoK8SConfigMapFile(fn)
_, err := PackIntoK8SConfigMapFile("./fixture/p1")
if err != nil {
t.Fatal(err)
}