Compare commits

..

5 Commits

Author SHA1 Message Date
Minghe
74c0423f0d add docs for rust example (#398) 2019-12-06 14:58:30 +08:00
Minghe
06f87c4d8e fix deploy to aks issue, and update docs (#396)
* fix deploy to aks issue, and update docs
* update docs
2019-12-06 11:54:27 +08:00
Minghe
35262de828 release v0.8.7 (#394) 2019-12-06 10:04:40 +08:00
dependabot-preview[bot]
34a495984c Bump github.com/olekukonko/tablewriter from 0.0.3 to 0.0.4 (#393)
Bumps [github.com/olekukonko/tablewriter](https://github.com/olekukonko/tablewriter) from 0.0.3 to 0.0.4.
- [Release notes](https://github.com/olekukonko/tablewriter/releases)
- [Commits](https://github.com/olekukonko/tablewriter/compare/v0.0.3...v0.0.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-06 09:45:27 +08:00
Minghe
d7130c4e28 support a project with multiple files (#392)
* support a project with multiple files
* fix lint issue
* fix test
2019-12-05 17:57:39 +08:00
29 changed files with 431 additions and 219 deletions

View File

@@ -32,7 +32,7 @@ cli-test-ci:
./scripts/test_cli.sh 'js'
cli-test:
./scripts/test_cli.sh 'js rb py go php java d'
./scripts/test_cli.sh 'js rb py go php java d rs'
http-test:
./scripts/http_test.sh

View File

@@ -1,5 +1,6 @@
fx
------
Poor man's function as a service.
<br/>
![CI](https://github.com/metrue/fx/workflows/ci/badge.svg)
@@ -13,6 +14,7 @@ Poor man's function as a service.
- [Introduction](#introduction)
- [Installation](#installation)
- [Usage](#usage)
- [Manage Infrastructure](#manage-infrastructure)
- [Contribute](#contribute)
@@ -79,7 +81,7 @@ USAGE:
fx [global options] command [command options] [arguments...]
VERSION:
0.8.4
0.8.7
COMMANDS:
infra manage infrastructure
@@ -96,44 +98,36 @@ GLOBAL OPTIONS:
--version, -v print the version
```
1. Write a function
### Deploy your function to Docker
You can check out [examples](https://github.com/metrue/fx/tree/master/examples/functions) for reference. Let's write a function as an example, it calculates the sum of two numbers then returns:
```js
module.exports = (ctx) => {
ctx.body = 'hello world'
}
```
Then save it to a file `func.js`.
$ fx up --name hello-fx ./examples/functions/JavaScript/func.js
2. Deploy your function as a service
Give your service a port with `--port`, and name with `--name`, heath checking with `--healthcheck` if you want.
```shell
$ fx up -name fx_service_name -p 10001 --healthcheck func.js
2019/08/10 13:26:37 info Pack Service: ✓
2019/08/10 13:26:39 info Build Service: ✓
2019/08/10 13:26:39 info Run Service: ✓
2019/08/10 13:26:39 info Service (fx_service_name) is running on: 0.0.0.0:10001
2019/08/10 13:26:39 info up function fx_service_name(func.js) to machine localhost: ✓
+------------------------------------------------------------------+-----------+---------------+
| ID | NAME | ENDPOINT |
+------------------------------------------------------------------+-----------+---------------+
| 5b24d36608ee392c937a61a530805f74551ddec304aea3aca2ffa0fabcf98cf3 | /hello-fx | 0.0.0.0:58328 |
+------------------------------------------------------------------+-----------+---------------+
```
if you want see what the source code of your service looks like, you can export it into a dirctory,
### Deploy your function to Kubernetes
```shell
$ fx image export -o <path of dir> func.js
2019/09/25 19:31:19 info exported to <path of dir>: ✓
```
$ KUBECONFIG=~/.kube/config ./build/fx up examples/functions/JavaScript/func.js --name hello-fx
+-------------------------------+------+----------------+
| ID | NAME | ENDPOINT |
+----+--------------------------+-----------------------+
| 5b24d36608ee392c937a | hello-fx | 10.0.242.75:80 |
+------------------------+-------------+----------------+
```
3. Test your service
### Test your service
then you can test your service:
```shell
$ curl -v 0.0.0.0:10001
$ curl -v 0.0.0.0:58328
GET / HTTP/1.1
@@ -155,39 +149,32 @@ hello world
```
## Docker
## Manage Infrastructure
**fx** is originally designed to turn a function into a runnable Docker container in a easiest way, on a host with Docker running, you can just deploy your function with `fx up` command,
**fx** is originally designed to turn a function into a runnable Docker container in a easiest way, on a host with Docker running, you can just deploy your function with `fx up` command, and now **fx** supports deploy function to be a service onto Kubernetes cluster infrasture, and we encourage you to do that other than on bare Docker environment, there are lots of advantage to run your function on Kubernetes like self-healing, load balancing, easy horizontal scaling, etc. It's pretty simple to deploy your function onto Kubernetes with **fx**, you just set KUBECONFIG in your enviroment.
By default. **fx** use localhost as target infrastructure to run your service, and you can also setup your remote virtual machines as **fx**'s infrastructure and deploy your functions onto it.
### `fx infra create`
You can create types (docker and k8s) of infrastructures for **fx** to deploy functions
```shell
fx up --name hello-svc --port 7777 hello.js # onto localhost
DOCKER_REMOTE_HOST_ADDR=xx.xx.xx.xx DOCKER_REMOTE_HOST_USER=xxxx DOCKER_REMOTE_HOST_PASSWORD=xxxx fx up --name hello-svc --port 7777 hello.js # onto remote host
$ fx infra create --name infra_us --type docker --host <user>@<ip> ## create docker type infrasture on <ip>
$ fx infra create --name infra_bj --type k8s --master <user>@<ip> --agents '<user1>@<ip1>,<user2>@<ip2>' ## create k8s type infrasture use <ip> as master node, and <ip1> and <ip2> as agents nodes
```
## Kubernetes
### `fx infra use`
**fx** supports deploy function to be a service onto Kubernetes cluster infrasture, and we encourage you to do that other than on bare Docker environment, there are lots of advantage to run your function on Kubernetes like self-healing, load balancing, easy horizontal scaling, etc. It's pretty simple to deploy your function onto Kubernetes with **fx**, you just set KUBECONFIG in your enviroment.
To use a infrastructure, you can use `fx infra use` command to activate it.
```shell
KUBECONFIG=<Your KUBECONFIG> fx deploy -n fx-service-abc_js -p 12349 examples/functions/JavaScript/func.js # function will be deploy to your Kubernetes cluster and expose a IP address of your loadbalencer
fx infra use <infrastructure name>
```
or
and you can list your infrastructure with `fx infra list`
```shell
$ export KUBECONFIG=<Your KUBECONFIG>
$ fx deploy -n fx-service-abc_js -p 12349 examples/functions/JavaScript/func.js # function will be deploy to your Kubernetes cluster and expose a IP address of your loadbalencer
```
* Local Kubernetes Cluster
Docker for Mac and Docker for Windows already support Kubernetes with single node cluster, we can use it directly, and the default `KUBECONFIG` is `~/.kube/config`.
```shell
$ export KUBECONFIG=~/.kube/config # then fx will take the config to deloy function
```
if you have multiple Kubernetes clusters configured, you have to set context correctly. FYI [configure-access-multiple-clusters](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/)
## Use Public Cloud Kubernetes Service as infrastructure to run your functions
* Azure Kubernetes Service (AKS)

47
examples/functions/Rust/README.md vendored Normal file
View File

@@ -0,0 +1,47 @@
# Make a Rust function a service with fx
Write a function like,
```rust
pub mod fns {
#[derive(Serialize)]
pub struct Response {
pub result: i32,
}
#[derive(Deserialize)]
pub struct Request {
pub a: i32,
pub b: i32,
}
pub fn func(req: Request) -> Response {
Response {
result: req.a + req.b,
}
}
}
```
then deploy it with `fx up` command,
```shell
$ fx up -p 8080 func.rs
```
test it using `curl`
```shell
$ curl -X 'POST' --header 'Content-Type: application/json' --data '{"a":1,"b":1}' '0.0.0.0:3000'
HTTP/1.1 200 OK
Content-Length: 12
Content-Type: application/json
Date: Fri, 06 Dec 2019 06:45:14 GMT
Server: Rocket
{
"result": 2
}
```

2
fx.go
View File

@@ -16,7 +16,7 @@ import (
"github.com/urfave/cli"
)
const version = "0.8.6"
const version = "0.8.71"
func init() {
go checkForUpdate()

3
go.mod
View File

@@ -27,9 +27,10 @@ require (
github.com/mitchellh/go-homedir v1.1.0
github.com/morikuni/aec v1.0.0 // indirect
github.com/nwaples/rardecode v1.0.0 // indirect
github.com/olekukonko/tablewriter v0.0.3
github.com/olekukonko/tablewriter v0.0.4
github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
github.com/opencontainers/image-spec v1.0.1 // indirect
github.com/otiai10/copy v1.0.2
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
github.com/pierrec/lz4 v0.0.0-20190222153722-062282ea0dcf // indirect
github.com/pkg/errors v0.8.1

8
go.sum
View File

@@ -163,6 +163,8 @@ github.com/mattn/go-isatty v0.0.8 h1:HLtExJ+uU2HOZ+wI0Tt5DtUDrx8yhUqDcp7fYERX4CE
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-runewidth v0.0.6 h1:V2iyH+aX9C5fsYCpK60U8BYIvmhqxuOL3JZcqc1NB7k=
github.com/mattn/go-runewidth v0.0.6/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/mattn/go-runewidth v0.0.7 h1:Ei8KR0497xHyKJPAv59M1dkC+rOZCMBJ+t3fZ+twI54=
github.com/mattn/go-runewidth v0.0.7/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/metrue/go-ssh-client v0.0.0-20191125030649-4ac058ee958b h1:JGD0sJ44XzhsT1voOg00zji4ubuMNcVNK3m7d9GI88k=
github.com/metrue/go-ssh-client v0.0.0-20191125030649-4ac058ee958b/go.mod h1:ERHOEBrDy6+8vfoJjjmhdmBpOzdvvP7bLtwYTTK6LOs=
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
@@ -187,6 +189,8 @@ github.com/nwaples/rardecode v1.0.0 h1:r7vGuS5akxOnR4JQSkko62RJ1ReCMXxQRPtxsiFMB
github.com/nwaples/rardecode v1.0.0/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0=
github.com/olekukonko/tablewriter v0.0.3 h1:i0LBnzgiChAWHJYTQAZJDOgf8MNxAVYZJ2m63SIDimI=
github.com/olekukonko/tablewriter v0.0.3/go.mod h1:YZeBtGzYYEsCHp2LST/u/0NDwGkRoBtmn1cIWCJiS6M=
github.com/olekukonko/tablewriter v0.0.4 h1:vHD/YYe1Wolo78koG299f7V/VAS08c6IpCLn+Ejf/w8=
github.com/olekukonko/tablewriter v0.0.4/go.mod h1:zq6QwlOf5SlnkVbMSr5EoBv3636FWnp+qbPhuoO21uA=
github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
@@ -196,6 +200,10 @@ github.com/opencontainers/go-digest v1.0.0-rc1 h1:WzifXhOVOEOuFYOJAW6aQqW0TooG2i
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVojFA6h/TRcI=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/otiai10/copy v1.0.2 h1:DDNipYy6RkIkjMwy+AWzgKiNTyj2RUI9yEMeETEpVyc=
github.com/otiai10/copy v1.0.2/go.mod h1:c7RpqBkwMom4bYTSkLSym4VSJz/XtncWRAj/J4PEIMY=
github.com/otiai10/curr v0.0.0-20150429015615-9b4961190c95/go.mod h1:9qAhocn7zKJG+0mI8eUu6xqkFDYS2kb2saOteoSB3cE=
github.com/otiai10/mint v1.3.0/go.mod h1:F5AjcsTsWUqX+Na9fpHb52P8pcRX2CI6A3ctIT91xUo=
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2 h1:JhzVVoYvbOACxoUmOs6V/G4D5nPVUW73rKvXxP4XUJc=

View File

@@ -1,43 +1,11 @@
package handlers
import (
"io/ioutil"
"strings"
"github.com/apex/log"
"github.com/metrue/fx/context"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
)
// Call command handle
func Call(ctx context.Contexter) error {
cli := ctx.GetCliContext()
_ = strings.Join(cli.Args()[1:], " ")
file := cli.Args().First()
src, err := ioutil.ReadFile(file)
if err != nil {
log.Fatalf("Read Source: %v", err)
return err
}
log.Info("Read Source: \u2713")
lang := utils.GetLangFromFileName(file)
fn := types.Func{
Language: lang,
Source: string(src),
}
if _, err := packer.Pack(file, fn); err != nil {
panic(err)
}
// TODO not supported
// if err := api.MustCreate(host.Host, constants.AgentPort).
// Call(file, params, project); err != nil {
// log.Fatalf("call functions on machine %s with %v failed: %v", name, params, err)
// }
return nil
}

View File

@@ -9,8 +9,14 @@ import (
// Up command handle
func Up(ctx context.Contexter) (err error) {
fn := ctx.Get("fn").(types.Func)
image := ctx.Get("image").(string)
fn, ok := ctx.Get("data").(string)
if !ok {
fn = ""
}
image, ok := ctx.Get("image").(string)
if !ok {
image = ""
}
name := ctx.Get("name").(string)
deployer := ctx.Get("deployer").(infra.Deployer)
bindings := ctx.Get("bindings").([]types.PortBinding)

View File

@@ -8,7 +8,6 @@ import (
mockCtx "github.com/metrue/fx/context/mocks"
mockDeployer "github.com/metrue/fx/infra/mocks"
"github.com/metrue/fx/types"
fxTypes "github.com/metrue/fx/types"
)
func TestUp(t *testing.T) {
@@ -18,17 +17,17 @@ func TestUp(t *testing.T) {
ctx := mockCtx.NewMockContexter(ctrl)
deployer := mockDeployer.NewMockDeployer(ctrl)
fn := fxTypes.Func{}
bindings := []types.PortBinding{}
name := "sample-name"
image := "sample-image"
ctx.EXPECT().Get("fn").Return(fn)
data := "sample-data"
ctx.EXPECT().Get("name").Return(name)
ctx.EXPECT().Get("image").Return(image)
ctx.EXPECT().Get("deployer").Return(deployer)
ctx.EXPECT().Get("bindings").Return(bindings)
ctx.EXPECT().Get("data").Return(data)
ctx.EXPECT().GetContext().Return(context.Background()).Times(2)
deployer.EXPECT().Deploy(gomock.Any(), fn, name, image, bindings).Return(nil)
deployer.EXPECT().Deploy(gomock.Any(), data, name, image, bindings).Return(nil)
deployer.EXPECT().GetStatus(gomock.Any(), name).Return(types.Service{
ID: "id-1",
Name: name,

View File

@@ -22,7 +22,7 @@ func CreateClient(client containerruntimes.ContainerRuntime) (d *Deployer, err e
}
// Deploy create a Docker container from given image, and bind the constants.FxContainerExposePort to given port
func (d *Deployer) Deploy(ctx context.Context, fn types.Func, name string, image string, ports []types.PortBinding) (err error) {
func (d *Deployer) Deploy(ctx context.Context, fn string, name string, image string, ports []types.PortBinding) (err error) {
spinner.Start("deploying " + name)
defer func() {
spinner.Stop("deploying "+name, err)

View File

@@ -14,7 +14,7 @@ type Provisioner interface {
// Deployer deploy interface
type Deployer interface {
Deploy(ctx context.Context, fn types.Func, name string, image string, bindings []types.PortBinding) error
Deploy(ctx context.Context, fn string, name string, image string, bindings []types.PortBinding) error
Destroy(ctx context.Context, name string) error
Update(ctx context.Context, name string) error
GetStatus(ctx context.Context, name string) (types.Service, error)

View File

@@ -5,6 +5,7 @@ import (
"os"
"testing"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/types"
)
@@ -31,16 +32,12 @@ func TestK8SDeployer(t *testing.T) {
t.Fatal(err)
}
fn := types.Func{
Language: "node",
Source: `
module.exports = (ctx) => {
ctx.body = 'hello world'
}
`,
data, err := packer.PackIntoK8SConfigMapFile("./fixture")
if err != nil {
t.Fatal(err)
}
ctx := context.Background()
if err := k8s.Deploy(ctx, fn, name, name, bindings); err != nil {
if err := k8s.Deploy(ctx, data, name, name, bindings); err != nil {
t.Fatal(err)
}

View File

@@ -6,7 +6,6 @@ import (
"os"
"github.com/metrue/fx/infra"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/types"
"k8s.io/client-go/kubernetes"
@@ -41,18 +40,13 @@ func Create(kubeconfig string) (*K8S, error) {
// Deploy a image to be a service
func (k *K8S) Deploy(
ctx context.Context,
fn types.Func,
fn string,
name string,
image string,
ports []types.PortBinding,
) error {
// put source code of function docker project into k8s config map
tree, err := packer.PackIntoK8SConfigMapFile(fn)
if err != nil {
return err
}
data := map[string]string{}
data[ConfigMap.AppMetaEnvName] = tree
data[ConfigMap.AppMetaEnvName] = fn
if _, err := k.CreateOrUpdateConfigMap(namespace, name, data); err != nil {
return err
}

View File

@@ -0,0 +1,5 @@
FROM metrue/fx-node-base
COPY . .
EXPOSE 3000
CMD ["node", "app.js"]

9
infra/k8s/fixture/app.js Normal file
View File

@@ -0,0 +1,9 @@
const Koa = require('koa');
const bodyParser = require('koa-bodyparser');
const fx = require('./fx');
const app = new Koa();
app.use(bodyParser());
app.use(fx);
app.listen(3000);

3
infra/k8s/fixture/fx.js Normal file
View File

@@ -0,0 +1,3 @@
module.exports = (ctx) => {
ctx.body = 'hello world'
}

View File

@@ -88,7 +88,7 @@ func (m *MockDeployer) EXPECT() *MockDeployerMockRecorder {
}
// Deploy mocks base method
func (m *MockDeployer) Deploy(ctx context.Context, fn types.Func, name, image string, bindings []types.PortBinding) error {
func (m *MockDeployer) Deploy(ctx context.Context, fn, name, image string, bindings []types.PortBinding) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Deploy", ctx, fn, name, image, bindings)
ret0, _ := ret[0].(error)
@@ -227,7 +227,7 @@ func (mr *MockInfraMockRecorder) HealthCheck() *gomock.Call {
}
// Deploy mocks base method
func (m *MockInfra) Deploy(ctx context.Context, fn types.Func, name, image string, bindings []types.PortBinding) error {
func (m *MockInfra) Deploy(ctx context.Context, fn, name, image string, bindings []types.PortBinding) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Deploy", ctx, fn, name, image, bindings)
ret0, _ := ret[0].(error)

View File

@@ -5,11 +5,11 @@ import (
"os"
"time"
"github.com/metrue/fx/config"
containerruntimes "github.com/metrue/fx/container_runtimes"
"github.com/metrue/fx/context"
"github.com/metrue/fx/packer"
"github.com/metrue/fx/pkg/spinner"
"github.com/metrue/fx/types"
)
// Build image
@@ -20,36 +20,44 @@ func Build(ctx context.Contexter) (err error) {
spinner.Stop(task, err)
}()
cli := ctx.GetCliContext()
name := cli.String("name")
fn := ctx.Get("fn").(types.Func)
docker := ctx.Get("docker").(containerruntimes.ContainerRuntime)
workdir := fmt.Sprintf("/tmp/fx-%d", time.Now().Unix())
defer os.RemoveAll(workdir)
if err := packer.PackIntoDir(fn, workdir); err != nil {
return err
}
if err := docker.BuildImage(ctx.GetContext(), workdir, name); err != nil {
if err := packer.Pack(workdir, ctx.Get("sources").([]string)...); err != nil {
return err
}
nameWithTag := name + ":latest"
if err := docker.TagImage(ctx.GetContext(), name, nameWithTag); err != nil {
return err
}
ctx.Set("image", nameWithTag)
cloudType := ctx.Get("cloud_type").(string)
name := ctx.Get("name").(string)
if cloudType == config.CloudTypeK8S && os.Getenv("K3S") == "" {
data, err := packer.PackIntoK8SConfigMapFile(workdir)
if err != nil {
return err
}
ctx.Set("data", data)
} else {
docker := ctx.Get("docker").(containerruntimes.ContainerRuntime)
if err := docker.BuildImage(ctx.GetContext(), workdir, name); err != nil {
return err
}
if os.Getenv("K3S") != "" {
name := cli.String("name")
username := os.Getenv("DOCKER_USERNAME")
password := os.Getenv("DOCKER_PASSWORD")
if username != "" && password != "" {
if _, err := docker.PushImage(ctx.GetContext(), name); err != nil {
return err
nameWithTag := name + ":latest"
if err := docker.TagImage(ctx.GetContext(), name, nameWithTag); err != nil {
return err
}
ctx.Set("image", nameWithTag)
if os.Getenv("K3S") != "" {
username := os.Getenv("DOCKER_USERNAME")
password := os.Getenv("DOCKER_PASSWORD")
if username != "" && password != "" {
if _, err := docker.PushImage(ctx.GetContext(), name); err != nil {
return err
}
ctx.Set("image", username+"/"+name)
}
ctx.Set("image", username+"/"+name)
}
}
return nil
}

View File

@@ -1,11 +1,9 @@
package middlewares
import (
"io/ioutil"
"os"
"github.com/metrue/fx/context"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
"github.com/pkg/errors"
)
@@ -15,18 +13,18 @@ func Parse(action string) func(ctx context.Contexter) (err error) {
cli := ctx.GetCliContext()
switch action {
case "up":
funcFile := cli.Args().First()
lang := utils.GetLangFromFileName(funcFile)
body, err := ioutil.ReadFile(funcFile)
if err != nil {
return errors.Wrap(err, "read source failed")
sources := []string{}
for _, s := range cli.Args() {
sources = append(sources, s)
}
fn := types.Func{
Language: lang,
Source: string(body),
if len(sources) == 0 {
pwd, err := os.Getwd()
if err != nil {
return err
}
sources = append(sources, pwd)
}
ctx.Set("fn", fn)
ctx.Set("sources", sources)
name := cli.String("name")
ctx.Set("name", name)
port := cli.Int("port")

View File

@@ -20,7 +20,13 @@ func Provision(ctx context.Contexter) (err error) {
cloud := fxConfig.Clouds[fxConfig.CurrentCloud]
var deployer infra.Deployer
if cloud["type"] == config.CloudTypeDocker {
if os.Getenv("KUBECONFIG") != "" {
deployer, err = k8sInfra.CreateDeployer(os.Getenv("KUBECONFIG"))
if err != nil {
return err
}
ctx.Set("cloud_type", config.CloudTypeK8S)
} else if cloud["type"] == config.CloudTypeDocker {
provisioner := dockerInfra.CreateProvisioner(cloud["host"], cloud["user"])
ok, err := provisioner.HealthCheck()
if err != nil {
@@ -43,13 +49,13 @@ func Provision(ctx context.Contexter) (err error) {
if err != nil {
return err
}
ctx.Set("cloud_type", config.CloudTypeDocker)
} else if cloud["type"] == config.CloudTypeK8S {
if os.Getenv("KUBECONFIG") != "" {
deployer, err = k8sInfra.CreateDeployer(cloud["kubeconfig"])
if err != nil {
return err
}
deployer, err = k8sInfra.CreateDeployer(cloud["kubeconfig"])
if err != nil {
return err
}
ctx.Set("cloud_type", config.CloudTypeK8S)
} else {
return fmt.Errorf("unsupport cloud type %s, please make sure you config is correct", cloud["type"])
}

View File

@@ -8,7 +8,7 @@ import (
"github.com/metrue/fx/types"
)
func TestPacker(t *testing.T) {
func TestDockerPacker(t *testing.T) {
ctrl := gomock.NewController(t)
defer ctrl.Finish()

View File

@@ -0,0 +1,5 @@
FROM metrue/fx-node-base
COPY . .
EXPOSE 3000
CMD ["node", "app.js"]

9
packer/fixture/p1/app.js Normal file
View File

@@ -0,0 +1,9 @@
const Koa = require('koa');
const bodyParser = require('koa-bodyparser');
const fx = require('./fx');
const app = new Koa();
app.use(bodyParser());
app.use(fx);
app.listen(3000);

3
packer/fixture/p1/fx.js Normal file
View File

@@ -0,0 +1,3 @@
module.exports = (ctx) => {
ctx.body = 'hello world'
}

3
packer/fixture/p2/fx.js Normal file
View File

@@ -0,0 +1,3 @@
module.exports = (ctx) => {
ctx.body = 'hello world'
}

6
packer/fixture/p3/fx.js Normal file
View File

@@ -0,0 +1,6 @@
const say = require('./helper')
module.exports = (ctx) => {
say("hi")
ctx.body = 'hello world'
}

View File

@@ -0,0 +1,5 @@
const say = (msg) => {
console.log(msg)
}
module.exports = say

View File

@@ -1,9 +1,12 @@
package packer
import (
"fmt"
"io/ioutil"
"os"
"path/filepath"
"strings"
"time"
"encoding/base64"
"encoding/json"
@@ -11,15 +14,106 @@ import (
"github.com/gobuffalo/packr"
"github.com/metrue/fx/types"
"github.com/metrue/fx/utils"
"github.com/otiai10/copy"
"github.com/pkg/errors"
)
// Packer interface
type Packer interface {
Pack(serviceName string, fn types.Func) (types.Project, error)
// Pack pack a file or directory into a Docker project
func Pack(output string, input ...string) error {
if len(input) == 0 {
return fmt.Errorf("source file or directory required")
}
if len(input) == 1 {
file := input[0]
stat, err := os.Stat(file)
if err != nil {
return err
}
if !stat.IsDir() {
lang := utils.GetLangFromFileName(file)
body, err := ioutil.ReadFile(file)
if err != nil {
return errors.Wrap(err, "read source failed")
}
fn := types.Func{
Language: lang,
Source: string(body),
}
if err := PackIntoDir(fn, output); err != nil {
return err
}
return nil
}
}
workdir := fmt.Sprintf("./fx-%d", time.Now().Unix())
defer os.RemoveAll(workdir)
for _, f := range input {
if err := copy.Copy(f, filepath.Join(workdir, f)); err != nil {
return err
}
}
if dockerfile, has := hasDockerfileInDir(workdir); has {
return copy.Copy(filepath.Dir(dockerfile), output)
}
if f, has := hasFxHandleFileInDir(workdir); has {
lang := utils.GetLangFromFileName(f)
body, err := ioutil.ReadFile(f)
if err != nil {
return errors.Wrap(err, "read source failed")
}
fn := types.Func{
Language: lang,
Source: string(body),
}
if err := PackIntoDir(fn, output); err != nil {
return err
}
return copy.Copy(filepath.Dir(f), output)
}
return fmt.Errorf("input directories or files has no Dockerfile or file with fx as name, e.g. fx.js")
}
func hasDockerfileInDir(dir string) (string, bool) {
var dockerfile string
if err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
// nolint
if !info.IsDir() && info.Name() == "Dockerfile" {
dockerfile = path
}
return nil
}); err != nil {
return "", false
}
if dockerfile == "" {
return "", false
}
return dockerfile, true
}
func hasFxHandleFileInDir(dir string) (string, bool) {
var handleFile string
if err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if !info.IsDir() && isHandler(info.Name()) {
handleFile = path
}
return nil
}); err != nil {
return "", false
}
if handleFile == "" {
return "", false
}
return handleFile, true
}
// Pack a function to be a docker project which is web service, handle the imcome request with given function
func Pack(svcName string, fn types.Func) (types.Project, error) {
func pack(svcName string, fn types.Func) (types.Project, error) {
box := packr.NewBox("./images")
pkr := NewDockerPacker(box)
return pkr.Pack(svcName, fn)
@@ -27,7 +121,7 @@ func Pack(svcName string, fn types.Func) (types.Project, error) {
// PackIntoDir pack service code into directory
func PackIntoDir(fn types.Func, outputDir string) error {
project, err := Pack("", fn)
project, err := pack("", fn)
if err != nil {
return err
}
@@ -44,16 +138,24 @@ func PackIntoDir(fn types.Func, outputDir string) error {
}
// PackIntoK8SConfigMapFile pack function a K8S config map file
func PackIntoK8SConfigMapFile(fn types.Func) (string, error) {
project, err := Pack("", fn)
if err != nil {
return "", err
}
func PackIntoK8SConfigMapFile(dir string) (string, error) {
tree := map[string]string{}
for _, file := range project.Files {
tree[file.Path] = file.Body
if err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if !info.IsDir() {
relpath := strings.Replace(path, dir, "", 1)
body, err := ioutil.ReadFile(path)
if err != nil {
return err
}
tree[relpath] = string(body)
}
return nil
}); err != nil {
return "", nil
}
data, err := json.Marshal(tree)
if err != nil {
return "", err

View File

@@ -1,79 +1,122 @@
package packer
import (
"os"
"testing"
"github.com/metrue/fx/types"
)
func TestPack(t *testing.T) {
mockSource := `
func TestPacker(t *testing.T) {
t.Run("Pack directory with Dockerfile in it", func(t *testing.T) {
input := "./fixture/p1"
output := "output-1"
defer func() {
os.RemoveAll(output)
}()
if err := Pack(output, input); err != nil {
t.Fatal(err)
}
})
t.Run("Pack directory only fx.js in it", func(t *testing.T) {
input := "./fixture/p2"
output := "output-2"
defer func() {
os.RemoveAll(output)
}()
if err := Pack(output, input); err != nil {
t.Fatal(err)
}
})
t.Run("Pack directory with fx.js and helper in it", func(t *testing.T) {
input := "./fixture/p3"
output := "output-3"
defer func() {
os.RemoveAll(output)
}()
if err := Pack(output, input); err != nil {
t.Fatal(err)
}
})
t.Run("Pack files list with fx.js in it", func(t *testing.T) {
handleFile := "./fixture/p3/fx.js"
helperFile := "./fixture/p3/helper.js"
output := "output-4"
defer func() {
os.RemoveAll(output)
}()
if err := Pack(output, handleFile, helperFile); err != nil {
t.Fatal(err)
}
})
t.Run("Pack files list without fx.js in it", func(t *testing.T) {
f1 := "./fixture/p3/helper.js"
f2 := "./fixture/p3/helper.js"
output := "output-5"
defer func() {
os.RemoveAll(output)
}()
if err := Pack(output, f1, f2); err == nil {
t.Fatalf("should report error when there is not Dockerfile or fx.[ext] in it")
}
})
t.Run("pack", func(t *testing.T) {
mockSource := `
module.exports = ({a, b}) => {
return a + b
}
`
fn := types.Func{
Language: "node",
Source: mockSource,
}
fn := types.Func{
Language: "node",
Source: mockSource,
}
serviceName := "service-mock"
project, err := Pack(serviceName, fn)
if err != nil {
t.Fatal(err)
}
serviceName := "service-mock"
project, err := pack(serviceName, fn)
if err != nil {
t.Fatal(err)
}
if project.Name != serviceName {
t.Fatalf("should get %s but got %s", serviceName, project.Name)
}
if project.Name != serviceName {
t.Fatalf("should get %s but got %s", serviceName, project.Name)
}
if project.Language != "node" {
t.Fatal("incorrect Language")
}
if project.Language != "node" {
t.Fatal("incorrect Language")
}
if len(project.Files) != 3 {
t.Fatal("node project should have 3 files")
}
if len(project.Files) != 3 {
t.Fatal("node project should have 3 files")
}
for _, file := range project.Files {
if file.Path == "fx.js" {
if file.IsHandler == false {
t.Fatal("fx.js should be handler")
}
if file.Body != mockSource {
t.Fatalf("should get %s but got %v", mockSource, file.Body)
}
} else if file.Path == "Dockerfile" {
if file.IsHandler == true {
t.Fatalf("should get %v but got %v", false, file.IsHandler)
}
} else {
if file.IsHandler == true {
t.Fatalf("should get %v but %v", false, file.IsHandler)
for _, file := range project.Files {
if file.Path == "fx.js" {
if file.IsHandler == false {
t.Fatal("fx.js should be handler")
}
if file.Body != mockSource {
t.Fatalf("should get %s but got %v", mockSource, file.Body)
}
} else if file.Path == "Dockerfile" {
if file.IsHandler == true {
t.Fatalf("should get %v but got %v", false, file.IsHandler)
}
} else {
if file.IsHandler == true {
t.Fatalf("should get %v but %v", false, file.IsHandler)
}
}
}
}
})
}
func TestTreeAndUnTree(t *testing.T) {
mockSource := `
package fx;
import org.json.JSONObject;
public class Fx {
public int handle(JSONObject input) {
String a = input.get("a").toString();
String b = input.get("b").toString();
return Integer.parseInt(a) + Integer.parseInt(b);
}
}
`
fn := types.Func{
Language: "java",
Source: mockSource,
}
_, err := PackIntoK8SConfigMapFile(fn)
_, err := PackIntoK8SConfigMapFile("./fixture/p1")
if err != nil {
t.Fatal(err)
}