fn: rename from fnctl (#321)

* fn: rename from fnctl

* fn: predicting the release version for installer

* fn: predicting the release version for installer
This commit is contained in:
C Cirello
2016-11-21 17:24:26 +01:00
committed by GitHub
parent 0343c4990c
commit c0512a4cbd
45 changed files with 172 additions and 172 deletions

3
fn/.gitignore vendored Normal file
View File

@@ -0,0 +1,3 @@
fn
vendor/
/fn.exe

13
fn/Dockerfile Normal file
View File

@@ -0,0 +1,13 @@
FROM alpine
RUN apk --update upgrade && \
apk add curl ca-certificates && \
update-ca-certificates && \
rm -rf /var/cache/apk/*
COPY entrypoint.sh /
COPY fn /
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

19
fn/Makefile Normal file
View File

@@ -0,0 +1,19 @@
all: vendor
go build -o fn
./fn
docker: vendor
GOOS=linux go build -o fn
docker build -t iron/fn .
docker push iron/fn
vendor:
glide install -v
test:
go test -v $(shell glide nv)
release: docker
GOOS=linux go build -o fn_linux
GOOS=darwin go build -o fn_mac
GOOS=windows go build -o fn.exe

204
fn/README.md Normal file
View File

@@ -0,0 +1,204 @@
# IronFunctions CLI
## Creating Functions
### init
Init will help you create a [function file](../docs/function-file.md) (func.yaml) in the current directory.
To make things simple, we try to use convention over configuration, so `init` will look for a file named `func.{language-extension}`. For example,
if you are using Node, put the code that you want to execute in the file `func.js`. If you are using Python, use `func.py`. Ruby, use `func.rb`. Go, `func.go`. Etc.
Run:
```sh
fn init <DOCKER_HUB_USERNAME>/<FUNCTION_NAME>
```
If you want to override the convention with configuration, you can do that as well using:
```sh
fn init [--runtime node] [--entrypoint "node hello.js"] <DOCKER_HUB_USERNAME>/<FUNCTION_NAME>
```
Or, if you want full control, just make a Dockerfile. If `init` finds a Dockerfile, it will use that instead of runtime and entrypoint.
### Build, Bump, Run, Push
`fn` provides a few commands you'll use while creating and updating your functions: `build`, `bump`, `run` and `push`.
Build will build the image for your function.
```sh
fn build
```
Bump will bump the version number in your func.yaml file. Versions must be in [semver](http://semver.org/) format.
```sh
fn bump
```
Run will help you test your function. Functions read input from STDIN, so you can pipe the payload into the function like this:
```sh
cat `payload.json` | fn run
```
Push will push the function image to Docker Hub.
```sh
fn push
```
## Using the API
You can operate IronFunctions from the command line.
```sh
$ fn apps # list apps
myapp
$ fn apps create otherapp # create new app
otherapp created
$ fn apps describe otherapp # describe an app
app: otherapp
no specific configuration
$ fn apps
myapp
otherapp
$ fn routes myapp # list routes of an app
path image
/hello iron/hello
$ fn routes create otherapp /hello iron/hello # create route
/hello created with iron/hello
$ fn routes delete otherapp hello # delete route
/hello deleted
```
## Application level configuration
When creating an application, you can configure it to tweak its behavior and its
routes' with an appropriate flag, `config`.
Thus a more complete example of an application creation will look like:
```sh
fn apps create --config DB_URL=http://example.org/ otherapp
```
`--config` is a map of values passed to the route runtime in the form of
environment variables.
Repeated calls to `fn apps create` will trigger an update of the given
route, thus you will be able to change any of these attributes later in time
if necessary.
## Route level configuration
When creating a route, you can configure it to tweak its behavior, the possible
choices are: `memory`, `type` and `config`.
Thus a more complete example of route creation will look like:
```sh
fn routes create --memory 256 --type async --config DB_URL=http://example.org/ otherapp /hello iron/hello
```
`--memory` is number of usable MiB for this function. If during the execution it
exceeds this maximum threshold, it will halt and return an error in the logs.
`--type` is the type of the function. Either `sync`, in which the client waits
until the request is successfully completed, or `async`, in which the clients
dispatches a new request, gets a task ID back and closes the HTTP connection.
`--config` is a map of values passed to the route runtime in the form of
environment variables.
Repeated calls to `fn route create` will trigger an update of the given
route, thus you will be able to change any of these attributes later in time
if necessary.
## Changing target host
`fn` is configured by default to talk http://localhost:8080.
You may reconfigure it to talk to a remote installation by updating a local
environment variable (`$API_URL`):
```sh
$ export API_URL="http://myfunctions.example.org/"
$ fn ...
```
## Publish
Also there is the publish command that is going to scan all local directory for
functions, rebuild them and push them to Docker Hub and update them in
IronFunction.
```sh
$ fn publish
path result
/app/hello done
/app/hello-sync error: no Dockerfile found for this function
/app/test done
```
It works by scanning all children directories of the current working directory,
following this convention:
<pre><code>┌───────┐
│ ./ │
└───┬───┘
│ ┌───────┐
├────▶│ myapp │
│ └───┬───┘
│ │ ┌───────┐
│ ├────▶│route1 │
│ │ └───────┘
│ │ │ ┌─────────┐
│ │ ├────▶│subroute1│
│ │ │ └─────────┘
│ ┌───────┐
├────▶│ other │
│ └───┬───┘
│ │ ┌───────┐
│ ├────▶│route1 │
│ │ └───────┘</code></pre>
It will render this pattern of updates:
```sh
$ fn publish
path result
/myapp/route1/subroute1 done
/other/route1 done
```
It means that first subdirectory are always considered app names (e.g. `myapp`
and `other`), each subdirectory of these firsts are considered part of the route
(e.g. `route1/subroute1`).
`fn publish` expects that each directory to contain a file `func.yaml`
which instructs `fn` on how to act with that particular update, and a
Dockerfile which it is going to use to build the image and push to Docker Hub.
## Contributing
Ensure you have Go configured and installed in your environment. Once it is
done, run:
```sh
$ make
```
It will build fn compatible with your local environment. You can test this
CLI, right away with:
```sh
$ ./fn
```

236
fn/apps.go Normal file
View File

@@ -0,0 +1,236 @@
package main
import (
"encoding/json"
"errors"
"fmt"
"os"
"text/tabwriter"
"github.com/iron-io/functions_go"
"github.com/urfave/cli"
)
type appsCmd struct {
*functions.AppsApi
}
func apps() cli.Command {
a := appsCmd{AppsApi: functions.NewAppsApi()}
return cli.Command{
Name: "apps",
Usage: "list apps",
ArgsUsage: "fn apps",
Action: a.list,
Subcommands: []cli.Command{
{
Name: "create",
Usage: "create a new app",
Action: a.create,
Flags: []cli.Flag{
cli.StringSliceFlag{
Name: "config",
Usage: "application configuration",
},
},
},
{
Name: "config",
Usage: "operate an application configuration set",
Action: a.configList,
Flags: []cli.Flag{
cli.BoolFlag{
Name: "shell",
Usage: "output in shell format",
},
cli.BoolFlag{
Name: "json",
Usage: "output in JSON format",
},
},
Subcommands: []cli.Command{
{
Name: "set",
Description: "store a configuration key for this application",
Usage: "<app> <key> <value>",
Action: a.configSet,
},
{
Name: "unset",
Description: "remove a configuration key for this application",
Usage: "<app> <key> <value>",
Action: a.configUnset,
},
},
},
},
}
}
func (a *appsCmd) list(c *cli.Context) error {
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
wrapper, _, err := a.AppsGet()
if err != nil {
return fmt.Errorf("error getting app: %v", err)
}
if len(wrapper.Apps) == 0 {
fmt.Println("no apps found")
return nil
}
for _, app := range wrapper.Apps {
fmt.Println(app.Name)
}
return nil
}
func (a *appsCmd) create(c *cli.Context) error {
if c.Args().First() == "" {
return errors.New("error: app creating takes one argument, an app name")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
body := functions.AppWrapper{App: functions.App{
Name: c.Args().Get(0),
Config: extractEnvConfig(c.StringSlice("config")),
}}
wrapper, _, err := a.AppsPost(body)
if err != nil {
return fmt.Errorf("error creating app: %v", err)
}
fmt.Println(wrapper.App.Name, "created")
return nil
}
func (a *appsCmd) configList(c *cli.Context) error {
if c.Args().First() == "" {
return errors.New("error: app description takes one argument, an app name")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
appName := c.Args().Get(0)
wrapper, _, err := a.AppsAppGet(appName)
if err != nil {
return fmt.Errorf("error creating app: %v", err)
}
config := wrapper.App.Config
if len(config) == 0 {
return errors.New("this application has no configurations")
}
if c.Bool("json") {
if err := json.NewEncoder(os.Stdout).Encode(config); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
} else if c.Bool("shell") {
for k, v := range wrapper.App.Config {
fmt.Print("export ", k, "=", v, "\n")
}
} else {
fmt.Println(wrapper.App.Name, "configuration:")
w := tabwriter.NewWriter(os.Stdout, 0, 8, 1, ' ', 0)
for k, v := range config {
fmt.Fprint(w, k, ":\t", v, "\n")
}
w.Flush()
}
return nil
}
func (a *appsCmd) configSet(c *cli.Context) error {
if c.Args().Get(0) == "" || c.Args().Get(1) == "" || c.Args().Get(2) == "" {
return errors.New("error: application configuration setting takes three arguments: an app name, a key and a value")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
appName := c.Args().Get(0)
key := c.Args().Get(1)
value := c.Args().Get(2)
wrapper, _, err := a.AppsAppGet(appName)
if err != nil {
return fmt.Errorf("error creating app: %v", err)
}
config := wrapper.App.Config
if config == nil {
config = make(map[string]string)
}
config[key] = value
if err := a.storeApp(appName, config); err != nil {
return fmt.Errorf("error updating app configuration: %v", err)
}
fmt.Println(wrapper.App.Name, "updated", key, "with", value)
return nil
}
func (a *appsCmd) configUnset(c *cli.Context) error {
if c.Args().Get(0) == "" || c.Args().Get(1) == "" {
return errors.New("error: application configuration setting takes three arguments: an app name, a key and a value")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
appName := c.Args().Get(0)
key := c.Args().Get(1)
wrapper, _, err := a.AppsAppGet(appName)
if err != nil {
return fmt.Errorf("error creating app: %v", err)
}
config := wrapper.App.Config
if config == nil {
config = make(map[string]string)
}
if _, ok := config[key]; !ok {
return fmt.Errorf("configuration key %s not found", key)
}
delete(config, key)
if err := a.storeApp(appName, config); err != nil {
return fmt.Errorf("error updating app configuration: %v", err)
}
fmt.Println(wrapper.App.Name, "removed", key)
return nil
}
func (a *appsCmd) storeApp(appName string, config map[string]string) error {
body := functions.AppWrapper{App: functions.App{
Name: appName,
Config: config,
}}
if _, _, err := a.AppsPost(body); err != nil {
return fmt.Errorf("error updating app configuration: %v", err)
}
return nil
}

46
fn/build.go Normal file
View File

@@ -0,0 +1,46 @@
package main
import (
"fmt"
"os"
"github.com/urfave/cli"
)
func build() cli.Command {
cmd := buildcmd{commoncmd: &commoncmd{}}
flags := append([]cli.Flag{}, cmd.flags()...)
return cli.Command{
Name: "build",
Usage: "build function version",
Flags: flags,
Action: cmd.scan,
}
}
type buildcmd struct {
*commoncmd
}
func (b *buildcmd) scan(c *cli.Context) error {
b.commoncmd.scan(b.walker)
return nil
}
func (b *buildcmd) walker(path string, info os.FileInfo, err error) error {
walker(path, info, err, b.build)
return nil
}
// build will take the found valid function and build it
func (b *buildcmd) build(path string) error {
fmt.Fprintln(b.verbwriter, "building", path)
ff, err := b.buildfunc(path)
if err != nil {
return err
}
fmt.Printf("Function %v built successfully.\n", ff.FullName())
return nil
}

83
fn/bump.go Normal file
View File

@@ -0,0 +1,83 @@
package main
import (
"fmt"
"os"
"strings"
bumper "github.com/giantswarm/semver-bump/bump"
"github.com/giantswarm/semver-bump/storage"
"github.com/urfave/cli"
)
var (
initialVersion = "0.0.1"
)
func bump() cli.Command {
cmd := bumpcmd{commoncmd: &commoncmd{}}
flags := append([]cli.Flag{}, cmd.flags()...)
return cli.Command{
Name: "bump",
Usage: "bump function version",
Flags: flags,
Action: cmd.scan,
}
}
type bumpcmd struct {
*commoncmd
}
func (b *bumpcmd) scan(c *cli.Context) error {
b.commoncmd.scan(b.walker)
return nil
}
func (b *bumpcmd) walker(path string, info os.FileInfo, err error) error {
walker(path, info, err, b.bump)
return nil
}
// bump will take the found valid function and bump its version
func (b *bumpcmd) bump(path string) error {
fmt.Fprintln(b.verbwriter, "bumping version for", path)
funcfile, err := parsefuncfile(path)
if err != nil {
return err
}
funcfile.Name = cleanImageName(funcfile.Name)
if funcfile.Version == "" {
funcfile.Version = initialVersion
}
s, err := storage.NewVersionStorage("local", funcfile.Version)
if err != nil {
return err
}
version := bumper.NewSemverBumper(s, "")
newver, err := version.BumpPatchVersion("", "")
if err != nil {
return err
}
funcfile.Version = newver.String()
if err := storefuncfile(path, funcfile); err != nil {
return err
}
fmt.Println("Bumped to version", funcfile.Version)
return nil
}
func cleanImageName(name string) string {
if i := strings.Index(name, ":"); i != -1 {
name = name[:i]
}
return name
}

294
fn/common.go Normal file
View File

@@ -0,0 +1,294 @@
package main
import (
"bytes"
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
"strings"
"text/template"
"time"
"github.com/iron-io/functions/fn/langs"
"github.com/urfave/cli"
)
func isFuncfile(path string, info os.FileInfo) bool {
if info.IsDir() {
return false
}
basefn := filepath.Base(path)
for _, fn := range validfn {
if basefn == fn {
return true
}
}
return false
}
func walker(path string, info os.FileInfo, err error, f func(path string) error) {
if err := f(path); err != nil {
fmt.Fprintln(os.Stderr, path, err)
}
}
type commoncmd struct {
wd string
verbose bool
force bool
recursively bool
verbwriter io.Writer
}
func (c *commoncmd) flags() []cli.Flag {
return []cli.Flag{
cli.StringFlag{
Name: "d",
Usage: "working directory",
Destination: &c.wd,
EnvVar: "WORK_DIR",
Value: "./",
},
cli.BoolFlag{
Name: "v",
Usage: "verbose mode",
Destination: &c.verbose,
},
cli.BoolFlag{
Name: "f",
Usage: "force updating of all functions that are already up-to-date",
Destination: &c.force,
},
cli.BoolFlag{
Name: "r",
Usage: "recursively scan all functions",
Destination: &c.recursively,
},
}
}
func (c *commoncmd) scan(walker func(path string, info os.FileInfo, err error) error) {
c.verbwriter = ioutil.Discard
if c.verbose {
c.verbwriter = os.Stderr
}
var walked bool
err := filepath.Walk(c.wd, func(path string, info os.FileInfo, err error) error {
if !c.recursively && path != c.wd && info.IsDir() {
return filepath.SkipDir
}
if !isFuncfile(path, info) {
return nil
}
if c.recursively && !c.force && !isstale(path) {
return nil
}
e := walker(path, info, err)
now := time.Now()
os.Chtimes(path, now, now)
walked = true
return e
})
if err != nil {
fmt.Fprintf(c.verbwriter, "file walk error: %s\n", err)
}
if !walked {
fmt.Println("No function file found.")
return
}
}
// Theory of operation: this takes an optimistic approach to detect whether a
// package must be rebuild/bump/published. It loads for all files mtime's and
// compare with functions.json own mtime. If any file is younger than
// functions.json, it triggers a rebuild.
// The problem with this approach is that depending on the OS running it, the
// time granularity of these timestamps might lead to false negatives - that is
// a package that is stale but it is not recompiled. A more elegant solution
// could be applied here, like https://golang.org/src/cmd/go/pkg.go#L1111
func isstale(path string) bool {
fi, err := os.Stat(path)
if err != nil {
return true
}
fnmtime := fi.ModTime()
dir := filepath.Dir(path)
err = filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if info.IsDir() {
return nil
}
if info.ModTime().After(fnmtime) {
return errors.New("found stale package")
}
return nil
})
return err != nil
}
func (c commoncmd) buildfunc(path string) (*funcfile, error) {
funcfile, err := parsefuncfile(path)
if err != nil {
return nil, err
}
if err := c.localbuild(path, funcfile.Build); err != nil {
return nil, err
}
if err := c.dockerbuild(path, funcfile); err != nil {
return nil, err
}
return funcfile, nil
}
func (c commoncmd) localbuild(path string, steps []string) error {
for _, cmd := range steps {
exe := exec.Command("/bin/sh", "-c", cmd)
exe.Dir = filepath.Dir(path)
exe.Stderr = c.verbwriter
exe.Stdout = c.verbwriter
fmt.Fprintf(c.verbwriter, "- %s:\n", cmd)
if err := exe.Run(); err != nil {
return fmt.Errorf("error running command %v (%v)", cmd, err)
}
}
return nil
}
func (c commoncmd) dockerbuild(path string, ff *funcfile) error {
dir := filepath.Dir(path)
var helper langs.LangHelper
dockerfile := filepath.Join(dir, "Dockerfile")
if !exists(dockerfile) {
err := writeTmpDockerfile(dir, ff)
defer os.Remove(filepath.Join(dir, "Dockerfile"))
if err != nil {
return err
}
helper, err = langs.GetLangHelper(*ff.Runtime)
if err != nil {
return err
}
if helper.HasPreBuild() {
err := helper.PreBuild()
if err != nil {
return err
}
}
}
fmt.Printf("Building image %v\n", ff.FullName())
cmd := exec.Command("docker", "build", "-t", ff.FullName(), ".")
cmd.Dir = dir
cmd.Stderr = os.Stderr
cmd.Stdout = os.Stdout
if err := cmd.Run(); err != nil {
return fmt.Errorf("error running docker build: %v", err)
}
if helper != nil {
err := helper.AfterBuild()
if err != nil {
return err
}
}
return nil
}
func exists(name string) bool {
if _, err := os.Stat(name); err != nil {
if os.IsNotExist(err) {
return false
}
}
return true
}
var acceptableFnRuntimes = map[string]string{
"elixir": "iron/elixir",
"erlang": "iron/erlang",
"gcc": "iron/gcc",
"go": "iron/go",
"java": "iron/java",
"leiningen": "iron/leiningen",
"mono": "iron/mono",
"node": "iron/node",
"perl": "iron/perl",
"php": "iron/php",
"python": "iron/python",
"ruby": "iron/ruby",
"scala": "iron/scala",
}
const tplDockerfile = `FROM {{ .BaseImage }}
WORKDIR /function
ADD . /function/
ENTRYPOINT [{{ .Entrypoint }}]
`
func writeTmpDockerfile(dir string, ff *funcfile) error {
if ff.Entrypoint == nil || *ff.Entrypoint == "" {
return errors.New("entrypoint is missing")
}
runtime, tag := ff.RuntimeTag()
rt, ok := acceptableFnRuntimes[runtime]
if !ok {
return fmt.Errorf("cannot use runtime %s", runtime)
}
if tag != "" {
rt = fmt.Sprintf("%s:%s", rt, tag)
}
fd, err := os.Create(filepath.Join(dir, "Dockerfile"))
if err != nil {
return err
}
// convert entrypoint string to slice
epvals := strings.Fields(*ff.Entrypoint)
var buffer bytes.Buffer
for i, s := range epvals {
if i > 0 {
buffer.WriteString(", ")
}
buffer.WriteString("\"")
buffer.WriteString(s)
buffer.WriteString("\"")
}
fmt.Println(buffer.String())
t := template.Must(template.New("Dockerfile").Parse(tplDockerfile))
err = t.Execute(fd, struct {
BaseImage, Entrypoint string
}{rt, buffer.String()})
fd.Close()
return err
}
func extractEnvConfig(configs []string) map[string]string {
c := make(map[string]string)
for _, v := range configs {
kv := strings.SplitN(v, "=", 2)
c[kv[0]] = os.ExpandEnv(kv[1])
}
return c
}

7
fn/entrypoint.sh Normal file
View File

@@ -0,0 +1,7 @@
#!/bin/sh
HOST=$(/sbin/ip route|awk '/default/ { print $3 }')
echo "$HOST default localhost localhost.local" > /etc/hosts
/fn "$@"

13
fn/errors.go Normal file
View File

@@ -0,0 +1,13 @@
package main
type notFoundError struct {
S string
}
func (e *notFoundError) Error() string {
return e.S
}
func newNotFoundError(s string) *notFoundError {
return &notFoundError{S: s}
}

125
fn/funcfile.go Normal file
View File

@@ -0,0 +1,125 @@
package main
import (
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"strings"
yaml "gopkg.in/yaml.v2"
)
var (
validfn = [...]string{
"func.yaml",
"func.yml",
"func.json",
}
errUnexpectedFileFormat = errors.New("unexpected file format for function file")
)
type funcfile struct {
App *string `yaml:"app,omitempty",json:"app,omitempty"`
Name string `yaml:"name,omitempty",json:"name,omitempty"`
Version string `yaml:"version,omitempty",json:"version,omitempty"`
Runtime *string `yaml:"runtime,omitempty",json:"runtime,omitempty"`
Entrypoint *string `yaml:"entrypoint,omitempty",json:"entrypoint,omitempty"`
Route *string `yaml:"route,omitempty",json:"route,omitempty"`
Type *string `yaml:"type,omitempty",json:"type,omitempty"`
Memory *int64 `yaml:"memory,omitempty",json:"memory,omitempty"`
Config map[string]string `yaml:"config,omitempty",json:"config,omitempty"`
Build []string `yaml:"build,omitempty",json:"build,omitempty"`
}
func (ff *funcfile) FullName() string {
fname := ff.Name
if ff.Version != "" {
fname = fmt.Sprintf("%s:%s", fname, ff.Version)
}
return fname
}
func (ff *funcfile) RuntimeTag() (runtime, tag string) {
if ff.Runtime == nil {
return "", ""
}
rt := *ff.Runtime
tagpos := strings.Index(rt, ":")
if tagpos == -1 {
return rt, ""
}
return rt[:tagpos], rt[tagpos+1:]
}
func findFuncfile() (*funcfile, error) {
for _, fn := range validfn {
if exists(fn) {
return parsefuncfile(fn)
}
}
return nil, newNotFoundError("could not find function file")
}
func parsefuncfile(path string) (*funcfile, error) {
ext := filepath.Ext(path)
switch ext {
case ".json":
return decodeFuncfileJSON(path)
case ".yaml", ".yml":
return decodeFuncfileYAML(path)
}
return nil, errUnexpectedFileFormat
}
func storefuncfile(path string, ff *funcfile) error {
ext := filepath.Ext(path)
switch ext {
case ".json":
return encodeFuncfileJSON(path, ff)
case ".yaml", ".yml":
return encodeFuncfileYAML(path, ff)
}
return errUnexpectedFileFormat
}
func decodeFuncfileJSON(path string) (*funcfile, error) {
f, err := os.Open(path)
if err != nil {
return nil, fmt.Errorf("could not open %s for parsing. Error: %v", path, err)
}
ff := new(funcfile)
err = json.NewDecoder(f).Decode(ff)
return ff, err
}
func decodeFuncfileYAML(path string) (*funcfile, error) {
b, err := ioutil.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("could not open %s for parsing. Error: %v", path, err)
}
ff := new(funcfile)
err = yaml.Unmarshal(b, ff)
return ff, err
}
func encodeFuncfileJSON(path string, ff *funcfile) error {
f, err := os.Open(path)
if err != nil {
return fmt.Errorf("could not open %s for encoding. Error: %v", path, err)
}
return json.NewEncoder(f).Encode(ff)
}
func encodeFuncfileYAML(path string, ff *funcfile) error {
b, err := yaml.Marshal(ff)
if err != nil {
return fmt.Errorf("could not encode function file. Error: %v", err)
}
return ioutil.WriteFile(path, b, os.FileMode(0644))
}

127
fn/glide.lock generated Normal file
View File

@@ -0,0 +1,127 @@
hash: a7faac39f56e73fb3987d7a00062a52817432ba5e9620cc83dca15b75496f926
updated: 2016-11-09T20:59:19.840009806+01:00
imports:
- name: github.com/aws/aws-sdk-go
version: 90dec2183a5f5458ee79cbaf4b8e9ab910bc81a6
subpackages:
- aws
- aws/awserr
- aws/awsutil
- aws/client
- aws/client/metadata
- aws/corehandlers
- aws/credentials
- aws/credentials/ec2rolecreds
- aws/defaults
- aws/ec2metadata
- aws/request
- aws/session
- aws/signer/v4
- private/endpoints
- private/protocol
- private/protocol/json/jsonutil
- private/protocol/jsonrpc
- private/protocol/rest
- private/protocol/restjson
- service/lambda
- name: github.com/Azure/go-ansiterm
version: fa152c58bc15761d0200cb75fe958b89a9d4888e
subpackages:
- winterm
- name: github.com/coreos/go-semver
version: 8ab6407b697782a06568d4b7f1db25550ec2e4c6
subpackages:
- semver
- name: github.com/docker/docker
version: fae5a9e053ad06bea0429babae2507762d8cc1de
subpackages:
- api/types/blkiodev
- api/types/container
- api/types/filters
- api/types/mount
- api/types/strslice
- api/types/swarm
- api/types/versions
- opts
- pkg/archive
- pkg/fileutils
- pkg/homedir
- pkg/idtools
- pkg/ioutils
- pkg/jsonlog
- pkg/jsonmessage
- pkg/longpath
- pkg/pools
- pkg/promise
- pkg/stdcopy
- pkg/system
- pkg/term
- pkg/term/windows
- name: github.com/docker/go-connections
version: f512407a188ecb16f31a33dbc9c4e4814afc1b03
subpackages:
- nat
- name: github.com/docker/go-units
version: 8a7beacffa3009a9ac66bad506b18ffdd110cf97
- name: github.com/fsouza/go-dockerclient
version: 5cfde1d138cd2cdc13e4aa36af631beb19dcbe9c
- name: github.com/giantswarm/semver-bump
version: 7ec6ac8985c24dd50b4942f9a908d13cdfe70f23
subpackages:
- bump
- storage
- name: github.com/go-ini/ini
version: 6e4869b434bd001f6983749881c7ead3545887d8
- name: github.com/go-resty/resty
version: 24dc7ba4bc1ef9215048b28e7248f99c42901db5
- name: github.com/hashicorp/go-cleanhttp
version: ad28ea4487f05916463e2423a55166280e8254b5
- name: github.com/iron-io/functions_go
version: 7f5bf75abece5380e916b594e17a552be365276d
- name: github.com/iron-io/iron_go3
version: b50ecf8ff90187fc5fabccd9d028dd461adce4ee
subpackages:
- api
- config
- worker
- name: github.com/iron-io/lambda
version: d883e4b5ef216c3fcda72cf6628d9d72dd53be49
subpackages:
- lambda
- name: github.com/jmespath/go-jmespath
version: 3433f3ea46d9f8019119e7dd41274e112a2359a9
- name: github.com/juju/errgo
version: 08cceb5d0b5331634b9826762a8fd53b29b86ad8
subpackages:
- errors
- name: github.com/Microsoft/go-winio
version: ce2922f643c8fd76b46cadc7f404a06282678b34
- name: github.com/opencontainers/runc
version: 49ed0a10e4edba88f9221ec730d668099f6d6de8
subpackages:
- libcontainer/system
- libcontainer/user
- name: github.com/satori/go.uuid
version: 879c5887cd475cd7864858769793b2ceb0d44feb
- name: github.com/Sirupsen/logrus
version: 4b6ea7319e214d98c938f12692336f7ca9348d6b
- name: github.com/urfave/cli
version: d86a009f5e13f83df65d0d6cee9a2e3f1445f0da
- name: golang.org/x/crypto
version: c10c31b5e94b6f7a0283272dc2bb27163dcea24b
subpackages:
- ssh/terminal
- name: golang.org/x/net
version: f315505cf3349909cdf013ea56690da34e96a451
subpackages:
- context
- context/ctxhttp
- publicsuffix
- name: golang.org/x/sys
version: c200b10b5d5e122be351b67af224adc6128af5bf
subpackages:
- unix
- windows
- name: gopkg.in/yaml.v2
version: a5b47d31c556af34a302ce5d659e6fea44d90de0
testImports: []

22
fn/glide.yaml Normal file
View File

@@ -0,0 +1,22 @@
package: github.com/iron-io/functions/fn
import:
- package: github.com/docker/docker
subpackages:
- pkg/jsonmessage
- package: github.com/giantswarm/semver-bump
subpackages:
- bump
- storage
- package: github.com/iron-io/functions_go
version: 7f5bf75abece5380e916b594e17a552be365276d
- package: github.com/iron-io/iron_go3
subpackages:
- config
- package: github.com/iron-io/lambda
subpackages:
- lambda
- package: github.com/urfave/cli
- package: golang.org/x/crypto
subpackages:
- ssh/terminal
- package: gopkg.in/yaml.v2

165
fn/init.go Normal file
View File

@@ -0,0 +1,165 @@
package main
/*
usage: fn init <name>
If there's a Dockerfile found, this will generate the basic file with just the image name. exit
It will then try to decipher the runtime based on the files in the current directory, if it can't figure it out, it will ask.
It will then take a best guess for what the entrypoint will be based on the language, it it can't guess, it will ask.
*/
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/iron-io/functions/fn/langs"
"github.com/urfave/cli"
)
var (
fileExtToRuntime = map[string]string{
".go": "go",
".js": "node",
".rb": "ruby",
}
fnInitRuntimes []string
)
func init() {
for rt := range fileExtToRuntime {
fnInitRuntimes = append(fnInitRuntimes, rt)
}
}
type initFnCmd struct {
name string
force bool
runtime *string
entrypoint *string
}
func initFn() cli.Command {
a := initFnCmd{}
return cli.Command{
Name: "init",
Usage: "create a local func.yaml file",
Description: "Creates a func.yaml file in the current directory. ",
ArgsUsage: "<DOCKERHUB_USERNAME/FUNCTION_NAME>",
Action: a.init,
Flags: []cli.Flag{
cli.BoolFlag{
Name: "force, f",
Usage: "overwrite existing func.yaml",
Destination: &a.force,
},
cli.StringFlag{
Name: "runtime",
Usage: "choose an existing runtime - " + strings.Join(fnInitRuntimes, ", "),
Destination: a.runtime,
},
cli.StringFlag{
Name: "entrypoint",
Usage: "entrypoint is the command to run to start this function - equivalent to Dockerfile ENTRYPOINT.",
Destination: a.entrypoint,
},
},
}
}
func (a *initFnCmd) init(c *cli.Context) error {
if !a.force {
ff, err := findFuncfile()
if _, ok := err.(*notFoundError); !ok && err != nil {
return err
}
if ff != nil {
return errors.New("function file already exists")
}
}
err := a.buildFuncFile(c)
if err != nil {
return err
}
ff := &funcfile{
Name: a.name,
Runtime: a.runtime,
Version: initialVersion,
Entrypoint: a.entrypoint,
}
if err := encodeFuncfileYAML("func.yaml", ff); err != nil {
return err
}
fmt.Println("func.yaml created.")
return nil
}
func (a *initFnCmd) buildFuncFile(c *cli.Context) error {
pwd, err := os.Getwd()
if err != nil {
return fmt.Errorf("error detecting current working directory: %s\n", err)
}
a.name = c.Args().First()
if a.name == "" || strings.Contains(a.name, ":") {
return errors.New("Please specify a name for your function in the following format <DOCKERHUB_USERNAME>/<FUNCTION_NAME>")
}
if exists("Dockerfile") {
fmt.Println("Dockerfile found, will use that to build.")
return nil
}
var rt string
if a.runtime == nil || *a.runtime == "" {
rt, err = detectRuntime(pwd)
if err != nil {
return err
}
a.runtime = &rt
fmt.Printf("assuming %v runtime\n", rt)
}
if _, ok := acceptableFnRuntimes[*a.runtime]; !ok {
return fmt.Errorf("init does not support the %s runtime, you'll have to create your own Dockerfile for this function", *a.runtime)
}
if a.entrypoint == nil || *a.entrypoint == "" {
ep, err := detectEntrypoint(*a.runtime)
if err != nil {
return fmt.Errorf("could not detect entrypoint for %v, use --entrypoint to add it explicitly. %v", *a.runtime, err)
}
a.entrypoint = &ep
}
return nil
}
func detectRuntime(path string) (runtime string, err error) {
for ext, runtime := range fileExtToRuntime {
fn := filepath.Join(path, fmt.Sprintf("func%s", ext))
if exists(fn) {
return runtime, nil
}
}
return "", fmt.Errorf("no supported files found to guess runtime, please set runtime explicitly with --runtime flag")
}
func detectEntrypoint(runtime string) (string, error) {
helper, err := langs.GetLangHelper(runtime)
if err != nil {
return "", err
}
return helper.Entrypoint(), nil
}

90
fn/install.sh Normal file
View File

@@ -0,0 +1,90 @@
#!/bin/sh
set -e
# Install script to install fn
release='0.1.15'
command_exists() {
command -v "$@" > /dev/null 2>&1
}
case "$(uname -m)" in
*64)
;;
*)
echo >&2 'Error: you are not using a 64bit platform.'
echo >&2 'Functions CLI currently only supports 64bit platforms.'
exit 1
;;
esac
if command_exists fn ; then
echo >&2 'Warning: "fn" command appears to already exist.'
echo >&2 'If you are just upgrading your functions cli client, ignore this and wait a few seconds.'
echo >&2 'You may press Ctrl+C now to abort this process.'
( set -x; sleep 5 )
fi
user="$(id -un 2>/dev/null || true)"
sh_c='sh -c'
if [ "$user" != 'root' ]; then
if command_exists sudo; then
sh_c='sudo -E sh -c'
elif command_exists su; then
sh_c='su -c'
else
echo >&2 'Error: this installer needs the ability to run commands as root.'
echo >&2 'We are unable to find either "sudo" or "su" available to make this happen.'
exit 1
fi
fi
curl=''
if command_exists curl; then
curl='curl -sSL -o'
elif command_exists wget; then
curl='wget -qO'
elif command_exists busybox && busybox --list-modules | grep -q wget; then
curl='busybox wget -qO'
else
echo >&2 'Error: this installer needs the ability to run wget or curl.'
echo >&2 'We are unable to find either "wget" or "curl" available to make this happen.'
exit 1
fi
url='https://github.com/iron-io/functions/releases/download'
# perform some very rudimentary platform detection
case "$(uname)" in
Linux)
$sh_c "$curl /usr/local/bin/fn $url/$release/fn_linux"
$sh_c "chmod +x /usr/local/bin/fn"
fn --version
exit 0
;;
Darwin)
$sh_c "$curl /usr/local/bin/fn $url/$release/fn_mac"
$sh_c "chmod +x /usr/local/bin/fn"
fn --version
exit 0
;;
WindowsNT)
$sh_c "$curl $url/$release/fn.exe"
# TODO how to make executable? chmod?
fn.exe --version
exit 0
;;
esac
cat >&2 <<'EOF'
Either your platform is not easily detectable or is not supported by this
installer script (yet - PRs welcome! [fn/install]).
Please visit the following URL for more detailed installation instructions:
https://github.com/iron-io/functions
EOF
exit 1

520
fn/lambda.go Normal file
View File

@@ -0,0 +1,520 @@
package main
import (
"archive/zip"
"bytes"
"errors"
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"path/filepath"
"strings"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
aws_lambda "github.com/aws/aws-sdk-go/service/lambda"
"github.com/docker/docker/pkg/jsonmessage"
lambdaImpl "github.com/iron-io/lambda/lambda"
"github.com/urfave/cli"
yaml "gopkg.in/yaml.v2"
)
func init() {
if len(runtimeCreateHandlers) != len(runtimeImportHandlers) {
panic("incomplete implementation of runtime support")
}
}
func lambda() cli.Command {
var flags []cli.Flag
flags = append(flags, getFlags()...)
return cli.Command{
Name: "lambda",
Usage: "create and publish lambda functions",
ArgsUsage: "fn lambda",
Subcommands: []cli.Command{
{
Name: "create-function",
Usage: `create Docker image that can run your Lambda function, where files are the contents of the zip file to be uploaded to AWS Lambda.`,
ArgsUsage: "name runtime handler /path [/paths...]",
Action: create,
Flags: flags,
},
{
Name: "test-function",
Usage: `runs local dockerized Lambda function and writes output to stdout.`,
ArgsUsage: "name [--payload <value>]",
Action: test,
Flags: flags,
},
{
Name: "aws-import",
Usage: `converts an existing Lambda function to an image, where the function code is downloaded to a directory in the current working directory that has the same name as the Lambda function.`,
ArgsUsage: "arn region image/name [--profile <aws profile>] [--version <version>] [--download-only]",
Action: awsImport,
Flags: flags,
},
},
}
}
func getFlags() []cli.Flag {
return []cli.Flag{
cli.StringFlag{
Name: "payload",
Usage: "Payload to pass to the Lambda function. This is usually a JSON object.",
Value: "{}",
},
cli.StringFlag{
Name: "version",
Usage: "Version of the function to import.",
Value: "$LATEST",
},
cli.BoolFlag{
Name: "download-only",
Usage: "Only download the function into a directory. Will not create a Docker image.",
},
cli.StringSliceFlag{
Name: "config",
Usage: "function configuration",
},
}
}
func transcribeEnvConfig(configs []string) map[string]string {
c := make(map[string]string)
for _, v := range configs {
kv := strings.SplitN(v, "=", 2)
if len(kv) == 1 {
// TODO: Make sure it is compatible cross platform
c[kv[0]] = fmt.Sprintf("$%s", kv[0])
} else {
c[kv[0]] = kv[1]
}
}
return c
}
func create(c *cli.Context) error {
args := c.Args()
if len(args) < 4 {
return fmt.Errorf("Expected at least 4 arguments, NAME RUNTIME HANDLER and file %d", len(args))
}
functionName := args[0]
runtime := args[1]
handler := args[2]
fileNames := args[3:]
files := make([]fileLike, 0, len(fileNames))
opts := createImageOptions{
Name: functionName,
Base: fmt.Sprintf("iron/lambda-%s", runtime),
Package: "",
Handler: handler,
OutputStream: newdockerJSONWriter(os.Stdout),
RawJSONStream: true,
Config: transcribeEnvConfig(c.StringSlice("config")),
}
if handler == "" {
return errors.New("No handler specified.")
}
rh, ok := runtimeCreateHandlers[runtime]
if !ok {
return fmt.Errorf("unsupported runtime %v", runtime)
}
if err := rh(fileNames, &opts); err != nil {
return err
}
for _, fileName := range fileNames {
file, err := os.Open(fileName)
if err != nil {
return err
}
defer file.Close()
files = append(files, file)
}
return createDockerfile(opts, files...)
}
var runtimeCreateHandlers = map[string]func(filenames []string, opts *createImageOptions) error{
"nodejs": func(filenames []string, opts *createImageOptions) error { return nil },
"python2.7": func(filenames []string, opts *createImageOptions) error { return nil },
"java8": func(filenames []string, opts *createImageOptions) error {
if len(filenames) != 1 {
return errors.New("Java Lambda functions can only include 1 file and it must be a JAR file.")
}
if filepath.Ext(filenames[0]) != ".jar" {
return errors.New("Java Lambda function package must be a JAR file.")
}
opts.Package = filepath.Base(filenames[0])
return nil
},
}
func test(c *cli.Context) error {
args := c.Args()
if len(args) < 1 {
return fmt.Errorf("Missing NAME argument")
}
functionName := args[0]
exists, err := lambdaImpl.ImageExists(functionName)
if err != nil {
return err
}
if !exists {
return fmt.Errorf("Function %s does not exist.", functionName)
}
payload := c.String("payload")
// Redirect output to stdout.
return lambdaImpl.RunImageWithPayload(functionName, payload)
}
func awsImport(c *cli.Context) error {
args := c.Args()
if len(args) < 3 {
return fmt.Errorf("Missing arguments ARN, REGION and/or IMAGE")
}
version := c.String("version")
downloadOnly := c.Bool("download-only")
profile := c.String("profile")
arn := args[0]
region := args[1]
image := args[2]
function, err := getFunction(profile, region, version, arn)
if err != nil {
return err
}
functionName := *function.Configuration.FunctionName
err = os.Mkdir(fmt.Sprintf("./%s", functionName), os.ModePerm)
if err != nil {
return err
}
tmpFileName, err := downloadToFile(*function.Code.Location)
if err != nil {
return err
}
defer os.Remove(tmpFileName)
if downloadOnly {
// Since we are a command line program that will quit soon, it is OK to
// let the OS clean `files` up.
return err
}
opts := createImageOptions{
Name: functionName,
Base: fmt.Sprintf("iron/lambda-%s", *function.Configuration.Runtime),
Package: "",
Handler: *function.Configuration.Handler,
OutputStream: newdockerJSONWriter(os.Stdout),
RawJSONStream: true,
Config: transcribeEnvConfig(c.StringSlice("config")),
}
runtime := *function.Configuration.Runtime
rh, ok := runtimeImportHandlers[runtime]
if !ok {
return fmt.Errorf("unsupported runtime %v", runtime)
}
files, err := rh(functionName, tmpFileName, &opts)
if err != nil {
return nil
}
if image != "" {
opts.Name = image
}
return createDockerfile(opts, files...)
}
var (
runtimeImportHandlers = map[string]func(functionName, tmpFileName string, opts *createImageOptions) ([]fileLike, error){
"nodejs": basicImportHandler,
"python2.7": basicImportHandler,
"java8": func(functionName, tmpFileName string, opts *createImageOptions) ([]fileLike, error) {
fmt.Println("Found Java Lambda function. Going to assume code is a single JAR file.")
path := filepath.Join(functionName, "function.jar")
if err := os.Rename(tmpFileName, path); err != nil {
return nil, err
}
fd, err := os.Open(path)
if err != nil {
return nil, err
}
files := []fileLike{fd}
opts.Package = filepath.Base(files[0].(*os.File).Name())
return files, nil
},
}
)
func basicImportHandler(functionName, tmpFileName string, opts *createImageOptions) ([]fileLike, error) {
return unzipAndGetTopLevelFiles(functionName, tmpFileName)
}
func createFunctionYaml(opts createImageOptions) error {
strs := strings.Split(opts.Name, "/")
route := fmt.Sprintf("/%s", strs[1])
funcDesc := &funcfile{
App: &strs[0],
Name: opts.Name,
Route: &route,
Config: opts.Config,
}
out, err := yaml.Marshal(funcDesc)
if err != nil {
return err
}
return ioutil.WriteFile(filepath.Join(opts.Name, "func.yaml"), out, 0644)
}
type createImageOptions struct {
Name string
Base string
Package string // Used for Java, empty string for others.
Handler string
OutputStream io.Writer
RawJSONStream bool
Config map[string]string
}
type fileLike interface {
io.Reader
Stat() (os.FileInfo, error)
}
var errNoFiles = errors.New("No files to add to image")
// Create a Dockerfile that adds each of the files to the base image. The
// expectation is that the base image sets up the current working directory
// inside the image correctly. `handler` is set to be passed to node-lambda
// for now, but we may have to change this to accomodate other stacks.
func makeDockerfile(base string, pkg string, handler string, files ...fileLike) ([]byte, error) {
var buf bytes.Buffer
fmt.Fprintf(&buf, "FROM %s\n", base)
for _, file := range files {
// FIXME(nikhil): Validate path, no parent paths etc.
info, err := file.Stat()
if err != nil {
return buf.Bytes(), err
}
fmt.Fprintf(&buf, "ADD [\"%s\", \"./%s\"]\n", info.Name(), info.Name())
}
fmt.Fprint(&buf, "CMD [")
if pkg != "" {
fmt.Fprintf(&buf, "\"%s\", ", pkg)
}
// FIXME(nikhil): Validate handler.
fmt.Fprintf(&buf, `"%s"`, handler)
fmt.Fprint(&buf, "]\n")
return buf.Bytes(), nil
}
// Creates a docker image called `name`, using `base` as the base image.
// `handler` is the runtime-specific name to use for a lambda invocation (i.e.
// <module>.<function> for nodejs). `files` should be a list of files+dirs
// *relative to the current directory* that are to be included in the image.
func createDockerfile(opts createImageOptions, files ...fileLike) error {
if len(files) == 0 {
return errNoFiles
}
df, err := makeDockerfile(opts.Base, opts.Package, opts.Handler, files...)
if err != nil {
return err
}
fmt.Printf("Creating directory: %s ... ", opts.Name)
if err := os.MkdirAll(opts.Name, os.ModePerm); err != nil {
return err
}
fmt.Println("OK")
fmt.Printf("Creating Dockerfile: %s ... ", filepath.Join(opts.Name, "Dockerfile"))
outputFile, err := os.Create(filepath.Join(opts.Name, "Dockerfile"))
if err != nil {
return err
}
fmt.Println("OK")
for _, f := range files {
fstat, err := f.Stat()
if err != nil {
return err
}
fmt.Printf("Copying file: %s ... ", filepath.Join(opts.Name, fstat.Name()))
src, err := os.Create(filepath.Join(opts.Name, fstat.Name()))
if err != nil {
return err
}
if _, err := io.Copy(src, f); err != nil {
return err
}
fmt.Println("OK")
}
if _, err = outputFile.Write(df); err != nil {
return err
}
fmt.Print("Creating func.yaml ... ")
if err := createFunctionYaml(opts); err != nil {
return err
}
fmt.Println("OK")
return nil
}
type dockerJSONWriter struct {
under io.Writer
w io.Writer
}
func newdockerJSONWriter(under io.Writer) *dockerJSONWriter {
r, w := io.Pipe()
go func() {
err := jsonmessage.DisplayJSONMessagesStream(r, under, 1, true, nil)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}()
return &dockerJSONWriter{under, w}
}
func (djw *dockerJSONWriter) Write(p []byte) (int, error) {
return djw.w.Write(p)
}
func downloadToFile(url string) (string, error) {
downloadResp, err := http.Get(url)
if err != nil {
return "", err
}
defer downloadResp.Body.Close()
// zip reader needs ReaderAt, hence the indirection.
tmpFile, err := ioutil.TempFile("", "lambda-function-")
if err != nil {
return "", err
}
if _, err := io.Copy(tmpFile, downloadResp.Body); err != nil {
return "", err
}
if err := tmpFile.Close(); err != nil {
return "", err
}
return tmpFile.Name(), nil
}
func unzipAndGetTopLevelFiles(dst, src string) (files []fileLike, topErr error) {
files = make([]fileLike, 0)
zipReader, err := zip.OpenReader(src)
if err != nil {
return files, err
}
defer zipReader.Close()
var fd *os.File
for _, f := range zipReader.File {
path := filepath.Join(dst, f.Name)
fmt.Printf("Extracting '%s' to '%s'\n", f.Name, path)
if f.FileInfo().IsDir() {
if err := os.Mkdir(path, 0644); err != nil {
return nil, err
}
// Only top-level dirs go into the list since that is what CreateImage expects.
if filepath.Dir(f.Name) == filepath.Base(f.Name) {
fd, topErr = os.Open(path)
if topErr != nil {
break
}
files = append(files, fd)
}
} else {
// We do not close fd here since we may want to use it to dockerize.
fd, topErr = os.OpenFile(path, os.O_RDWR|os.O_CREATE, 0644)
if topErr != nil {
break
}
var zipFd io.ReadCloser
zipFd, topErr = f.Open()
if topErr != nil {
break
}
if _, topErr = io.Copy(fd, zipFd); topErr != nil {
// OK to skip closing fd here.
break
}
if err := zipFd.Close(); err != nil {
return nil, err
}
// Only top-level files go into the list since that is what CreateImage expects.
if filepath.Dir(f.Name) == "." {
if _, topErr = fd.Seek(0, 0); topErr != nil {
break
}
files = append(files, fd)
} else {
if err := fd.Close(); err != nil {
return nil, err
}
}
}
}
return
}
func getFunction(awsProfile, awsRegion, version, arn string) (*aws_lambda.GetFunctionOutput, error) {
creds := credentials.NewChainCredentials([]credentials.Provider{
&credentials.EnvProvider{},
&credentials.SharedCredentialsProvider{
Filename: "", // Look in default location.
Profile: awsProfile,
},
})
conf := aws.NewConfig().WithCredentials(creds).WithCredentialsChainVerboseErrors(true).WithRegion(awsRegion)
sess := session.New(conf)
conn := aws_lambda.New(sess)
resp, err := conn.GetFunction(&aws_lambda.GetFunctionInput{
FunctionName: aws.String(arn),
Qualifier: aws.String(version),
})
return resp, err
}

23
fn/langs/base.go Normal file
View File

@@ -0,0 +1,23 @@
package langs
import "fmt"
// GetLangHelper returns a LangHelper for the passed in language
func GetLangHelper(lang string) (LangHelper, error) {
switch lang {
case "go":
return &GoLangHelper{}, nil
case "node":
return &NodeLangHelper{}, nil
case "ruby":
return &RubyLangHelper{}, nil
}
return nil, fmt.Errorf("No language helper found for %v", lang)
}
type LangHelper interface {
Entrypoint() string
HasPreBuild() bool
PreBuild() error
AfterBuild() error
}

46
fn/langs/go.go Normal file
View File

@@ -0,0 +1,46 @@
package langs
import (
"fmt"
"os"
"os/exec"
"strings"
)
type GoLangHelper struct {
}
func (lh *GoLangHelper) Entrypoint() string {
return "./func"
}
func (lh *GoLangHelper) HasPreBuild() bool {
return true
}
// PreBuild for Go builds the binary so the final image can be as small as possible
func (lh *GoLangHelper) PreBuild() error {
wd, err := os.Getwd()
if err != nil {
return err
}
// todo: this won't work if the function is more complex since the import paths won't match up, need to fix
pbcmd := fmt.Sprintf("docker run --rm -v %s:/go/src/github.com/x/y -w /go/src/github.com/x/y iron/go:dev go build -o func", wd)
fmt.Println("Running prebuild command:", pbcmd)
parts := strings.Fields(pbcmd)
head := parts[0]
parts = parts[1:len(parts)]
cmd := exec.Command(head, parts...)
// cmd.Dir = dir
cmd.Stderr = os.Stderr
cmd.Stdout = os.Stdout
if err := cmd.Run(); err != nil {
return fmt.Errorf("error running docker build: %v", err)
}
return nil
}
func (lh *GoLangHelper) AfterBuild() error {
return os.Remove("func")
}

21
fn/langs/node.go Normal file
View File

@@ -0,0 +1,21 @@
package langs
type NodeLangHelper struct {
}
func (lh *NodeLangHelper) Entrypoint() string {
return "node func.js"
}
func (lh *NodeLangHelper) HasPreBuild() bool {
return false
}
// PreBuild for Go builds the binary so the final image can be as small as possible
func (lh *NodeLangHelper) PreBuild() error {
return nil
}
func (lh *NodeLangHelper) AfterBuild() error {
return nil
}

21
fn/langs/ruby.go Normal file
View File

@@ -0,0 +1,21 @@
package langs
type RubyLangHelper struct {
}
func (lh *RubyLangHelper) Entrypoint() string {
return "ruby func.rb"
}
func (lh *RubyLangHelper) HasPreBuild() bool {
return false
}
// PreBuild for Go builds the binary so the final image can be as small as possible
func (lh *RubyLangHelper) PreBuild() error {
return nil
}
func (lh *RubyLangHelper) AfterBuild() error {
return nil
}

52
fn/main.go Normal file
View File

@@ -0,0 +1,52 @@
package main
import (
"fmt"
"net/url"
"os"
functions "github.com/iron-io/functions_go"
"github.com/urfave/cli"
)
func main() {
app := cli.NewApp()
app.Name = "fn"
app.Version = "0.1.0"
app.Authors = []cli.Author{{Name: "iron.io"}}
app.Usage = "IronFunctions command line tools"
app.UsageText = `Check the manual at https://github.com/iron-io/functions/blob/master/fn/README.md
ENVIRONMENT VARIABLES:
API_URL - IronFunctions remote API address`
app.CommandNotFound = func(c *cli.Context, cmd string) { fmt.Fprintf(os.Stderr, "command not found: %v\n", cmd) }
app.Commands = []cli.Command{
apps(),
build(),
bump(),
call(),
lambda(),
publish(),
push(),
routes(),
run(),
initFn(),
}
app.Run(os.Args)
}
func resetBasePath(c *functions.Configuration) error {
apiURL := os.Getenv("API_URL")
if apiURL == "" {
apiURL = "http://localhost:8080/"
}
u, err := url.Parse(apiURL)
if err != nil {
return err
}
u.Path = "/v1"
c.BasePath = u.String()
return nil
}

159
fn/publish.go Normal file
View File

@@ -0,0 +1,159 @@
package main
import (
"fmt"
"net/http"
"os"
"os/exec"
"path/filepath"
functions "github.com/iron-io/functions_go"
"github.com/urfave/cli"
)
func publish() cli.Command {
cmd := publishcmd{
commoncmd: &commoncmd{},
RoutesApi: functions.NewRoutesApi(),
}
var flags []cli.Flag
flags = append(flags, cmd.flags()...)
flags = append(flags, cmd.commoncmd.flags()...)
return cli.Command{
Name: "publish",
Usage: "scan local directory for functions, build and publish them.",
Flags: flags,
Action: cmd.scan,
}
}
type publishcmd struct {
*commoncmd
*functions.RoutesApi
skippush bool
}
func (p *publishcmd) flags() []cli.Flag {
return []cli.Flag{
cli.BoolFlag{
Name: "skip-push",
Usage: "does not push Docker built images onto Docker Hub - useful for local development.",
Destination: &p.skippush,
},
}
}
func (p *publishcmd) scan(c *cli.Context) error {
p.commoncmd.scan(p.walker)
return nil
}
func (p *publishcmd) walker(path string, info os.FileInfo, err error) error {
walker(path, info, err, p.publish)
return nil
}
// publish will take the found function and check for the presence of a
// Dockerfile, and run a three step process: parse functions file, build and
// push the container, and finally it will update function's route. Optionally,
// the route can be overriden inside the functions file.
func (p *publishcmd) publish(path string) error {
fmt.Fprintln(p.verbwriter, "publishing", path)
funcfile, err := p.buildfunc(path)
if err != nil {
return err
}
if p.skippush {
return nil
}
if err := p.dockerpush(funcfile); err != nil {
return err
}
return p.route(path, funcfile)
}
func (p publishcmd) dockerpush(ff *funcfile) error {
cmd := exec.Command("docker", "push", ff.FullName())
cmd.Stderr = os.Stderr
cmd.Stdout = os.Stdout
if err := cmd.Run(); err != nil {
return fmt.Errorf("error running docker push: %v", err)
}
return nil
}
func (p *publishcmd) route(path string, ff *funcfile) error {
if err := resetBasePath(&p.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
an, r := extractAppNameRoute(path)
if ff.App == nil {
ff.App = &an
}
if ff.Route == nil {
ff.Route = &r
}
if ff.Memory == nil {
ff.Memory = new(int64)
}
if ff.Type == nil {
ff.Type = new(string)
}
body := functions.RouteWrapper{
Route: functions.Route{
Path: *ff.Route,
Image: ff.FullName(),
Memory: *ff.Memory,
Type_: *ff.Type,
Config: expandEnvConfig(ff.Config),
},
}
fmt.Fprintf(p.verbwriter, "updating API with app: %s route: %s name: %s \n", *ff.App, *ff.Route, ff.Name)
wrapper, resp, err := p.AppsAppRoutesPost(*ff.App, body)
if err != nil {
return fmt.Errorf("error getting routes: %v", err)
}
if resp.StatusCode == http.StatusBadRequest {
return fmt.Errorf("error storing this route: %s", wrapper.Error_.Message)
}
return nil
}
func expandEnvConfig(configs map[string]string) map[string]string {
for k, v := range configs {
configs[k] = os.ExpandEnv(v)
}
return configs
}
func extractAppNameRoute(path string) (appName, route string) {
// The idea here is to extract the root-most directory name
// as application name, it turns out that stdlib tools are great to
// extract the deepest one. Thus, we revert the string and use the
// stdlib as it is - and revert back to its normal content. Not fastest
// ever, but it is simple.
rpath := reverse(path)
rroute, rappName := filepath.Split(rpath)
route = filepath.Dir(reverse(rroute))
return reverse(rappName), route
}
func reverse(s string) string {
r := []rune(s)
for i, j := 0, len(r)-1; i < len(r)/2; i, j = i+1, j-1 {
r[i], r[j] = r[j], r[i]
}
return string(r)
}

60
fn/push.go Normal file
View File

@@ -0,0 +1,60 @@
package main
import (
"fmt"
"os"
functions "github.com/iron-io/functions_go"
"github.com/urfave/cli"
)
func push() cli.Command {
cmd := pushcmd{
publishcmd: &publishcmd{
commoncmd: &commoncmd{},
RoutesApi: functions.NewRoutesApi(),
},
}
var flags []cli.Flag
flags = append(flags, cmd.commoncmd.flags()...)
return cli.Command{
Name: "push",
Usage: "push function to Docker Hub",
Flags: flags,
Action: cmd.scan,
}
}
type pushcmd struct {
*publishcmd
}
func (p *pushcmd) scan(c *cli.Context) error {
p.commoncmd.scan(p.walker)
return nil
}
func (p *pushcmd) walker(path string, info os.FileInfo, err error) error {
walker(path, info, err, p.push)
return nil
}
// push will take the found function and check for the presence of a
// Dockerfile, and run a three step process: parse functions file,
// push the container, and finally it will update function's route. Optionally,
// the route can be overriden inside the functions file.
func (p *pushcmd) push(path string) error {
fmt.Fprintln(p.verbwriter, "pushing", path)
funcfile, err := parsefuncfile(path)
if err != nil {
return err
}
if err := p.dockerpush(funcfile); err != nil {
return err
}
fmt.Printf("Function %v pushed successfully to Docker Hub.\n", funcfile.FullName())
return nil
}

393
fn/routes.go Normal file
View File

@@ -0,0 +1,393 @@
package main
import (
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"path"
"strings"
"text/tabwriter"
functions "github.com/iron-io/functions_go"
"github.com/urfave/cli"
"golang.org/x/crypto/ssh/terminal"
)
type routesCmd struct {
*functions.RoutesApi
}
func routes() cli.Command {
r := routesCmd{RoutesApi: functions.NewRoutesApi()}
return cli.Command{
Name: "routes",
Usage: "list routes",
ArgsUsage: "fn routes",
Action: r.list,
Subcommands: []cli.Command{
{
Name: "call",
Usage: "call a route",
ArgsUsage: "appName /path",
Action: r.call,
Flags: runflags(),
},
{
Name: "create",
Usage: "create a route",
ArgsUsage: "appName /path image/name",
Action: r.create,
Flags: []cli.Flag{
cli.Int64Flag{
Name: "memory",
Usage: "memory in MiB",
Value: 128,
},
cli.StringFlag{
Name: "type",
Usage: "route type - sync or async",
Value: "sync",
},
cli.StringSliceFlag{
Name: "config",
Usage: "route configuration",
},
},
},
{
Name: "delete",
Usage: "delete a route",
ArgsUsage: "appName /path",
Action: r.delete,
},
{
Name: "config",
Usage: "operate a route configuration set",
Action: r.configList,
Flags: []cli.Flag{
cli.BoolFlag{
Name: "shell",
Usage: "output in shell format",
},
cli.BoolFlag{
Name: "json",
Usage: "output in JSON format",
},
},
Subcommands: []cli.Command{
{
Name: "set",
Description: "store a configuration key for this route",
Usage: "<app> <key> <value>",
Action: r.configSet,
},
{
Name: "unset",
Description: "remove a configuration key for this route",
Usage: "<app> <key> <value>",
Action: r.configUnset,
},
},
},
},
}
}
func call() cli.Command {
r := routesCmd{RoutesApi: functions.NewRoutesApi()}
return cli.Command{
Name: "call",
Usage: "call a remote function",
ArgsUsage: "appName /path",
Flags: runflags(),
Action: r.call,
}
}
func (a *routesCmd) list(c *cli.Context) error {
if c.Args().First() == "" {
return errors.New("error: routes listing takes one argument, an app name")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
appName := c.Args().Get(0)
wrapper, _, err := a.AppsAppRoutesGet(appName)
if err != nil {
return fmt.Errorf("error getting routes: %v", err)
}
baseURL, err := url.Parse(a.Configuration.BasePath)
if err != nil {
return fmt.Errorf("error parsing base path: %v", err)
}
w := tabwriter.NewWriter(os.Stdout, 0, 8, 0, '\t', 0)
fmt.Fprint(w, "path", "\t", "image", "\t", "endpoint", "\n")
for _, route := range wrapper.Routes {
u, err := url.Parse("../")
u.Path = path.Join(u.Path, "r", appName, route.Path)
if err != nil {
return fmt.Errorf("error parsing functions route path: %v", err)
}
fmt.Fprint(w, route.Path, "\t", route.Image, "\t", baseURL.ResolveReference(u).String(), "\n")
}
w.Flush()
return nil
}
func (a *routesCmd) call(c *cli.Context) error {
if c.Args().Get(0) == "" || c.Args().Get(1) == "" {
return errors.New("error: routes listing takes three arguments: an app name and a route")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
baseURL, err := url.Parse(a.Configuration.BasePath)
if err != nil {
return fmt.Errorf("error parsing base path: %v", err)
}
appName := c.Args().Get(0)
route := c.Args().Get(1)
u, err := url.Parse("../")
u.Path = path.Join(u.Path, "r", appName, route)
var content io.Reader
if !terminal.IsTerminal(int(os.Stdin.Fd())) {
content = os.Stdin
}
req, err := http.NewRequest("POST", baseURL.ResolveReference(u).String(), content)
if err != nil {
return fmt.Errorf("error running route: %v", err)
}
req.Header.Set("Content-Type", "application/json")
envAsHeader(req, c.StringSlice("e"))
resp, err := http.DefaultClient.Do(req)
if err != nil {
return fmt.Errorf("error running route: %v", err)
}
io.Copy(os.Stdout, resp.Body)
return nil
}
func envAsHeader(req *http.Request, selectedEnv []string) {
detectedEnv := os.Environ()
if len(selectedEnv) > 0 {
detectedEnv = selectedEnv
}
for _, e := range detectedEnv {
kv := strings.Split(e, "=")
name := kv[0]
req.Header.Set(name, os.Getenv(name))
}
}
func (a *routesCmd) create(c *cli.Context) error {
if c.Args().Get(0) == "" || c.Args().Get(1) == "" {
return errors.New("error: routes creation takes three arguments: an app name, a route path and an image")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
appName := c.Args().Get(0)
route := c.Args().Get(1)
image := c.Args().Get(2)
if image == "" {
ff, err := findFuncfile()
if err != nil {
if _, ok := err.(*notFoundError); ok {
return errors.New("error: image name is missing or no function file found")
} else {
return err
}
}
image = ff.FullName()
}
body := functions.RouteWrapper{
Route: functions.Route{
AppName: appName,
Path: route,
Image: image,
Memory: c.Int64("memory"),
Type_: c.String("type"),
Config: extractEnvConfig(c.StringSlice("config")),
},
}
wrapper, _, err := a.AppsAppRoutesPost(appName, body)
if err != nil {
return fmt.Errorf("error creating route: %v", err)
}
if wrapper.Route.Path == "" || wrapper.Route.Image == "" {
return fmt.Errorf("could not create this route (%s at %s), check if route path is correct", route, appName)
}
fmt.Println(wrapper.Route.Path, "created with", wrapper.Route.Image)
return nil
}
func (a *routesCmd) delete(c *cli.Context) error {
if c.Args().Get(0) == "" || c.Args().Get(1) == "" {
return errors.New("error: routes listing takes three arguments: an app name and a path")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
appName := c.Args().Get(0)
route := c.Args().Get(1)
_, err := a.AppsAppRoutesRouteDelete(appName, route)
if err != nil {
return fmt.Errorf("error deleting route: %v", err)
}
fmt.Println(route, "deleted")
return nil
}
func (a *routesCmd) configList(c *cli.Context) error {
if c.Args().Get(0) == "" || c.Args().Get(1) == "" {
return errors.New("error: route configuration description takes two arguments: an app name and a route")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
appName := c.Args().Get(0)
route := c.Args().Get(1)
wrapper, _, err := a.AppsAppRoutesRouteGet(appName, route)
if err != nil {
return fmt.Errorf("error loading route information: %v", err)
}
config := wrapper.Route.Config
if len(config) == 0 {
return errors.New("this route has no configurations")
}
if c.Bool("json") {
if err := json.NewEncoder(os.Stdout).Encode(config); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
} else if c.Bool("shell") {
for k, v := range config {
fmt.Print("export ", k, "=", v, "\n")
}
} else {
fmt.Println(wrapper.Route.AppName, wrapper.Route.Path, "configuration:")
w := tabwriter.NewWriter(os.Stdout, 0, 8, 1, ' ', 0)
for k, v := range config {
fmt.Fprint(w, k, ":\t", v, "\n")
}
w.Flush()
}
return nil
}
func (a *routesCmd) configSet(c *cli.Context) error {
if c.Args().Get(0) == "" || c.Args().Get(1) == "" || c.Args().Get(2) == "" {
return errors.New("error: route configuration setting takes four arguments: an app name, a route, a key and a value")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
appName := c.Args().Get(0)
route := c.Args().Get(1)
key := c.Args().Get(2)
value := c.Args().Get(3)
wrapper, _, err := a.AppsAppRoutesRouteGet(appName, route)
if err != nil {
return fmt.Errorf("error creating app: %v", err)
}
config := wrapper.Route.Config
if config == nil {
config = make(map[string]string)
}
config[key] = value
wrapper.Route.Config = config
if _, err := a.AppsAppRoutesRouteDelete(appName, route); err != nil {
return fmt.Errorf("error deleting to force update route: %v", err)
}
if _, _, err := a.AppsAppRoutesPost(appName, *wrapper); err != nil {
return fmt.Errorf("error updating route configuration: %v", err)
}
fmt.Println(wrapper.Route.AppName, wrapper.Route.Path, "updated", key, "with", value)
return nil
}
func (a *routesCmd) configUnset(c *cli.Context) error {
if c.Args().Get(0) == "" || c.Args().Get(1) == "" || c.Args().Get(2) == "" {
return errors.New("error: route configuration setting takes four arguments: an app name, a route and a key")
}
if err := resetBasePath(&a.Configuration); err != nil {
return fmt.Errorf("error setting endpoint: %v", err)
}
appName := c.Args().Get(0)
route := c.Args().Get(1)
key := c.Args().Get(2)
wrapper, _, err := a.AppsAppRoutesRouteGet(appName, route)
if err != nil {
return fmt.Errorf("error creating app: %v", err)
}
config := wrapper.Route.Config
if config == nil {
config = make(map[string]string)
}
if _, ok := config[key]; !ok {
return fmt.Errorf("configuration key %s not found", key)
}
delete(config, key)
wrapper.Route.Config = config
if _, err := a.AppsAppRoutesRouteDelete(appName, route); err != nil {
return fmt.Errorf("error deleting to force update route: %v", err)
}
if _, _, err := a.AppsAppRoutesPost(appName, *wrapper); err != nil {
return fmt.Errorf("error updating route configuration: %v", err)
}
fmt.Println(wrapper.Route.AppName, wrapper.Route.Path, "removed", key)
return nil
}

25
fn/routes_test.go Normal file
View File

@@ -0,0 +1,25 @@
package main
import (
"net/http"
"os"
"testing"
)
func TestEnvAsHeader(t *testing.T) {
const expectedValue = "v=v"
os.Setenv("k", expectedValue)
cases := [][]string{
nil,
[]string{},
[]string{"k"},
}
for _, selectedEnv := range cases {
req, _ := http.NewRequest("GET", "http://www.example.com", nil)
envAsHeader(req, selectedEnv)
if found := req.Header.Get("k"); found != expectedValue {
t.Errorf("not found expected header: %v", found)
}
}
}

106
fn/run.go Normal file
View File

@@ -0,0 +1,106 @@
package main
import (
"errors"
"fmt"
"os"
"os/exec"
"strings"
"github.com/urfave/cli"
)
func run() cli.Command {
r := runCmd{}
return cli.Command{
Name: "run",
Usage: "run a function locally",
ArgsUsage: "USERNAME/image:tag",
Flags: append(runflags(), []cli.Flag{}...),
Action: r.run,
}
}
type runCmd struct{}
func runflags() []cli.Flag {
return []cli.Flag{
cli.StringSliceFlag{
Name: "e",
Usage: "limit the environment variables sent to function, if ommited then all are sent.",
},
}
}
func (r *runCmd) run(c *cli.Context) error {
image := c.Args().First()
if image == "" {
ff, err := findFuncfile()
if err != nil {
if _, ok := err.(*notFoundError); ok {
return errors.New("error: image name is missing or no function file found")
}
return err
}
image = ff.FullName()
}
sh := []string{"docker", "run", "--rm", "-i"}
var env []string
detectedEnv := os.Environ()
if se := c.StringSlice("e"); len(se) > 0 {
detectedEnv = se
}
for _, e := range detectedEnv {
shellvar, envvar := extractEnvVar(e)
sh = append(sh, shellvar...)
env = append(env, envvar)
}
dockerenv := []string{"DOCKER_TLS_VERIFY", "DOCKER_HOST", "DOCKER_CERT_PATH", "DOCKER_MACHINE_NAME"}
for _, e := range dockerenv {
env = append(env, fmt.Sprint(e, "=", os.Getenv(e)))
}
sh = append(sh, image)
cmd := exec.Command(sh[0], sh[1:]...)
// Check if stdin is being piped, and if not, create our own pipe with nothing in it
// http://stackoverflow.com/questions/22744443/check-if-there-is-something-to-read-on-stdin-in-golang
stat, err := os.Stdin.Stat()
if err != nil {
// On Windows, this gets an error if nothing is piped in.
// If something is piped in, it works fine.
// Turns out, this works just fine in our case as the piped stuff works properly and the non-piped doesn't hang either.
// See: https://github.com/golang/go/issues/14853#issuecomment-260170423
// log.Println("Warning: couldn't stat stdin, you are probably on Windows. Be sure to pipe something into this command, eg: 'echo \"hello\" | fn run'")
} else {
if (stat.Mode() & os.ModeCharDevice) == 0 {
// log.Println("data is being piped to stdin")
cmd.Stdin = os.Stdin
} else {
// log.Println("stdin is from a terminal")
cmd.Stdin = strings.NewReader("")
}
}
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Env = env
return cmd.Run()
}
func extractEnvVar(e string) ([]string, string) {
kv := strings.Split(e, "=")
name := toEnvName("HEADER", kv[0])
sh := []string{"-e", name}
env := fmt.Sprintf("%s=%s", name, os.Getenv(kv[0]))
return sh, env
}
// From server.toEnvName()
func toEnvName(envtype, name string) string {
name = strings.ToUpper(strings.Replace(name, "-", "_", -1))
return fmt.Sprintf("%s_%s", envtype, name)
}