More doc updates, explanation of serverless. (#228)

* More doc updates, explanation of serverless.

* Moved howto directory to examples and some minor doc updates.

* Added apps and routes docs.

* Fixes for Carlos' comments.

* Added bit about importing lambda functions.

CONTRIBUTING.md

@@ -1,36 +1,45 @@
-## Contributing
+# Contributing to IronFunctions
+
+We welcome all contributions! 
+
+## How to contribute
+
+* Fork the repo
+* Fix an issue or create an issue and fix it
+* Create a Pull Request that fixes the issue
+* Sign the CLA
+* Good Job! Thanks for being awesome!
+
+## How to build and get up and running
 
 ### Build
 
-```
+The first time after you fork or after dependencies get updated, run:
-make all
+
+```sh
+make dep
 ```
 
-### Run Functions service
+Then after every change, run:
 
-First let's start our IronFunctions API
+```sh
-
+make build
-##### Run in Docker
-
-```
-make run-docker
 ```
 
-will start Functions using an embedded `Bolt` database running on `:8080`. 
+to build the `functions` binary.
 
-##### Running on Metal (recommended only on Linux)
+### Run
 
 ```
 ./functions
 ```
 
-will start Functions with a default of 1 async runner
+will start IronFunctions using an embedded `Bolt` database running on port `8080`. 
 
-### Contributing
+#### Run in Docker
 
-##### Code
+```
-* Fork the repo
+make run-docker
-* Fix an issue
+```
-* Create a Pull Request
+
-* Sign the CLA
+will start IronFunctions inside a Docker container.
-* Good Job! Thanks for being awesome!

README.md

@@ -1,12 +1,63 @@
 # IronFunctions
 
-Welcome to IronFunctions!  The open source Functions as a Service platform.
+[![CircleCI](https://circleci.com/gh/iron-io/functions.svg?style=svg)](https://circleci.com/gh/iron-io/functions) 
+[![GoDoc](https://godoc.org/github.com/iron-io/functions?status.svg)](https://godoc.org/github.com/iron-io/functions)
+
+Welcome to IronFunctions!  The open source serverless platform.
+
+## What is IronFunctions?
+
+IronFunctions is an open source serverless platform, or as we like to refer to it, Functions as a 
+Service -- FaaS -- platform that you can run anywhere. 
+
+* [Run anywhere](docs/faq.md#where-can-run-ironfunctions)
+  * Public cloud, hybrid, on-premise
+  * [Import Lambda functions](docs/lambda/import.md) from AWS and run them wherever you want
+* [Any language](docs/faq.md#which-languages-are-supported)
+  * [AWS Lambda support](docs/lambda/README.md)
+* Easy to use
+* Easy to scale
+
+## What is Serverless/FaaS?
+
+Serverless is a new paradigm in computing that enables simplicity, efficiency and scalability for both developers 
+and operators. It's important to distinguish the two, because the benefits differ:
+
+### Benefits for developers
+
+The main benefits that most people refer to are on the developer side and they include:
+
+* No servers to manage (serverless) -- you just upload your code and the platform deals with the infrastructure
+* Super simple coding -- no more monoliths! Just simple little bits of code
+* Pay by the milliseconds your code is executing -- unlike a typical application that runs 24/7, and you're paying
+  24/7, functions only run when needed
+
+Since you'll be running IronFunctions yourself, the paying part may not apply, but it does apply to 
+cost savings on your infrastructure bills as you'll read below. 
+
+### Benefits for operators
+
+If you will be operating IronFunctions (the person who has to manage the servers behind the serverless),
+then the benefits are different, but related.
+
+* Extremely efficient use of resources
+  * Unlike an app/API/microservice that consumes resources 24/7 whether they 
+    are in use or not, functions are time sliced across your infrastructure and only consume resources while they are
+    actually doing something
+* Easy to manage and scale
+  * Single system for code written in any language or any technology
+  * Single system to monitor
+  * Scaling is the same for all functions, you don't scale each app independently
+  * Scaling is simply adding more IronFunctions nodes
+
+There is a lot more reading you can do on the topic, just search for ["what is serverless"](https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=what%20is%20serverless) 
+and you'll find plenty of information. We have pretty thorough post on the Iron.io blog called [What is Serverless Computing and Why is it Important].
 
 ## Join Our Community
 
 First off, join the community!
 
-[![Slack Status](https://open-iron.herokuapp.com/badge.svg)](https://open-iron.herokuapp.com)
+[![Slack Status](https://open-iron.herokuapp.com/badge.svg)](http://get.iron.io/open-slack)
 
 ## Quickstart
 
@@ -20,23 +71,14 @@ To get started quickly with IronFunctions, you can just fire up an `iron/functio
 docker run --rm -it --name functions --privileged -v $PWD/data:/app/data -p 8080:8080 iron/functions
 ```
 
-**Note**: A list of configurations via env variables can be found [here](docs/options.md).*
+**Note**: A list of configurations via env variables can be found [here](docs/options.md).
 
 ### CLI tool
 
-You can easily operate IronFunctions with its CLI tool. Install it with:
+The IronFunctions CLI tool is optional, but it makes things easier. Install it with:
 
 ```sh
-curl -sSL https://fn.iron.io/install | sh
+curl -sSL http://get.iron.io/fnctl | sh
-```
-
-If you're concerned about the [potential insecurity](http://curlpipesh.tumblr.com/)
-of using `curl | sh`, feel free to use a two-step version of our installation and examine our
-installation script:
-
-```bash
-curl -f -sSL https://fn.iron.io/install -O
-sh install
 ```
 
 ### Create an Application
@@ -47,25 +89,31 @@ An application is essentially a grouping of functions, that put together, form a
 fnctl apps create myapp
 ```
 
-Or using a cURL call:
+Or using a cURL:
+
 ```sh
 curl -H "Content-Type: application/json" -X POST -d '{
     "app": { "name":"myapp" }
 }' http://localhost:8080/v1/apps
 ```
 
+[More on apps](docs/apps.md).
+
 Now that we have an app, we can map routes to functions.
 
 ### Add a Route
 
 A route is a way to define a path in your application that maps to a function. In this example, we'll map
-`/path` to a simple `Hello World!` image called `iron/hello`.
+`/hello` to a simple `Hello World!` function called `iron/hello` which is a function we already made that you
+can use -- yes, you can share functions! The source code for this function is in the [examples directory](examples/hello-go). 
+You can read more about [writing your own functions here](docs/writing.md).
 
 ```sh
 fnctl routes create myapp /hello iron/hello
 ```
 
-Or using a cURL call:
+Or using cURL:
+
 ```sh
 curl -H "Content-Type: application/json" -X POST -d '{
     "route": {
@@ -75,31 +123,37 @@ curl -H "Content-Type: application/json" -X POST -d '{
 }' http://localhost:8080/v1/apps/myapp/routes
 ```
 
+[More on routes](docs/routes.md).
+
 ### Calling your Function
 
 Calling your function is as simple as requesting a URL. Each app has it's own namespace and each route mapped to the app.
-The app `myapp` that we created above along with the `/hello` route we added would be called via the following URL.
+The app `myapp` that we created above along with the `/hello` route we added would be called via the following 
+URL: http://localhost:8080/r/myapp/hello
+
+Either surf to it in your browser or use `fnctl`:
 
 ```sh
 fnctl routes run myapp /hello
 ```
 
-Or using a cURL call:
+Or using a cURL:
+
 ```sh
 curl http://localhost:8080/r/myapp/hello
 ```
 
-You also may just surf to it: http://localhost:8080/r/myapp/hello
-
 ### Passing data into a function
 
-Your function will get the body of the HTTP request via STDIN, and the headers of the request will be passed in as env vars. Try this:
+Your function will get the body of the HTTP request via STDIN, and the headers of the request will be passed in
+as env vars. You can test a function with the CLI tool:
 
 ```sh
 echo '{"name":"Johnny"}' | fnctl routes run myapp /hello
 ```
 
-Or using a cURL call:
+Or using cURL:
+
 ```sh
 curl -H "Content-Type: application/json" -X POST -d '{
     "name":"Johnny"
@@ -151,16 +205,23 @@ If you watch the logs, you will see the function actually runs in the background
 
 Read more on [logging](docs/logging.md).
 
-## Client Libraries
-
-- [Go](https://github.com/iron-io/functions_go)
-- [Ruby](https://github.com/iron-io/functions_ruby)
-- [Javascript](https://github.com/iron-io/functions_js)
-
 ## Writing Functions
 
-See [Writing Functions)(docs/writing.md).
+See [Writing Functions](docs/writing.md).
 
 ## More Documentation
 
 See [docs/](docs/README.md) for full documentation.
+
+## Want to contribute to IronFunctions?
+
+See [contributing](CONTRIBUTING.md).
+
+## Support
+
+You can get community support via:
+
+* [Stack Overflow](http://stackoverflow.com/questions/tagged/ironfunctions)
+* [Slack](https://get.iron.io/open-slack)
+
+You can get commercial support by contacting [Iron.io](https://iron.io)

docs/README.md

@@ -1,16 +1,25 @@
-
 # IronFunctions Documentation
 
-* [IronFunctions Run Options](options.md)
+## For Developers
-* [Writing Functions](function-format.md)
+
+If you are a developer using IronFunctions through the API, this section is for you.
+
+* [Quickstart](https://github.com/iron-io/functions#quickstart)
+* [CLI tool - fnctl](../fnctl/README.md)
+* [Writing functions](writing.md)
+* [Packaging functions](packaging.md)
+* [Open Function Format](function-format.md)
 * [API Reference](https://app.swaggerhub.com/api/iron/functions/)
+
+## For Operators
+
+If you are operating IronFunctions, this section is for you.
+
 * [Running in Production](production.md)
 * [Databases](databases/README.md)
 * [Message Queues](mqs/README.md)
 * [Logging](logging.md)
 * [Metrics](metrics.md)
-* [Function Memory Requirements](memory.md)
 * [Triggers](triggers.md)
 * [Extending IronFunctions](extending.md)
 * [Docker Configuration](docker.md)
-* [Function Format](function-format.md)

docs/apps.md

@@ -0,0 +1,9 @@
+# Applications
+
+Applications are the top level object that groups routes together to create an API. 
+
+## App level configuration
+
+When creating or updating an app, you can pass in a map of config variables. 
+
+TODO: link to swagger doc on swaggerhub after it's updated. 

docs/faq.md

@@ -0,0 +1,21 @@
+# Frequently Asked Questions
+
+## Which languages are supported?
+
+Since we use containers are the base building block, all languages can be used. There there may not be higher level 
+helper libraries like our Lambda wrapper for every language, but you can use any language if you follow the 
+base [function format](function-format.md).
+
+## Where can I run IronFunctions?
+
+Anywhere. Any cloud, on-premise, on your laptop. As long as you can run a Docker container, you can run IronFunctions.
+
+## Which orchestration platforms does IronFunctions support?
+
+IronFunctions can run using any orchestration tool, any server 
+
+## Does IronFunctions require Docker?
+
+For now, we do require Docker primarily for the packaging and distribution via Docker Registries. 
+But we've built IronFunctions in a way that abstracts the container technology so we can support others as
+needed. For instance, we'll probably add rkt support shortly.

docs/lambda/README.md

@@ -1,6 +1,6 @@
 # Lambda everywhere.
 
-AWS Lambda introduced server-less computing to the masses. Wouldn't it be nice
+AWS Lambda introduced serverless computing to the masses. Wouldn't it be nice
 if you could run the same Lambda functions on any platform, in any cloud?
 Iron.io is proud to release a set of tools that allow just this. Package your
 Lambda function in a Docker container and run it anywhere with an environment
@@ -25,20 +25,17 @@ write a simple function and have an "executable" ready to go.
 ## How does it work?
 
 We provide base Docker images for the various runtimes that AWS Lambda
-supports. The `fnclt` tool helps package up your Lambda function into
+supports. The `fnctl` tool helps package up your Lambda function into
 a Docker image layered on the base image. We provide a bootstrap script and
 utilities that provide a AWS Lambda environment to your code. You can then run
 the Docker image on any platform that supports Docker. This allows you to
 easily move Lambda functions to any cloud provider, or host it yourself.
 
-The Docker container has to be run with a certain configuration, described
-[here](./docker-configuration.md)
-
 ## Next steps
 
 Write, package and run your Lambda functions with our [Getting started
 guide](./getting-started.md). [Here is the environment](./environment.md) that
-Lambda provides. `fnclt lambda` lists the commands to work with Lambda
+Lambda provides. `fnctl lambda` lists the commands to work with Lambda
 functions locally.
 
 You can [import](./import.md) existing Lambda functions hosted on Amazon!

docs/lambda/import.md

@@ -23,7 +23,7 @@ region. You can use the `aws` tool to set this up. Full instructions are in the
 The aws-import command is constructed as follows:
 
 ```bash
-fnclt lambda aws-import <arn> <region> <image>
+fnctl lambda aws-import <arn> <region> <image>
 ```
 
 * arn: describes the ARN formats which uniquely identify the AWS lambda resource
@@ -31,9 +31,11 @@ fnclt lambda aws-import <arn> <region> <image>
 * image: the name of the created docker image which should have the format <username>/<image-name>
 
 Assuming you have a lambda with the following arn `arn:aws:lambda:us-west-2:123141564251:function:my-function`, the following command:
-```bash
+
-fnclt lambda aws-import arn:aws:lambda:us-west-2:123141564251:function:my-function us-east-1 user/my-function
+```sh
+fnctl lambda aws-import arn:aws:lambda:us-west-2:123141564251:function:my-function us-east-1 user/my-function
 ```
+
 will import the function code from the region `us-east-1` to a directory called `./my-function`. It will
 then create a docker image called `my-function`.
 

docs/mqs/README.md

@@ -1,4 +1,3 @@
-
 # Message Queues
 
 A message queue is used to coordinate asynchronous function calls that run through IronFunctions.

docs/options.md

@@ -1,4 +1,6 @@
-# IronFunctions Configuration Options
+# IronFunctions Runtime Options
+
+## Configuration
 
 When starting IronFunctions, you can pass in the following configuration variables as environment variables. Use `-e VAR_NAME=VALUE` in 
 docker run.  For example:
@@ -14,19 +16,19 @@ docker run -e VAR_NAME=VALUE ...
 </tr>
 <tr>
 <td>DB</td>
-<td>The database URL to use in URL format. See Databases below for more information. Default: BoltDB in current working directory `bolt.db`.</td>
+<td>The database URL to use in URL format. See [Databases](databases/README.md) for more information. Default: BoltDB in current working directory `bolt.db`.</td>
 </tr>
 <tr>
 <td>MQ</td>
-<td>The message queue to use in URL format. See Message Queues below for more information. Default: BoltDB in current working directory `queue.db`.</td>
+<td>The message queue to use in URL format. See [Message Queues](mqs/README.md) for more information. Default: BoltDB in current working directory `queue.db`.</td>
 </tr>
 <tr>
 <td>API_URL</td>
-<td>The primary functions api URL to pull tasks from (the address is that of another running functions process).</td>
+<td>The primary IronFunctions API URL to that this instance will talk to. In a production environment, this would be your load balancer URL.</td>
 </tr>
 <tr>
 <td>PORT</td>
-<td>Default (8080), sets the port to run on.</td>
+<td>Sets the port to run on. Default: `8080`.</td>
 </tr>
 <tr>
 <td>NUM_ASYNC</td>
@@ -34,6 +36,33 @@ docker run -e VAR_NAME=VALUE ...
 </tr>
 <tr>
 <td>LOG_LEVEL</td>
-<td>Set to `DEBUG` to enable debugging. Default is INFO.</td>
+<td>Set to `DEBUG` to enable debugging. Default: INFO.</td>
 </tr>
 </table>
+
+## Starting without Docker in Docker
+
+The default way to run IronFunctions, as it is in the Quickstart guide, is to use docker-in-docker (dind). There are
+a couple reasons why we did it this way:
+
+* It's clean. Once the container exits, there is nothing left behind including all the function images.
+* You can set resource restrictions for the entire IronFunctions instance. For instance, you can set `--memory` on
+the docker run command to set the max memory for the IronFunctions instance AND all of the functions it's running. 
+
+There are some reasons you may not want to use dind, such as using the image cache during testing or you're running
+[Windows](windows.md).
+
+### Mount the Host Docker
+
+One way is to mount the host Docker. Everything is essentially the same except you add a `-v` flag:
+
+```sh
+docker run --rm --name functions -it -v /var/run/docker.sock:/var/run/docker.sock -v $PWD/data:/app/data -p 8080:8080 iron/functions
+```
+
+### Run outside Docker
+
+You can of course just run the binary directly, you'll just have to change how you set the environment variables above.
+
+See [contributing doc](../CONTRIBUTING.md) for information on how to build and run.
+

docs/packaging.md

@@ -0,0 +1,50 @@
+# Packaging your Function
+
+Packaging a function has two parts:
+
+* Create a Docker image for your function with an ENTRYPOINT
+* Push your Docker image to a registry (Docker Hub by default)
+
+Once it's pushed to a registry, you can use it by referencing it when adding a route. 
+
+## Creating an image
+
+The basic Dockerfile for most languages is along these lines:
+
+```
+# Choose base image
+FROM iron/go
+# Set the working directory
+WORKDIR /function
+# Add your binary or code to the working directory
+ADD funcbin /function/
+# Set what will run when a container is started for this image
+ENTRYPOINT ["./funcbin"]
+```
+
+Then you simply build your function:
+
+```sh
+docker run --rm -v "$PWD":/go/src/$FUNCPKG -w /go/src/$FUNCPKG iron/go:dev go build -o funcbin
+docker build -t $USERNAME/myfunction .
+```
+
+Or using [fnctl](../fnctl/README.md):
+
+```sh
+fnctl build
+```
+
+## Push your image
+
+This part is simple:
+
+```sh
+docker push $USERNAME/myfunction 
+```
+
+Or using [fnctl](../fnctl/README.md):
+
+```sh
+fnctl push
+```

docs/routes.md

@@ -1,16 +1,26 @@
-# Function Memory Requirements
+# IronFunctions Routes
+
+Routes have a many-to-one mapping to an [app](apps.md). 
 
 A good practice to get the best performance on your IronFunctions API is define the required memory for each function.
 
+## Route level configuration
+
+When creating or updating a route, you can pass in a map of config variables. 
+
+Note: Route level configuration overrides app level configuration. 
+
+TODO: link to swagger doc on swaggerhub after it's updated.
+
 ## Understanding IronFunctions memory management
 
 When IronFunctions starts it registers the total available memory in your system in order to know during its runtime if the system has the required amount of free memory to run each function.
 Every function starts the runner reduces the amount of memory used by that function from the available memory register.
 When the function finishes the runner returns the used memory to the available memory register.
 
-By default the required memory of a function is *128 mb*.
+Default memory is 128MB. 
 
-## Defining function's required memory
+## Defining function's memory requirement
 
 You can define the function's required memory in the route creation or updating it.
 

docs/writing.md

@@ -1,7 +1,7 @@
 # Writing Functions
 
-This will give you the basic overview of writing base level functions. You can also use higher level abstractions that make it easier such
+This will give you the basic overview of writing base level functions. You can also use higher level 
-as [lambda](lambda.md).
+abstractions that make it easier such as [lambda](lambda/README.md).
 
 ## Code
 
@@ -62,20 +62,7 @@ In Ruby:
 STDERR.puts("hi")
 ```
 
-## Packaging
+## Next Steps
 
-Packaging a function is essentially just creating a Docker image for your function with an ENTRYPOINT.
+* [Packaging your function](packaging.md)
-
-The basic Dockerfile for most languages is something like this:
-
-```
-# Choose base image
-FROM iron/go
-# Set th working directory
-WORKDIR /function
-# Add your binary or code to the working directory
-ADD hello /function/
-# Set what will run when a container is started for this image
-ENTRYPOINT ["./hello"]
-```
 

examples/hello-ruby/function.rb

@@ -2,7 +2,6 @@ require 'json'
 
 name = "World"
 
-
 payload = STDIN.read
 if payload != ""
     payload = JSON.parse(payload)

fnctl/.gitignore

@@ -1 +1,2 @@
 fnctl
+vendor/

fnctl/Makefile

@@ -8,7 +8,7 @@ docker: vendor
 	docker push iron/fnctl
 
 vendor:
-	go get -u .
+	glide install
 
 test:
 	go test -v $(shell glide nv)

fnctl/apps.go

@@ -18,7 +18,7 @@ func apps() cli.Command {
 	return cli.Command{
 		Name:      "apps",
 		Usage:     "list apps",
-		ArgsUsage: "fnclt apps",
+		ArgsUsage: "fnctl apps",
 		Flags:     append(confFlags(&a.Configuration), []cli.Flag{}...),
 		Action:    a.list,
 		Subcommands: []cli.Command{

fnctl/lambda.go

@@ -36,7 +36,7 @@ func lambda() cli.Command {
 	return cli.Command{
 		Name:      "lambda",
 		Usage:     "create and publish lambda functions",
-		ArgsUsage: "fnclt lambda",
+		ArgsUsage: "fnctl lambda",
 		Subcommands: []cli.Command{
 			{
 				Name:      "create-function",

fnctl/routes.go

@@ -27,7 +27,7 @@ func routes() cli.Command {
 	return cli.Command{
 		Name:      "routes",
 		Usage:     "list routes",
-		ArgsUsage: "fnclt routes",
+		ArgsUsage: "fnctl routes",
 		Flags:     flags,
 		Action:    r.list,
 		Subcommands: []cli.Command{