functions: hot containers (#332)

* functions: modify datastore to accomodate hot containers support

* functions: protocol between functions and hot containers

* functions: add hot containers clockwork

* fn: add hot containers support

docs/function-format.md

@@ -2,33 +2,27 @@
 
 This document will describe the details of how a function works, inputs/outputs, etc.
 
-## Input
+## Formats
 
 ### STDIN and Environment Variables
 
-While wanting to keep things simple, flexible and expandable, we decided to go back to the basics, using Unix input and output. Standard in is easy to use in any
-language and doesn't require anything extra. It also allows streaming input so we can do things like keeping a container running some time and stream
-requests into the container. 
+While wanting to keep things simple, flexible and expandable, we decided to go back to the basics, using Unix input and output. Standard in is easy to use in any language and doesn't require anything extra. It also allows streaming input so we can do things like keeping a container running some time and stream requests into the container.
 
 Configuration values, environment information and other things will be passed in through environment variables.
 
-### Input Formats
-
 The goals of the input format are the following:
 
 * Very easy to use and parse
 * Streamable for increasing performance (more than one call per container execution)
 * Ability to build higher level abstractions on top (ie: Lambda syntax compatible)
 
-The format is still up for discussion and in order to move forward and remain flexible, it's likely we will just allow different input formats and the
-function creator can decide what they want, on a per function basis. Default being the simplest format to use.
+The format is still up for discussion and in order to move forward and remain flexible, it's likely we will just allow different input formats and the function creator can decide what they want, on a per function basis. Default being the simplest format to use.
 
-#### Default Input Format
+#### Default I/O Format
 
-The default input format is simply the request body itself plus some environment variables. For instance, if someone were to post a JSON body, the unmodified body would 
-be sent in via STDIN. 
+The default I/O format is simply the request body itself plus some environment variables. For instance, if someone were to post a JSON body, the unmodified body would be sent in via STDIN. The result comes via STDOUT. When task is done, pipes are closed and the container running the function is terminated.
 
-Pros: 
+Pros:
 
 * Very simple to use
 
@@ -36,23 +30,31 @@ Cons:
 
 * Not streamable
 
-#### HTTP/1 Input Format (Not implemented)
+#### HTTP I/O Format
 
-`--input-format http`
+`--format http`
 
-HTTP format could be a good option as it is in very common use obviously, most languages have some semi-easy way to parse it, and it's streamable. The basic format
-is:
+HTTP format could be a good option as it is in very common use obviously, most languages have some semi-easy way to parse it, and it's streamable. The response will look like a HTTP response. The communication is still done via stdin/stdout, but these pipes are never closed unless the container is explicitly terminated. The basic format is:
 
+Request:
 ```
-REQUEST LINE
-HEADER
-BLANK LINE
-BODY
+GET / HTTP/1.1
+Content-Length: 5
+
+world
 ```
 
-The header keys and values would be populated with information about the function call such as the request URL and query parameters.  
+Response:
+```
+HTTP/1.1 200 OK
+Content-Length: 11
 
-Body length is determined by the [Content-Length](https://tools.ietf.org/html/rfc7230#section-3.3.3) header, which is mandatory.
+hello world
+```
+
+The header keys and values would be populated with information about the function call such as the request URL and query parameters.
+
+`Content-Length` is determined by the [Content-Length](https://tools.ietf.org/html/rfc7230#section-3.3.3) header, which is mandatory both for input and output. It is used by IronFunctions to know when stop writing to STDIN and reading from STDOUT.
 
 Pros:
 
@@ -64,11 +66,11 @@ Cons:
 * Requires a parsing library or fair amount of code to parse headers properly
 * Double parsing - headers + body (if body is to be parsed, such as json)
 
-#### JSON/HTTP Input Format (Not implemented)
+#### JSON I/O Format (not implemented)
 
-`--input-format json-http`
+`--format json`
 
-The idea here is to keep the HTTP base structure, but make it a bit easier to parse by making the `request line` and `headers` a JSON struct. 
+The idea here is to keep the HTTP base structure, but make it a bit easier to parse by making the `request line` and `headers` a JSON struct.
 Eg:
 
 ```
@@ -91,18 +93,7 @@ Cons:
 
 * New, unknown format
 
-## Output
-
-### STDOUT
-
-For synchronous: True to form, whatever is written to standard out is returned as the response. If you want to return some JSON output, just write it directly to STDOUT.
-
-TODO: How to change response headers? Perhaps a similar style as input? Headers, then body. Default headers can be defined on the route and overridden on output. 
-
-For asynchronous: STDOUT will be written to /dev/null until [further notice](https://github.com/iron-io/functions/issues/173). We do not want to write this 
-to the logs now, then change it later, otherwise people will start to depend on it. 
-
 ### STDERR
 
-Standard error is reserved for logging, like it was meant to be. Anything you output to STDERR will show up in the logs. And if you use a log 
+Standard error is reserved for logging, like it was meant to be. Anything you output to STDERR will show up in the logs. And if you use a log
 collector like logspout, you can collect those logs in a central location. See [logging](logging.md).

docs/swagger.yml

@@ -6,7 +6,7 @@ swagger: '2.0'
 info:
   title: IronFunctions
   description: The open source serverless platform.
-  version: "0.1.17"
+  version: "0.1.18"
 # the domain of the service
 host: "127.0.0.1:8080"
 # array of all schemes that your API supports
@@ -338,6 +338,16 @@ definitions:
               - sync
               - async
             description: Route type
+          format:
+            enum:
+              - default
+              - http
+              - json
+            description: Payload format sent into function.
+          max_concurrency:
+            type: integer
+            format: int32
+            description: Maximum number of hot containers concurrency
           config:
             type: object
             description: Route configuration - overrides application configuration