3a9c48b8a3
* POC code for inotify UDS-io-socket * http-stream format introducing the `http-stream` format support in fn. there are many details for this, none of which can be linked from github :( -- docs are coming (I could even try to add some here?). this is kinda MVP-ish level, but does not implement the remaining spec, ie 'headers' fixing up / invoke fixing up. the thinking being we can land this to test fdks / cli with and start splitting work up on top of this. all other formats work the same as previous (no breakage, only new stuff) with the cli you can set `format: http-stream` and deploy, and then invoke a function via the `http-stream` format. this uses unix domain socket (uds) on the container instead of previous stdin/stdout, and fdks will have to support this in a new fashion (will see about getting docs on here). fdk-go works, which is here: https://github.com/fnproject/fdk-go/pull/30 . the output looks the same as an http format function when invoking a function. wahoo. there's some amount of stuff we can clean up here, enumerated: * the cleanup of the sock files is iffy, high pri here * permissions are a pain in the ass and i punted on dealing with them. you can run `sudo ./fnserver` if running locally, it may/may not work in dind(?) ootb * no pipe usage at all (yay), still could reduce buffer usage around the pipe behavior, we could clean this up potentially before removal (and tests) * my brain can’t figure out if dispatchOldFormats changes pipe behavior, but tests work * i marked XXX to do some clean up which will follow soon… need this to test fdk tho so meh, any thoughts on those marked would be appreciated however (1 less decision for me). mostly happy w/ general shape/plumbing tho * there are no tests atm, this is a tricky dance indeed. attempts were made. need to futz with the permission stuff before committing to adding any tests here, which I don't like either. also, need to get the fdk-go based test image updated according to the fdk-go, and there's a dance there too. rumba time.. * delaying the big big cleanup until we have good enough fdk support to kill all the other formats. open to ideas on how to maneuver landing stuff... * fix unmount * see if the tests work on ci... * add call id header * fix up makefile * add configurable iofs opts * add format file describing http-stream contract * rm some cruft * default iofs to /tmp, remove mounting out of the box fn we can't mount. /tmp will provide a memory backed fs for us on most systems, this will be fine for local developing and this can be configured to be wherever for anyone that wants to make things more difficult for themselves. also removes the mounting, this has to be done as root. we can't do this in the oss fn (short of requesting root, but no). in the future, we may want to have a knob here to have a function that can be configured in fn that allows further configuration here. since we don't know what we need in this dept really, not doing that yet (it may be the case that it could be done operationally outside of fn, eg, but not if each directory needs to be configured itself, which seems likely, anyway...) * add WIP note just in case...
3.9 KiB
Fn container contract
NOTE: THIS IS WORK IN PROGRESS AND ITS API IS SUBJECT TO CHANGE
This document will describe the details of how a function works, inputs/outputs, etc. (It is meant to replace (./function-format.md) - if the time has come, remove this line, for now it defines the 'http-stream' format option)
The basic idea is to handle http requests over a unix domain socket. Each container will only receive one request at a time, in that until a response is returned from a previous request, no new requests will be issued to a container's http server.
XXX(reed): motivation section for future us posterity?
FDK Contract outline
Function Development Kits (FDKs) are libraries for various languages that implement Fn's container contract for function input, output and configuration. In order to be a fully 'FDK compliant', below are the rules:
If FN_FORMAT variable is http-stream, then FDKs MUST parse FN_LISTENER environment variable.
FN_LISTENER must contain the listener address for FDKs to bind/listen to. Currently we only support unix-domain-socket (UDS) with SOCK_STREAM (TCP). This means FN_LISTENER prefix is always unix:/.
For example:
FN_LISTENER=unix:/var/run/fn/listener/listen.sock
FN_FORMAT=http-stream
If FN_FORMAT is http-stream, then absence of FN_LISTENER or "unix:" prefix in FN_LISTENER is an error and FDKs are REQUIRED to terminate/exit with error.
Before exiting, FDKs SHOULD remove the UDS file (from FN_LISTENER path).
FDKs upon creation of UDS file on disk with bind system call SHOULD be ready to receive and handle traffic. Upon bind call, the UDS file MUST be writable by fn-agent.
Path in FN_LISTENER (after "unix:" prefix) cannot be larger than 107 bytes.
FDKs MUST listen on the unix socket within 5 seconds of startup, Fn will enforce time limits and will terminate such FDK containers.
Once initialised the FDK should respond to HTTP requests by accepting connections on the created unix socket.
The FDK SHOULD NOT enforce any read or write timeouts on incoming requests or responses
The FDK SHOULD support HTTP/1.1 Keep alive behaviour
The agent MUST maintain no more than one concurrent HTTP connection to the FDK HTTP servers
Containers MUST implement HTTP/1.1
Any data sent to Stdout and Stderr will be logged by Fn and sent to any configured logging facility
Each function container is responsible for handling multiple function invocations against it, one at a time, for as long as the container lives.
Fn will make HTTP requests to the container on the /call URL of the containers HTTP UDS port using.
POST /call HTTP/1.1
Host: localhost:8080
Fn-Call-Id : <call_id>
Fn-Deadline: <date/time>
Content-type: application/cloudevent+json
<Body here>
HTTP/1.1 200 OK
Fn-Http-Status: 204
Fn-Http-H-My-Header: foo
Content-type: text/plain
<Body here>
Environment Variables
The below are the environment variables that a function can expect to use. FDKs SHOULD provide a facility to easily access these without having to use an OS library.
FN_ID- fn idFN_APP_ID- app id of the fnFN_NAME- name of the fnFN_APP_NAME- the name of the application of the fnFN_FORMAT-http-stream(DEPRECATED)FN_LISTENER- the path where a unix socket file should acceptFN_MEMORY- a number representing the amount of memory available to the call, in MBFN_TMPSIZE- a number representing the amount of writable disk space available under /tmp for the call, in MB
In addition to these, all config variables set on app.config or fn.config will be populated into the environment exactly as configured, for example if app.config = { "HAMMER": "TIME" } your environment will be populated with HAMMER=TIME.
Headers
Fn-Call-Id- id for the callFn-Deadline- RFC3339 timestamp indicating the deadline for a function callFn-*- reserved for future usage*- any other headers, potentially rooted from an http trigger