9edacae928
this patch gets rid of max concurrency for functions altogether, as discussed, since it will be challenging to support across functions nodes. as a result of doing so, the previous version of functions would fall over when offered 1000 functions, so there was some work needed in order to push this through. further work is necessary as docker basically falls over when trying to start enough containers at the same time, and with this patch essentially every function can scale infinitely. it seems like we could add some kind of adaptive restrictions based on task run length and configured wait time so that fast running functions will line up to run in a hot container instead of them all creating new hot containers. this patch takes a first cut at whacking out some of the insanity that was the previous concurrency model, which was problematic in that it limited concurrency significantly across all functions since every task went through the same unbuffered channel, which could create blocking issues for all functions if the channel is not picked off fast enough (it's not apparent that this was impossible in the previous implementation). in any event, each request has a goroutine already, there's no reason not to use it. not too hard to wrap a map in a lock, not sure what the benefits were (added insanity?) in effect this is marginally easier to understand and less insane (marginally). after getting rid of max c this adds a blocking mechanism for the first invocation of any function so that all other hot functions will wait on the first one to finish to avoid a herd issue (was making docker die...) -- this could be slightly improved, but works in a pinch. reduced some memory usage by having redundant maps of htfnsvr's and task.Requests (by a factor of 2!). cleaned up some of the protocol stuff, need to clean this up further. anyway, it's a first cut. have another patch that rewrites all of it but was getting into rabbit hole territory, would be happy to oblige if anybody else has problems understanding this rat's nest of channels. there is a good bit of work left to make this prod ready (regardless of removing max c). a warning that this will break the db schemas, didn't put the effort in to add migration stuff since this isn't deployed anywhere in prod... TODO need to clean out the htfnmgr bucket with LRU TODO need to clean up runner interface TODO need to unify the task running paths across protocols TODO need to move the ram checking stuff into worker for noted reasons TODO need better elasticity of hot f(x) containers
2.8 KiB
Function files
Functions files are used to assist fn to help you when creating functions.
The files can be named as:
- func.yaml
- func.json
An example of a function file:
name: funcy/hello
version: 0.0.1
type: sync
memory: 128
config:
key: value
key2: value2
keyN: valueN
headers:
content-type:
- text/plain
build:
- make
- make test
name is the name and tag to which this function will be pushed to and the
route updated to use it.
path (optional) allows you to overwrite the calculated route from the path
position. You may use it to override the calculated route. If you plan to use
fn test --remote="", this is mandatory.
version represents current version of the function. When deploying, it is
appended to the image as a tag.
type (optional) allows you to set the type of the route. sync, for functions
whose response are sent back to the requester; or async, for functions that
are started and return a task ID to customer while it executes in background.
Default: sync.
memory (optional) allows you to set a maximum memory threshold for this
function. If this function exceeds this limit during execution, it is stopped
and error message is logged. Default: 128.
timeout (optional) is the maximum time a function will be allowed to run. Default is 30 seconds.
headers (optional) is a set of HTTP headers to be returned in the response of
this function calls.
config (optional) is a set of configurations to be passed onto the route
setup. These configuration options shall override application configuration
during functions execution.
build (optional) is an array of local shell calls which are used to help
building the function.
Hot functions
hot functions support also adds two extra options to this configuration file.
format (optional) is one of the streaming formats covered at function-format.md.
idle_timeout (optional) is the time in seconds a container will remain alive without receiving any new requests;
hot functions will stay alive as long as they receive a request in this interval. Default: 30.
Testing functions
tests (optional) is an array of tests that can be used to valid functions both
locally and remotely. It has the following structure
tests:
- name: envvar
in: "inserted stdin"
out: "expected stdout"
err: "expected stderr"
env:
envvar: trololo
in (optional) is a string that is going to be sent to the file's declared
function.
out (optional) is the expected output for this function test. It is present
both in local and remote executions.
err (optional) similar to out, however it read from stderr. It is only
available for local machine tests.
env (optional) is a map of environment variables that are injected during
tests.