Commit Graph

12 Commits

Author SHA1 Message Date
Denis Makogon
24eacbbbbf Improving certain CLI commands: init, run (#207)
* Consolidating route type utilization

Fixes: #197

* Adding --version to fn init CLI command

* Updating functions_go to 0.1.36

* Updating route init process

* Updating tests

* Adding "memory" CLI flag to fn run

* Make CLI memory flag override what's in func.yaml

* Get rid of default value for memory flag

* Revert UX impact

 func.yaml remains the same, no nested maps
2017-08-18 11:14:29 -07:00
Reed Allman
9edacae928 clean up hotf(x) concurrency, rm max c
this patch gets rid of max concurrency for functions altogether, as discussed,
since it will be challenging to support across functions nodes. as a result of
doing so, the previous version of functions would fall over when offered 1000
functions, so there was some work needed in order to push this through.
further work is necessary as docker basically falls over when trying to start
enough containers at the same time, and with this patch essentially every
function can scale infinitely. it seems like we could add some kind of
adaptive restrictions based on task run length and configured wait time so
that fast running functions will line up to run in a hot container instead of
them all creating new hot containers.

this patch takes a first cut at whacking out some of the insanity that was the
previous concurrency model, which was problematic in that it limited
concurrency significantly across all functions since every task went through
the same unbuffered channel, which could create blocking issues for all
functions if the channel is not picked off fast enough (it's not apparent that
this was impossible in the previous implementation). in any event, each
request has a goroutine already, there's no reason not to use it. not too hard
to wrap a map in a lock, not sure what the benefits were (added insanity?) in effect
this is marginally easier to understand and less insane (marginally). after
getting rid of max c this adds a blocking mechanism for the first invocation
of any function so that all other hot functions will wait on the first one to
finish to avoid a herd issue (was making docker die...) -- this could be
slightly improved, but works in a pinch. reduced some memory usage by having
redundant maps of htfnsvr's and task.Requests (by a factor of 2!). cleaned up
some of the protocol stuff, need to clean this up further. anyway, it's a
first cut. have another patch that rewrites all of it but was getting into
rabbit hole territory, would be happy to oblige if anybody else has problems
understanding this rat's nest of channels. there is a good bit of work left to
make this prod ready (regardless of removing max c).

a warning that this will break the db schemas, didn't put the effort in to add
migration stuff since this isn't deployed anywhere in prod...

TODO need to clean out the htfnmgr bucket with LRU
TODO need to clean up runner interface
TODO need to unify the task running paths across protocols
TODO need to move the ram checking stuff into worker for noted reasons
TODO need better elasticity of hot f(x) containers
2017-06-05 20:04:13 -07:00
Denis Makogon
31b4ac4516 Address broken tests 2017-05-30 08:50:53 -07:00
Travis Reeder
69f0201818 Some small cleanup to docs. 2017-05-26 18:54:26 +00:00
Travis Reeder
0c6ca08a2f functions: 0.2.62 release [skip ci] 2017-05-15 14:04:01 -07:00
C Cirello
2f0520521c all: rename hot containers to hot functions (#465) 2016-12-29 20:07:41 +01:00
Pedro Nasser
811689f923 API definitions and docs update (#457)
* fix: using bolt config/header update now overwrites

* api definitions and doc updates

* merge docs
2016-12-27 19:03:02 +01:00
C Cirello
470e497e7b fn: improve funcfile doc (#376)
* fn: improve funcfile doc

* fn: improve wording of build key

* fn: make clear max_concurrency is per node
2016-12-05 17:15:20 +01:00
C Cirello
28f713ed11 fn: support for functions testing (#379)
* fn: add test framework

* fn: make routes creation smarter

* fn: add testframework examples

* fn: remove unnecessary dependency

* fn: update doc

* fn: fix consistenty between runff, runlocaltest and runremotetest
2016-12-05 17:05:06 +01:00
C Cirello
9ac2539aeb fn: improve UX and publish/deploy command (#359)
* fn: improve UX and publish/deploy command

* fn: remove wrong use cases for deploy

* fn: fix regression introduced by merge
2016-12-01 18:11:13 +01:00
C Cirello
c0512a4cbd fn: rename from fnctl (#321)
* fn: rename from fnctl

* fn: predicting the release version for installer

* fn: predicting the release version for installer
2016-11-21 17:24:26 +01:00
Travis Reeder
0d71e1e38e Docs update with new fnctl commands (#273)
* Added high level roadmap.

* Changed to funtion.yaml.

* Added logo

* updating quickstart code example, WIP, waiting on another merge.

* Minor updates.

* Changed function.yaml to func.yaml and updated fnctl README.
2016-11-15 01:40:05 +01:00