* fn: New timeout for LB Placer
Previously, LB Placers worked hard as long as
client contexts allowed for. Adding a Placer
config setting to bound this by 360 seconds by
default.
The new timeout is not accounted during actual
function execution and only applies to the amount
of wait time in Placers when the call is not
being executed.
* fn: cleanup of docker private registry code
Start using URL parsed ServerAddress and its subdomains
for easier image ensure/pull in docker driver. Previous
code to lookup substrings was faulty without proper
URL parse and hostname tokenization. When searching
for a registry config, if image name does not contain
a registry and if there's a private registry configured,
then search for hub.docker.com and index.docker.io. This
is similar to previous code but with correct subdomain
matching.
* fn-dataplane: take port into account in auth configs
* fn: agent eviction revisited
Previously, the hot-container eviction logic used
number of waiters of cpu/mem resources to decide to
evict a container. An ejection ticker used to wake up
its associated container every 1 sec to reasses system
load based on waiter count. However, this does not work
for non-blocking agent since there are no waiters for
non-blocking mode.
Background on blocking versus non-blocking agent:
*) Blocking agent holds a request until the
the request is serviced or client times out. It assumes
the request can be eventually serviced when idle
containers eject themselves or busy containers finish
their work.
*) Non-blocking mode tries to limit this wait time.
However non-blocking agent has never been truly
non-blocking. This simply means that we only
make a request wait if we take some action in
the system. Non-blocking agents are configured with
a much higher hot poll frequency to make the system
more responsive as well as to handle cases where an
too-busy event is missed by the request. This is because
the communication between hot-launcher and waiting
requests are not 1-1 and lossy if another request
arrives for the same slot queue and receives a
too-busy response before the original request.
Introducing an evictor where each hot container can
register itself, if it is idle for more than 1 seconds.
Upon registry, these idle containers become eligible
for eviction.
In hot container launcher, in non-blocking mode,
before we attempt to emit a too-busy response, now
we attempt an evict. If this is successful, then
we wait some more. This could result in requests
waiting for more than they used to only if a
container was evicted. For blocking-mode, the
hot launcher uses hot-poll period to assess if
a request has waited for too long, then eviction
is triggered.
Status calls should not directly use client
gRPC context deadlines/timeouts during Status
execution. Status should allow plenty of time
for the scheduler agent and docker to run and
emit useful error information.
Setting this timeout to 60 seconds, which should
surface disk I/O, docker, etc. issues.
This is useful in scenarios where gRPC client might want
to reliably observe/report the status latency metrics
and remove any possible duplicates. If the status query
was served from cache, then these latencies show last
execution latency.
* fn: runner status and docker load images
Introducing a function run for pure runner Status
calls. Previously, Status gRPC calls returned active
inflight request counts with the purpose of a simple
health checker. However this is not sufficient since
it does not show if agent or docker is healthy. With
this change, if pure runner is configured with a status
image, that image is executed through docker. The
call uses zero memory/cpu/tmpsize settings to ensure
resource tracker does not block it.
However, operators might not always have a docker
repository accessible/available for status image. Or
operators might not want the status to go over the
network. To allow such cases, and in general possibly
caching docker images, added a new environment variable
FN_DOCKER_LOAD_FILE. If this is set, fn-agent during
startup will load these images that were previously
saved with 'docker save' into docker.
The changes aim to be as minimal as possible:
* Remove no longer available options
* Add a schema version to the file for validation
* Add trigger list block
* Initial suypport for invoking tiggers
* dupe method
* tighten server constraints
* runner tests not working yet
* basic route tests passing
* post rebase fixes
* add hybrid support for trigger invoke and tests
* consoloidate all hybrid evil into one place
* cleanup and make triggers unique by source
* fix oops with Agent
* linting
* review fixes
LB agent reports lb placer latency. It should also report
how long it took for the runner to initiate the call as
well as execution time inside the container if the runner
has accepted (committed) to the call.
Added a note on the section about using the shell script to set up the proxy environmental variables before running the script. Also updated MacOS to macOS
I've found this to be extremely useful. Not that I expect anyone to be able to
find this document on their own accord considering the breadth of
documentation that we have, this can still be useful for linking to from slack
at least (what docs are really for, right?)
also the triggers doc stuck out as confusing considering all the triggers
stuff going on, I was unable to comprehend how exactly it was helpful other
than making people aware that openstack exists and they could build an
extension into fn for it if they want to, but this seems true of most things?
so, removed it, if anyone objects maybe we could improve it a little?
