* the dispatch span actually encloses dispatch and gives an accurate span now
* turning a call into an http request can't fail unless it's our fault, if
tests don't catch this, we don't deserve money
* moved http req creation inside of dispatch goroutine
there's further work to do cleaning up dispatch... removing the old formats
will make this slightly more clear, waiting for that. this was bugging me
anyway after seeing something else and was easy to fix up.
Streaming docker events is useful as we can record/capture some
asynchronous containers events such as out-of-memory. For now,
we record these in opencensus/prometheus stats.
Default fn server keys should be minimal (empty) since not
all stats have associated app name, fn id, etc.
API tags for requests should not include "status" as this is
part of responses.
If checkLaunch triggers evictions, it must wait
for these eviction to complete before returning.
Premature returning from checkLaunch will cause
checkLaunch to be called again by hot launcher.
This causes checkLaunch to receive an out of
capacity error and causes a 503.
The evictor is also improved with this PR and it
provides a slice of channels to wait on if evictions
are taking place.
Eviction token deletion is performed *after*
resource token close to ensure that once an
eviction is done, resource token is also free.
This simplifies resource tracker. Originally, logically we had
split the cpu/mem into two pools where a 20%
was kept specifically for sync calls to avoid
async calls dominating the system. However, resource
tracker should not handle such call prioritization.
Given the improvements to the evictor, I think
we can get rid of this code in resource tracker
for time being.
* Inital Refactor
Removing the repeated logic exposed some problems with the reponse
writers.
Currently, the trigger writer was overlaid on part of the header
writing. The main invoke blog writing into the different levels of the
overlays at different points in the logic.
Instead, by extending the types and embedded structs, the writer is
more transparent. So, at the end of the flow it goes over all the
headers available and removes our prefixes. This lets the invoke logic
just write to the top level.
Going to continue after lunch to try and remove some of the layers and
param passing.
* Try and repeat concurrency failure
* Nested FromHTTPFnRequest inside FromHTTPTriggerRequest
* Consolidate buffer pooling logic
* go fmt yourself
* fix import
*) removed faulty Idle state setter in runHot() since with
UDS wait, we need to wait until we can determine if a container
is idle. This is now moved to runHotReq().
*) evictor now more aggresive and no longer tied to pause
timer/configuration.
*) removed unnecessary optimization on timer=0 case for immediate
pause.
* adds parity level of testing http-stream invoke
the other formats had a gamut of tests, now http-stream does too. this makes
obvious some of its behaviors. some things changed / can change now that we
don't have pipes to worry about, the main one being that when containers blow
up now the uds client will get an EOF/ECONNREFUSED instead of the pipe getting
wedged up (allowing us to get the container error easily, previously). I made
my best 50% effort to make a reasonable error for when this happens (similar
to when http/json received garbage errors), open to ideas on verbiage / policy
there.
should be pretty straightforward. one thing to notice is that
http/json/default don't return our fancy new Fn-Http-Status or Fn-Http-H
headers... it's relatively easy to go add this to fdk-go just to test this,
but for invoke I'm really not sure we care (?) and for the gateway, the output
will be identical with the old formats bypassing the header decap. if anybody
has any feelings, feel free to express them.
* fix oomer up for new error
* Adding http header stripping to agent
Adding the header stripping into the agent, this should be low enough
that all routes to fns get treated the same.
