* Don't try to delete an app that wasn't successfully created in the case of failure
* Allow datastore implementations to inject additional annotations on objects
* Allow for datastores transparently adding annotations on apps, fns and triggers. Change NameIn filter to Name for apps.
* Move *List types including JSON annotations for App, Fn and Trigger into models
* Change return types for GetApps, GetFns and GetTriggers on datastore to
be models.*List and ove cursor generation into datastore
* Trigger cursor handling fixed into db layer
Also changes the name generation so that it is not in the same order
as the id (well is random), this means we are now testing our name ordering.
* GetFns now respects cursors
* Apps now feeds cursor back
* Mock fixes
* Fixing up api level cursor decoding
* Tidy up treatment of cursors in the db layer
* Adding conditions for non nil items lists
* fix mock test
* Fixed up a couple of incorrect response codes
* Standardise all entities on 204 with no return content on successful delete
* Fix failing Fn.delete() test
* Tidying and rationalising summary and description strings, some minor rejigging of elements for consistency.
* Removed a single solitary fullstop.
* Quoted a few more unquoted strings...
* Capitalized summaries, no full stops. Sentences for descriptions, yes full stops.
* More Capitals, less full stops...
Vast commit, includes:
* Introduces the Trigger domain entity.
* Introduces the Fns domain entity.
* V2 of the API for interacting with the new entities in swaggerv2.yml
* Adds v2 end points for Apps to support PUT updates.
* Rewrites the datastore level tests into a new pattern.
* V2 routes use entity ID over name as the path parameter.
This is a small tweak to the placer latency stats. If we have a cluster of values
around the 1-2s mark, then having a single relatively broad bucket that captures
the (1s, 10s] range will obscure that. In particular, typical Prometheus quartile
estimates may be distorted by this bucket size.
* add DateTime sans mgo
* change all uses of strfmt.DateTime to common.DateTime, remove test strfmt usage
* remove api tests, system-test dep on api test
multiple reasons to remove the api tests:
* awkward dependency with fn_go meant generating bindings on a branched fn to
vendor those to test new stuff. this is at a minimum not at all intuitive,
worth it, nor a fun way to spend the finite amount of time we have to live.
* api tests only tested a subset of functionality that the server/ api tests
already test, and we risk having tests where one tests some thing and the
other doesn't. let's not. we have too many test suites as it is, and these
pretty much only test that we updated the fn_go bindings, which is actually a
hassle as noted above and the cli will pretty quickly figure out anyway.
* fn_go relies on openapi, which relies on mgo, which is deprecated and we'd
like to remove as a dependency. openapi is a _huge_ dep built in a NIH
fashion, that cannot simply remove the mgo dep as users may be using it.
we've now stolen their date time and otherwise killed usage of it in fn core,
for fn_go it still exists but that's less of a problem.
* update deps
removals:
* easyjson
* mgo
* go-openapi
* mapstructure
* fn_go
* purell
* go-validator
also, had to lock docker. we shouldn't use docker on master anyway, they
strongly advise against that. had no luck with latest version rev, so i locked
it to what we were using before. until next time.
the rest is just playing dep roulette, those end up removing a ton tho
* fix exec test to work
* account for john le cache
* fn: remove confusing parallelism in test scripts
*) Tests should be consistent when run from makefile versus
running these test scripts from command line. Let go use
GOMAXPROCS instead of hardcoded 4 cpus in Makefile.
*) Moved docker pull for specific image versions into
helpers scripts as well. Easier to maintain image version
for tests in the same place.
*) Minor Makefile cleanup: removed unused makefile targets.
* fn: git-diff rename limit increase
* fn: test scripts should use well defined ports
Moved allocation of listener ports for mysql/minio/postgres
to helper script with a list of service list names.
* fn: makefile docker pull mysql version must match tests
part deux of https://github.com/fnproject/fn/issues/1067 -- I am planning to update CONTRIBUTING.md to say something along the lines of 'open an issue' to create an 'open an issue describing a problem' -> 'open a pr resolving said issue' kinda format, this is pretty standard and helps make users feel comfortable opening issues to discuss work they are planning to do rather than just going off and doing it (or being scared off). the rest of it is pretty barebones and copied from Docker mostly (I like their approach, some others are kind of intimidating, am open to ideas though). gif ideas welcome.
part 1 of #1067 - this will help users create issues with all (most of?) the context we need without having to ask. it also includes a policy to close bugs that are left stale after 7 days, which is convenient to keep issues from growing like a weed.
In pure-runner and LB agent, service providers might want to set specific driver options.
For example, to add cpu-shares to functions, LB can add the information as extensions
to the Call and pass this via gRPC to runners. Runners then pick these extensions from
gRPC call and pass it to driver. Using a custom driver implementation, pure-runners can
process these extensions to modify docker.CreateContainerOptions.
To achieve this, LB agents can now be configured using a call overrider.
Pure-runners can be configured using a custom docker driver.
RunnerCall and Call interfaces both expose call extensions.
An example to demonstrate this is implemented in test/fn-system-tests/system_test.go
which registers a call overrider for LB agent as well as a simple custom docker driver.
In this example, LB agent adds a key-value to extensions and runners add this key-value
as an environment variable to the container.
* Adding a way to inject a request ID
It is very useful to associate a request ID to each incoming request,
this change allows to provide a function to do that via Server Option.
The change comes with a default function which will generate a new
request ID. The request ID is put in the request context along with a
common logger which always logs the request-id
We add gRPC interceptors to the server so it can get the request ID out
of the gRPC metadata and put it in the common logger stored in the
context so as all the log lines using the common logger from the context
will have the request ID logged
* fn: introducing lb placer basic metrics
This change adds basic metrics to naive and consistent
hash LB placers. The stats show how many times we scanned
the full runner list, if runner pool failed to return a
runner list or if runner pool returned an empty list.
Placed and not placed status are also tracked along with
if TryExec returned an error or not. Most common error
code, Too-Busy is specifically tracked.
If client cancels/times out, this is also tracked as
a client cancel metric.
For placer latency, we would like to know how much time
the placer spent on searching for a runner until it
successfully places a call. This includes round-trip
times for NACK responses from the runners until a successful
TryExec() call. By excluding last successful TryExec() latency,
we try to exclude function execution & runner container
startup time from this metric in an attempt to isolate
Placer only latency.
* fn: latency and attempt tracker
Removing full scan metric. Tracking number of
runners attempted is a better metric for this
purpose.
Also, if rp.Runners() fail, this is an unrecoverable
error and we should bail out instead of retrying.
* fn: typo fix, ch placer finalize err return
* fn: enable LB placer metrics in WithAgentFromEnv if prometheus is enabled
* initial Db helper split - make SQL and datastore packages optional
* abstracting log store
* break out DB, MQ and log drivers as extensions
* cleanup
* fewer deps
* fixing docker test
* hmm dbness
* updating db startup
* Consolidate all your extensions into one convenient package
* cleanup
* clean up dep constraints
