Commit Graph

4 Commits

Author SHA1 Message Date
Tolga Ceylan
4ccde8897e fn: lb and pure-runner with non-blocking agent (#989)
* fn: lb and pure-runner with non-blocking agent

*) Removed pure-runner capacity tracking code. This did
not play well with internal agent resource tracker.
*) In LB and runner gRPC comm, removed ACK. Now,
upon TryCall, pure-runner quickly proceeds to call
Submit. This is good since at this stage pure-runner
already has all relevant data to initiate the call.
*) Unless pure-runner emits a NACK, LB immediately
streams http body to runners.
*) For retriable requests added a CachedReader for
http.Request Body.
*) Idempotenty/retry is similar to previous code.
After initial success in Engament, after attempting
a TryCall, unless we receive NACK, we cannot retry
that call.
*) ch and naive places now wraps each TryExec with
a cancellable context to clean up gRPC contexts
quicker.

* fn: err for simpler one-time read GetBody approach

This allows for a more flexible approach since we let
users to define GetBody() to allow repetitive http body
read. In default LB case, LB executes a one-time io.ReadAll
and sets of GetBody, which is detected by RunnerCall.RequestBody().

* fn: additional check for non-nil req.body

* fn: attempt to override IO errors with ctx for TryExec

* fn: system-tests log dest

* fn: LB: EOF send handling

* fn: logging for partial IO

* fn: use buffer pool for IO storage in lb agent

* fn: pure runner should use chunks for data msgs

* fn: required config validations and pass APIErrors

* fn: additional tests and gRPC proto simplification

*) remove ACK/NACK messages as Finish message type works
OK for this purpose.
*) return resp in api tests for check for status code
*) empty body json test in api tests for lb & pure-runner

* fn: buffer adjustments

*) setRequestBody result handling correction
*) switch to bytes.Reader for read-only safety
*) io.EOF can be returned for non-nil Body in request.

* fn: clarify detection of 503 / Server Too Busy
2018-05-17 12:09:03 -07:00
Owen Cliffe
d25b5af59d Add annotations to routes and apps (#866)
Adds 'annotations' attribute to Routes and Apps
2018-03-20 18:02:49 +00:00
Denis Makogon
9d6f0b2a05 Speed up API tests (#624)
* Adjust API tests internal API

* Refactor API tests to take less time

 - sqlite: tests 15s, overall time: 1m
 - mysql: tests 15s, overall time: 59s

* Use retry func to survive in faulty places

* Use retry func while trying to ping SQL datastore

 - implements retry func specifically for SQL datastore ping
 - fmt fixes
 - using sqlx.Db.PingContext instead of sqlx.Db.Ping
 - propogate context to SQL datastore

* Simplify TestCanCauseTimeout retry loop

* Call retry with sane timeout

* Fix TestOversizedLog, use retry func

* Increase number of attempts

 2 test cases are really faulty in CI, so they need a lot more time to finish.

* Increase TestCanCauseTimeout timeout

* Use retry at TestMultiLog to speed it up

* Use retry at TestCanWriteLogs to speed it up

* Use retry at TestGetCallsSuccess to speed it up

* Use retry at TestCanGetAsyncState to speed it up

* Use retry at TestListCallsSuccess to speed it up

* Remove sleep calls

* Remove dup test case

* Cleaup Calls API test

* Build API tests binary once

 This patch lets CI to build API tests binary once and reuse that whenever it needs it

* Swap API tests checks

* Build API test binary by default

 dirty fix for CircleCI

* Use retry func to determine if datastore is alive in tests

* go install should also reduce build time

* Fix rebase issues
2018-01-02 13:29:49 -06:00
Denis Makogon
3ae55af392 Adding JSON format validation API tests 2017-10-07 00:43:09 +03:00