Fixes async payload passing for #68.

This commit is contained in:
Travis Reeder
2017-06-20 11:32:51 -07:00
parent c94dab3d45
commit 8c96d3ba2f
23 changed files with 276 additions and 225 deletions

View File

@@ -13,6 +13,7 @@ If you are a developer using Oracle Functions through the API, this section is f
* [Open Function Format](function-format.md)
* [API Reference](http://petstore.swagger.io/?url=https://raw.githubusercontent.com/treeder/functions/master/docs/swagger.yml)
* [Hot functions](hot-functions.md)
* [Async functions](async.md)
* [FAQ](faq.md)
## For Operators

18
docs/async.md Normal file
View File

@@ -0,0 +1,18 @@
# Asynchronous Functions
Asynchronous (async) functions will run your function at some point in the future. The default mode is synchronous which means
a function is executed and the caller blocks while waiting for the response. Asynchronous on the other, puts the request into a
message queue and responds immediately to the caller. The function will then be executed at some point in the future, upon resource availability giving priority
to synchronous calls. Also, since it is using a message queue, you can safely queue up millions of function calls without worrying about
capacity.
Async will return immediately with a `call_id`, for example:
```json
{"call_id": "abc123"}
```
The `call_id` can then be used to retrieve the status at a later time.
Asynchronous function calls are great for tasks that are CPU heavy or take more than a few seconds to complete.
For instance, image processing, video processing, data processing, ETL, etc.

View File

@@ -555,58 +555,67 @@ definitions:
readOnly: true
Task:
allOf:
- $ref: "#/definitions/NewTask"
- type: object
properties:
group_name:
type: string
description: "Group this task belongs to."
readOnly: true
error:
type: string
description: "The error message, if status is 'error'. This is errors due to things outside the task itself. Errors from user code will be found in the log."
reason:
type: string
description: |
Machine usable reason for task being in this state.
Valid values for error status are `timeout | killed | bad_exit`.
Valid values for cancelled status are `client_request`.
For everything else, this is undefined.
enum:
- timeout
- killed
- bad_exit
- client_request
created_at:
type: string
format: date-time
description: Time when task was submitted. Always in UTC.
readOnly: true
started_at:
type: string
format: date-time
description: Time when task started execution. Always in UTC.
completed_at:
type: string
format: date-time
description: Time when task completed, whether it was successul or failed. Always in UTC.
# We maintain a doubly linked list of the retried task to the
# original task.
retry_of:
type: string
description: If this field is set, then this task is a retry of the ID in this field.
readOnly: true
retry_at:
type: string
description: If this field is set, then this task was retried by the task referenced in this field.
readOnly: true
env_vars:
# this is a map: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#model-with-mapdictionary-properties
type: object
description: Env vars for the task. Comes from the ones set on the Group.
additionalProperties:
type: string
type: object
required:
- image
properties:
image:
type: string
description: Name of Docker image to use. This is optional and can be used to override the image defined at the group level.
payload:
type: string
# 256k
# maxLength breaks ruby generator too: https://github.com/treeder/worker_ruby/blob/0aa9236ce5060af3f15758937712973f80dd54fe/lib/iron_titan/models/task.rb#L272
# maxLength: 268435456
description: Payload for the task. This is what you pass into each task to make it do something.
group_name:
type: string
description: "Group this task belongs to."
readOnly: true
error:
type: string
description: "The error message, if status is 'error'. This is errors due to things outside the task itself. Errors from user code will be found in the log."
reason:
type: string
description: |
Machine usable reason for task being in this state.
Valid values for error status are `timeout | killed | bad_exit`.
Valid values for cancelled status are `client_request`.
For everything else, this is undefined.
enum:
- timeout
- killed
- bad_exit
- client_request
created_at:
type: string
format: date-time
description: Time when task was submitted. Always in UTC.
readOnly: true
started_at:
type: string
format: date-time
description: Time when task started execution. Always in UTC.
completed_at:
type: string
format: date-time
description: Time when task completed, whether it was successul or failed. Always in UTC.
# We maintain a doubly linked list of the retried task to the
# original task.
retry_of:
type: string
description: If this field is set, then this task is a retry of the ID in this field.
readOnly: true
retry_at:
type: string
description: If this field is set, then this task was retried by the task referenced in this field.
readOnly: true
env_vars:
# this is a map: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#model-with-mapdictionary-properties
type: object
description: Env vars for the task. Comes from the ones set on the Group.
additionalProperties:
type: string
ErrorBody:
type: object
@@ -624,21 +633,6 @@ definitions:
error:
$ref: '#/definitions/ErrorBody'
NewTask:
type: object
required:
- image
properties:
image:
type: string
description: Name of Docker image to use. This is optional and can be used to override the image defined at the group level.
payload:
type: string
# 256k
# maxLength breaks ruby generator too: https://github.com/treeder/worker_ruby/blob/0aa9236ce5060af3f15758937712973f80dd54fe/lib/iron_titan/models/task.rb#L272
# maxLength: 268435456
description: Payload for the task. This is what you pass into each task to make it do something.
TaskWrapper:
type: object
required:

View File

@@ -87,7 +87,7 @@ You should see it say `Hello Johnny!` now instead of `Hello World!`.
Oracle Functions supports synchronous function calls like we just tried above, and asynchronous for background processing.
Asynchronous function calls are great for tasks that are CPU heavy or take more than a few seconds to complete.
[Asynchronous functions](async.md) are great for tasks that are CPU heavy or take more than a few seconds to complete.
For instance, image processing, video processing, data processing, ETL, etc.
Architecturally, the main difference between synchronous and asynchronous is that requests
to asynchronous functions are put in a queue and executed on upon resource availability so that they do not interfere with the fast synchronous responses required for an API.
@@ -106,6 +106,8 @@ curl -H "Content-Type: application/json" -X POST -d '{
}' http://localhost:8080/v1/apps/myapp/routes
```
or set `type: async` in your `func.yaml`.
Now if you request this route:
```sh