mirror of
https://github.com/fnproject/fn.git
synced 2022-10-28 21:29:17 +03:00
calls, apps, and routes listing were previously returning the entire data set, which just won't scale. this adds pagination with cursoring forward to each of these endpoints (see the [docs](docs/definitions.md)). the patch is really mostly tests, shouldn't be that bad to pick through. some blarble about implementation is in order: calls are sorted by ids but allow searching within certain `created_at` ranges (finally). this is because sorting by `created_at` isn't feasible when combined with paging, as `created_at` is not guaranteed to be unique -- id's are (eliding theoreticals). i.e. on a page boundary, if there are 200 calls with the same `created_at`, providing a `cursor` of that `created_at` will skip over the remaining N calls with that `created_at`. also using id will be better on the index anyway (well, less of them). yay having sortable ids! I can't discern any issues doing this, as even if 200 calls have the same created_at, they will have different ids, and the sort should allow paginating them just fine. ids are also url safe, so the id works as the cursor value just fine. apps and routes are sorted by alphabetical order. as they aren't guaranteed to be url safe, we are base64'ing them in the front end to a url safe format and then returning them, and then base64 decoding them when we get them. this does mean that they can be relatively large if the path/app is long, but if we don't want to add ids then they were going to be pretty big anyway. a bonus that this kind of obscures them. if somebody has better idea on formatting, by all means. notably, we are not using the sql paging facilities, and we are baking our own based on cursors, which ends up being much more efficient for querying longer lists of resources. this also should be easy to implement in other non-sql dbs and the cursoring formats we can change on the fly since we are just exposing them as opaque strings. the front end deals with the base64 / formatting, etc and the back end is taking raw values (strfmt.DateTime or the id for calls). the cursor that is being passed to/by the user is simply the last resource on the previous page, so in theory we don't even need to return it, but it does make it a little easier to use, also, cursor being blank on the last page depends on page full-ness, so sometimes users will get a cursor when there are no results on next page (1/N chance, and it's not really end of world -- actually searching for the next thing would make things more complex). there are ample tests for this behavior. I've turned off all query parameters allowing `LIKE` queries on certain listing endpoints, as we should not expose sql behavior through our API in the event that we end up not using a sql db down the road. I think we should only allow prefix matching, which sql can support as well as other types of databases relatively cheaply, but this is not hooked up here as it didn't 'just work' when I was fiddling with it (can add later, they're unnecessary and weren't wired in before in front end). * remove route listing across apps (unused) * fix panic when doing `/app//`. this is prob possible for other types of endpoints, out of scope here. added a guard in front of all endpoints for this * adds `from_time` and `to_time` query parameters to calls, so you can e.g. list the last hour of tasks. these are not required and default to oldest/newest. * hooked back up the datastore tests to the sql db, only run with sqlite atm, but these are useful, added a lot to them too. * added a bunch of tests to the front end, so pretty sure this all works now. * added to swagger, we'll need to re-gen. also wrote some words about pagination workings, I'm not sure how best to link to these, feedback welcome. * not sure how we want to manage indexes, but we may need to add some (looking at created_at, mostly) * `?route` changed to `?path` in routes listing, to keep consistency with everything else * don't 404 when searching for calls where the route doesn't exist, just return an empty list (it's a query param ffs) closes #141
289 lines
7.2 KiB
Markdown
289 lines
7.2 KiB
Markdown
# API Definitions
|
|
|
|
# Applications
|
|
|
|
Applications are the top level object that groups routes together to create an API.
|
|
|
|
### Creating applications
|
|
|
|
Using `fn`:
|
|
|
|
```sh
|
|
fn apps create --config k1=v1 --config k2=v2 myapp
|
|
```
|
|
|
|
Or using a cURL:
|
|
|
|
```sh
|
|
curl -H "Content-Type: application/json" -X POST -d '{
|
|
"app": {
|
|
"name":"myapp-curl",
|
|
"config": {
|
|
"k1": "v1",
|
|
"k2": "v2"
|
|
}
|
|
}
|
|
}' http://localhost:8080/v1/apps
|
|
```
|
|
|
|
### App Example
|
|
|
|
```json
|
|
{
|
|
"name": "myapp",
|
|
"myconfig": "config"
|
|
}
|
|
```
|
|
|
|
### Properties
|
|
|
|
#### name (string)
|
|
|
|
`name` is a property that references an unique app.
|
|
|
|
App names are immutable. When updating apps with `PATCH` requests, keep in mind that although you
|
|
are able to update an app's configuration set, you cannot really rename it.
|
|
|
|
#### config (object)
|
|
|
|
`config` is a map of values passed to the route runtime in the form of
|
|
environment variables.
|
|
|
|
Note: Route level configuration overrides app level configuration.
|
|
|
|
## Routes
|
|
|
|
### Creating routes
|
|
|
|
Using `fn`:
|
|
|
|
```sh
|
|
fn routes create myapp /path --config k1=v1 --config k2=v2 --image fnproject/hello
|
|
```
|
|
|
|
Or using cURL:
|
|
|
|
```sh
|
|
curl -H "Content-Type: application/json" -X POST -d '{
|
|
"app": {
|
|
"path": "/path",
|
|
"image": "image",
|
|
"config": {
|
|
"k1": "v1",
|
|
"k2": "v2"
|
|
}
|
|
}
|
|
}' http://localhost:8080/v1/apps/myapp/routes
|
|
```
|
|
|
|
### Route Example
|
|
```json
|
|
{
|
|
"path": "/hello",
|
|
"image": "fnproject/hello",
|
|
"type": "sync",
|
|
"memory": 128,
|
|
"config": {
|
|
"key": "value",
|
|
"key2": "value2",
|
|
"keyN": "valueN"
|
|
},
|
|
"headers": {
|
|
"content-type": [
|
|
"text/plain"
|
|
]
|
|
}
|
|
}
|
|
```
|
|
|
|
### Properties
|
|
|
|
#### path (string)
|
|
|
|
Represents a unique `route` in the API and is identified by the property `path` and `app`.
|
|
|
|
Every `route` belongs to an `app`.
|
|
|
|
Note: Route paths are immutable. If you need to change them, the appropriate approach
|
|
is to add a new route with the modified path.
|
|
|
|
#### image (string)
|
|
|
|
`image` is the name or registry URL that references to a valid container image located locally or in a remote registry (if provided any registry address).
|
|
|
|
If no registry is provided and image is not available locally the API will try pull it from a default public registry.
|
|
|
|
#### type (string)
|
|
|
|
Options: `sync` and `async`
|
|
|
|
`type` is defines how the function will be executed. If type is `sync` the request will be hold until the result is ready and flushed.
|
|
|
|
In `async` functions the request will be ended with a `call_id` and the function will be executed in the background.
|
|
|
|
#### memory (number)
|
|
|
|
`memory` defines the amount of memory (in megabytes) required to run this function.
|
|
|
|
#### config (object of string values)
|
|
|
|
`config` is a map of values passed to the route runtime in the form of
|
|
environment variables.
|
|
|
|
Note: Route level configuration overrides app level configuration.
|
|
|
|
#### headers (object of array of string)
|
|
|
|
`header` is a set of headers that will be sent in the function execution response. The header value is an array of strings.
|
|
|
|
#### format (string)
|
|
|
|
`format` defines if the function is running or not in `hot function` mode.
|
|
|
|
To define the function execution as `hot function` you set it as one of the following formats:
|
|
|
|
- `"http"`
|
|
|
|
### 'Hot function' Only Properties
|
|
|
|
This properties are only used if the function is in `hot function` mode
|
|
|
|
|
|
## Calls and their statuses
|
|
|
|
### Sync/Async Call statuses
|
|
|
|
With each function call, no matter would that be sync or async server makes a record of this it.
|
|
While execution async function server returns `call_id`:
|
|
|
|
```json
|
|
{
|
|
"call_id": "f5621e8b-725a-4ba9-8323-b8cdc02ce37e"
|
|
}
|
|
```
|
|
that can be used to track call status using following command:
|
|
|
|
```sh
|
|
|
|
curl -v -X GET ${API_URL}/v1/calls/f5621e8b-725a-4ba9-8323-b8cdc02ce37
|
|
|
|
```
|
|
|
|
```json
|
|
{
|
|
"message": "Successfully loaded call",
|
|
"call": {
|
|
"id": "f5621e8b-725a-4ba9-8323-b8cdc02ce37e",
|
|
"status": "success",
|
|
"completed_at": "2017-06-02T15:31:30.887+03:00",
|
|
"created_at": "2017-06-02T15:31:30.597+03:00",
|
|
"started_at": "2017-06-02T15:31:30.597+03:00",
|
|
"app_name": "newapp",
|
|
"path": "/envsync"
|
|
}
|
|
}
|
|
|
|
```
|
|
|
|
Server response contains timestamps(created, started, completed) and execution status for this call.
|
|
|
|
For sync call `call_id` can be retrieved from HTTP headers:
|
|
```sh
|
|
curl -v localhost:8080/r/newapp/envsync
|
|
* Trying ::1...
|
|
* TCP_NODELAY set
|
|
* Connected to localhost (::1) port 8080 (#0)
|
|
> GET /r/newapp/envsync HTTP/1.1
|
|
> Host: localhost:8080
|
|
> User-Agent: curl/7.51.0
|
|
> Accept: */*
|
|
>
|
|
< HTTP/1.1 200 OK
|
|
< Fn_call_id: f5621e8b-725a-4ba9-8323-b8cdc02ce37e
|
|
< Date: Fri, 02 Jun 2017 12:31:30 GMT
|
|
< Content-Length: 489
|
|
< Content-Type: text/plain; charset=utf-8
|
|
<
|
|
...
|
|
```
|
|
Corresponding HTTP header is `Fn_call_id`.
|
|
|
|
### Per-route calls
|
|
|
|
In order get list of per-route calls please use following command:
|
|
|
|
```sh
|
|
curl -X GET ${API_URL}/v1/app/{app}/calls/{route}
|
|
|
|
```
|
|
Server will replay with following JSON response:
|
|
|
|
```json
|
|
{
|
|
"message": "Successfully listed calls",
|
|
"calls": [
|
|
{
|
|
"id": "80b12325-4c0c-5fc1-b7d3-dccf234b48fc",
|
|
"status": "success",
|
|
"completed_at": "2017-06-02T15:31:22.976+03:00",
|
|
"created_at": "2017-06-02T15:31:22.691+03:00",
|
|
"started_at": "2017-06-02T15:31:22.691+03:00",
|
|
"app_name": "newapp",
|
|
"path": "/envsync"
|
|
},
|
|
{
|
|
"id": "beec888b-3868-59e3-878d-281f6b6f0cbc",
|
|
"status": "success",
|
|
"completed_at": "2017-06-02T15:31:30.887+03:00",
|
|
"created_at": "2017-06-02T15:31:30.597+03:00",
|
|
"started_at": "2017-06-02T15:31:30.597+03:00",
|
|
"app_name": "newapp",
|
|
"path": "/envsync"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
### Pagination
|
|
|
|
The fn api utilizes 'cursoring' to paginate large result sets on endpoints
|
|
that list resources. The parameters are read from query parameters on incoming
|
|
requests, and a cursor will be returned to the user if they receive a full
|
|
page of data to use to retrieve the next page. We'll walk through with a
|
|
concrete example in just a minute.
|
|
|
|
To begin paging through a results set, a user should provide a `?cursor` with an
|
|
empty string or omit the cursor query parameter altogether. A user may specify
|
|
how many results per page they would like to receive with the `?per_page`
|
|
query parameter, which defaults to 30 and has a max of 100. After calling a
|
|
list endpoint, a user may receive a `response.next_cursor` value in the
|
|
response, next to the list of resources. If `next_cursor` is an empty string,
|
|
then there is no further data to retrieve and the user may stop paging. If
|
|
`next_cursor` is a non-empty string, the user may provide it in the next
|
|
request's `?cursor` parameter to receive the next page.
|
|
|
|
briefly, what this means, is user code should look similar to this:
|
|
|
|
```
|
|
req = "http://my.fn.com/v1/apps/"
|
|
cursor = ""
|
|
|
|
for {
|
|
req_with_cursor = req + "?" + cursor
|
|
resp = call_http(req_with_cursor)
|
|
do_things_with_apps(resp["apps"])
|
|
|
|
if resp["next_cursor"] == "" {
|
|
break
|
|
}
|
|
cursor = resp["next_cursor"]
|
|
}
|
|
|
|
# done!
|
|
```
|
|
|
|
client libraries will have variables for each of these variables in their
|
|
respective languages to make this a bit easier, but may the for be with
|
|
you.
|
|
|