mirror of
https://github.com/fnproject/fn.git
synced 2022-10-28 21:29:17 +03:00
S3 loggyloo (#511)
* add minio-go dep, update deps * add minio s3 client minio has an s3 compatible api and is an open source project and, notably, is not amazon, so it seems best to use their client (fwiw the aws-sdk-go is a giant hair ball of things we don't need, too). it was pretty easy and seems to work, so rolling with it. also, minio is a totally feasible option for fn installs in prod / for demos / for local. * adds 's3' package for s3 compatible log storage api, for use with storing logs from calls and retrieving them. * removes DELETE /v1/apps/:app/calls/:call/log endpoint * removes internal log deletion api * changes the GetLog API to use an io.Reader, which is a backwards step atm due to the json api for logs, I have another branch lined up to make a plain text log API and this will be much more efficient (also want to gzip) * hooked up minio to the test suite and fixed up the test suite * add how to run minio docs and point fn at it docs some notes: notably we aren't cleaning up these logs. there is a ticket already to make a Mr. Clean who wakes up periodically and nukes old stuff, so am punting any api design around some kind of TTL deletion of logs. there are a lot of options really for Mr. Clean, we can notably defer to him when apps are deleted, too, so that app deletion is fast and then Mr. Clean will just clean them up later (seems like a good option). have not tested against BMC object store, which has an s3 compatible API. but in theory it 'just works' (the reason for doing this). in any event, that's part of the service land to figure out. closes #481 closes #473 * add log not found error to minio land
This commit is contained in:
@@ -10,7 +10,11 @@ docker run -e "LOGSTORE_URL=sqlite3:///functions/logs/fn.db" ...
|
||||
settings `LOGSTORE_URL` to `DB_URL` will put logs in the same database as
|
||||
other data, this is not recommended for production.
|
||||
|
||||
## sqlite3 (default)
|
||||
## sqlite3 / postgres / mysql (default)
|
||||
|
||||
NOTE: if you leave LOGSTORE_URL empty, it will default to DB_URL. this is
|
||||
recommended if you are not using a separate place for your logs for connection
|
||||
pooling reasons.
|
||||
|
||||
example URL: `sqlite3:///functions/logs/fn.db`
|
||||
|
||||
@@ -18,3 +22,19 @@ sqlite3 is an embedded database which stores to disk. If you want to use this, b
|
||||
the directory on your host. eg: `docker run -v $PWD/data:/functions/data -e LOGSTORE_URL=sqlite3:///functions/data/fn.db ...`
|
||||
|
||||
sqlite3 isn't recommended for production environments
|
||||
|
||||
## minio / s3
|
||||
|
||||
If you have an s3-compatible object store, we are using only `put_object` and
|
||||
`get_object` and you may point `LOGSTORE_URL` at that api's url appropriately.
|
||||
If you don't have one of those running, you may run minio, an example is
|
||||
below:
|
||||
|
||||
```sh
|
||||
$ docker run -d -p 9000:9000 --name minio -e "MINIO_ACCESS_KEY=admin" -e "MINIO_SECRET_KEY=password" minio/minio server /data
|
||||
$ docker run --privileged --link minio -e "LOGSTORE_URL=s3://admin:password@minio:9000/us-east-1/fnlogs" fnproject/functions:latest
|
||||
```
|
||||
|
||||
you may include any other necessary args for fnproject, this example only
|
||||
illustrates running a minio server and the required args for a functions
|
||||
server to use it.
|
||||
|
||||
Reference in New Issue
Block a user