2d8c528b48
* add minio-go dep, update deps * add minio s3 client minio has an s3 compatible api and is an open source project and, notably, is not amazon, so it seems best to use their client (fwiw the aws-sdk-go is a giant hair ball of things we don't need, too). it was pretty easy and seems to work, so rolling with it. also, minio is a totally feasible option for fn installs in prod / for demos / for local. * adds 's3' package for s3 compatible log storage api, for use with storing logs from calls and retrieving them. * removes DELETE /v1/apps/:app/calls/:call/log endpoint * removes internal log deletion api * changes the GetLog API to use an io.Reader, which is a backwards step atm due to the json api for logs, I have another branch lined up to make a plain text log API and this will be much more efficient (also want to gzip) * hooked up minio to the test suite and fixed up the test suite * add how to run minio docs and point fn at it docs some notes: notably we aren't cleaning up these logs. there is a ticket already to make a Mr. Clean who wakes up periodically and nukes old stuff, so am punting any api design around some kind of TTL deletion of logs. there are a lot of options really for Mr. Clean, we can notably defer to him when apps are deleted, too, so that app deletion is fast and then Mr. Clean will just clean them up later (seems like a good option). have not tested against BMC object store, which has an s3 compatible API. but in theory it 'just works' (the reason for doing this). in any event, that's part of the service land to figure out. closes #481 closes #473 * add log not found error to minio land
Function logs
We currently support the following function logs stores and they are passed in
via the LOGSTORE_URL environment variable. For example:
docker run -e "LOGSTORE_URL=sqlite3:///functions/logs/fn.db" ...
settings LOGSTORE_URL to DB_URL will put logs in the same database as
other data, this is not recommended for production.
sqlite3 / postgres / mysql (default)
NOTE: if you leave LOGSTORE_URL empty, it will default to DB_URL. this is recommended if you are not using a separate place for your logs for connection pooling reasons.
example URL: sqlite3:///functions/logs/fn.db
sqlite3 is an embedded database which stores to disk. If you want to use this, be sure you don't lose the data directory by mounting
the directory on your host. eg: docker run -v $PWD/data:/functions/data -e LOGSTORE_URL=sqlite3:///functions/data/fn.db ...
sqlite3 isn't recommended for production environments
minio / s3
If you have an s3-compatible object store, we are using only put_object and
get_object and you may point LOGSTORE_URL at that api's url appropriately.
If you don't have one of those running, you may run minio, an example is
below:
$ docker run -d -p 9000:9000 --name minio -e "MINIO_ACCESS_KEY=admin" -e "MINIO_SECRET_KEY=password" minio/minio server /data
$ docker run --privileged --link minio -e "LOGSTORE_URL=s3://admin:password@minio:9000/us-east-1/fnlogs" fnproject/functions:latest
you may include any other necessary args for fnproject, this example only illustrates running a minio server and the required args for a functions server to use it.