Commit Graph

54 Commits

Author SHA1 Message Date
Alexander Bransby-Sharples
c5ec0cc41e Add CORS support to fn api (#455)
The Gin middleware is being used if one or more Origins are specified. Default setup for each Origin is as follows:

- GET,POST, PUT, HEAD methods allowed
- Credentials share disabled
- Preflight requests cached for 12 hours

Which are the defaults gin-contrib/cors comes with out of the box.

Gin-cors will return a 403 if it gets a request with an Origin header that isn't on its' list. If no Origin header is specified then it will just return the servers response.

Start fn with CORS enabled:

`API_CORS="http://localhost:4000, http://localhost:3000" make run`
2017-11-16 15:37:26 +00:00
Reed Allman
61b416a9b5 automagic sql db migrations (#461)
* adds migrations

closes #57

migrations only run if the database is not brand new. brand new
databases will contain all the right fields when CREATE TABLE is called,
this is for readability mostly more than efficiency (do not want to have
to go through all of the database migrations to ascertain what columns a table
has). upon startup of a new database, the migrations will be analyzed and the
highest version set, so that future migrations will be run. this should also
avoid running through all the migrations, which could bork db's easily enough
(if the user just exits from impatience, say).

otherwise, all migrations that a db has not yet seen will be run against it
upon startup, this should be seamless to the user whether they had a db that
had 0 migrations run on it before or N. this means users will not have to
explicitly run any migrations on their dbs nor see any errors when we upgrade
the db (so long as things go well). if migrations do not go so well, users
will have to manually repair dbs (this is the intention of the `migrate`
library and it seems sane), this should be rare, and I'm unsure myself how
best to resolve not having gone through this myself, I would assume it will
require running down migrations and then manually updating the migration
field; in any case, docs once one of us has to go through this.

migrations are written to files and checked into version control, and then use
go-bindata to generate those files into go code and compiled in to be consumed
by the migrate library (so that we don't have to put migration files on any
servers) -- this is also in vcs. this seems to work ok. I don't like having to
use the separate go-bindata tool but it wasn't really hard to install and then
go generate takes care of the args. adding migrations should be relatively
rare anyway, but tried to make it pretty painless.

1 migration to add created_at to the route is done here as an example of how
to do migrations, as well as testing these things ;) -- `created_at` will be
`0001-01-01T00:00:00.000Z` for any existing routes after a user runs this
version. could spend the extra time adding 'today's date to any outstanding
records, but that's not really accurate, the main thing is nobody will have to
nuke their db with the migrations in place & we don't have any prod clusters
really to worry about. all future routes will correctly have `created_at` set,
and plan to add other timestamps but wanted to keep this patch as small as
possible so only did routes.created_at.

there are tests that a spankin new db will work as expected as well as a db
after running all down & up migrations works. the latter tests only run on mysql
and postgres, since sqlite3 does not like ALTER TABLE DROP COLUMN; up
migrations will need to be tested manually for sqlite3 only, but in theory if
they are simple and work on postgres and mysql, there is a good likelihood of
success; the new migration from this patch works on sqlite3 fine.

for now, we need to use `github.com/rdallman/migrate` to move forward, as
getting integrated into upstream is proving difficult due to
`github.com/go-sql-driver/mysql` being broken on master (yay dependencies).
Fortunately for us, we vendor a version of the `mysql` bindings that actually
works, thus, we are capable of using the `mattes/migrate` library with success
due to that. this also will require go1.9 to use the new `database/sql.Conn`
type, CI has been updated accordingly.

some doc fixes too from testing.. and of course updated all deps.

anyway, whew. this should let us add fields to the db without busting
everybody's dbs. open to feedback on better ways, but this was overall pretty
simple despite futzing with mysql.

* add migrate pkg to deps, update deps

use rdallman/migrate until we resolve in mattes land

* add README in migrations package

* add ref to mattes lib
2017-11-14 12:54:33 -08:00
Nigel Deakin
ae31944224 Add Prometheus statistics and an example to showcase them using Grafana 2017-10-05 16:21:31 +01:00
James Jeffrey
c7f3066c75 Update references remove refs to treeder oracle funcy (#376)
* Remove lots of refs to iron and funcy oracle etc..

* more ref replacements

* Replacing more refs. Treeder

* Use Fn not FN
2017-09-29 16:22:15 -07:00
Reed Allman
1a55c8050e rollback mysql driver
this https://github.com/go-sql-driver/mysql/pull/302 seems to have pretty much
crippled the ability to use mysql, so we need to lock a version before that
until that issue gets fixed.
2017-09-19 14:14:42 -07:00
Travis Reeder
3aecebdf48 Finally rid of capital Sirupsen?? 2017-09-18 23:33:47 -07:00
Travis Reeder
9f87b1ead0 Client generator update (#262)
* Generated a bunch of clients.

* Builds go client.

* Removed test line.

* Changed dependencies to fnproject/fn_go

* Fix tests.
2017-09-14 10:57:22 -07:00
Reed Allman
27e43c5d94 remove ccirrelo/supervisor, update
everything seems to work even though sirupsen is upper case?

:cyfap:
2017-09-05 11:36:47 -07:00
Travis Reeder
d7bf64bf66 Big dependency update, all lowercase sirupsen's for all dependencies. 2017-08-23 19:52:56 -07:00
Reed Allman
30f3c45dbc update vendor/ dir to latest w/o heroku, moby
had to lock a lot of things in place
2017-08-03 03:52:14 -07:00
Reed Allman
3ff28163db fix task memory
prior to this patch we were allowing 256MB for every function run, just
because that was the default for the docker driver and we were not using the
memory field on any given route configuration. this fixes that, now docker
containers will get the correct memory limit passed into the container from
the route. the default is still 128.

there is also an env var now, `MEMORY_MB` that is set on each function call,
see the linked issue below for rationale.

closes #186

ran the given function code from #186, and now i only see allocations up to
32MB before the function is killed. yay.

notes:

there is no max for memory. for open source fn i'm not sure we want to
cap it, really. in the services repo we probably should add a cap before prod.
since we don't know any given fn server's ram, we can't try to make sure the
setting on any given route is something that can even be run.

remove envconfig & bytefmt

this updates the glide.yaml file to remove the unused deps, but trying to
install fresh is broken atm so i couldn't remove from vendor/, going to fix
separately (next update we just won't get these). also changed the skip dir to
be the cli dir now that its name has changed (related to brokenness).

fix how ram slots were being allocated. integer division is significantly
slower than subtraction.
2017-08-02 19:09:16 -07:00
James
6ee7619b40 Move all endpoints on v1 to be under apps 2017-07-28 07:25:50 -07:00
Travis Reeder
48e3781d5e Rename to GitHub (#3)
* circle

* Rename to github and fn->cli

*  Rename to github and fn->cli
2017-07-26 10:50:19 -07:00
Reed Allman
dc5e67b6d2 add opentracing spans for metrics 2017-07-25 08:55:22 -07:00
Travis Reeder
c3630eaa41 Expiring cache 2017-07-20 08:44:56 -07:00
Travis Reeder
7a81aa5761 Merge branch 'middleware-update' into 'master'
Allow setting values in middleware context like gin context.

See merge request !99
2017-07-12 15:05:13 -07:00
Travis Reeder
1dcc93b32b Updated dependencies 2017-07-12 14:18:01 -07:00
Travis Reeder
98539fba8a Updated deps 2017-07-12 14:08:59 -07:00
Travis Reeder
bd9c9c5202 WIP, setting values in middleware context like gin context. 2017-07-07 16:34:04 -07:00
Reed Allman
4e52c595d2 merge datastores into sqlx package
replace default bolt option with sqlite3 option. the story here is that we
just need a working out of the box solution, and sqlite3 is just fine for that
(actually, likely better than bolt).

with sqlite3 supplanting bolt, we mostly have sql databases. so remove redis
and then we just have one package that has a `sql` implementation of the
`models.Datastore` and lean on sqlx to do query rewriting. this does mean
queries have to be formed a certain way and likely have to be ANSI-SQL (no
special features) but we weren't using them anyway and our base api is
basically done and we can easily extend this api as needed to only implement
certain methods in certain backends if we need to get cute.

* remove bolt & redis datastores (can still use as mqs)
* make sql queries work on all 3 (maybe?)
* remove bolt log store and use sqlite3
* shove the FnLog shit into the datastore shit for now (free pg/mysql logs...
just for demos, etc, not prod)
* fix up the docs to remove bolt references
* add sqlite3, sqlx dep
* fix up tests & mock stuff, make validator less insane
* remove put & get in datastore layer as nobody is using.

this passes tests which at least seem like they test all the different
backends. if we trust our tests then this seems to work great. (tests `make
docker-test-run-with-*` work now too)
2017-07-07 01:30:02 -07:00
Reed Allman
cc0f7cfb14 add sqlx package 2017-07-06 19:46:28 -07:00
Reed Allman
c9c7950d0f add funcy/functions_go & update docker/distribution 2017-06-30 11:22:56 -07:00
Reed Allman
161459192d Id gen suga 2017-06-19 10:40:26 -07:00
Reed Allman
6ee9c1fa0a remove dep files, update glide top to ignore fn/ 2017-06-11 02:03:02 -07:00
Reed Allman
c6e4aa1a63 hack logrus sirupsen name change
:'( not sure how to fix dep tool... punting (maybe we should remove it so that
nobody else will try to use it?)

https://github.com/sirupsen/logrus/issues/553#issuecomment-306591437
2017-06-06 16:45:45 -07:00
Denis Makogon
1a2a805740 Update functions_{lang} tool to recent git source 2017-06-06 19:50:33 +03:00
Travis Reeder
9c67c5024f Added glides files just in case. 2017-06-01 14:30:09 -07:00
Travis Reeder
10f3178ae9 Switching to new dep tool (#616)
* making things work

* #506 - Add ability to login to a private docker registry

* Rolling back "make things work" to test them out more.

* Rolling back "make things work" to test them out more.

* credentials from docker/config.json if ENV is missing

* should get docker auth info just in the init

* update glide lock

* update glide

* Switched to new go dep tool, glide is too frikin annoying.

* Updated circle builds to use dep

* Added GOPATH/bin to path.

* Added GOPATH/bin to path.

* Using regular make test, instead of docker one (not sure why it was using the docker one?).
2017-04-07 11:22:08 -07:00
Martin Pinto-Bazurco Mendieta
e4b3105d92 Fix #418 Added MySQL as DB storage layer. (#575)
* Fix #418 Added MySQL as DB storage layer.

* Make the mysql stuff work

* Make the mysql stuff work

* Make the mysql stuff work

* Make the mysql stuff work

* small fixes

* Switch to Go 1.8 installation inside CI (#589)

* Switch to Go 1.8 installation inside CI

Partially Addresses: #588

* Use url.Hostname() instead of custom method

* Added PR review changes.

* Added missing check for error.

* Changed * with name, config

* Removed unused import.

* Added check for NoRows

* Merged changes with HEAD

* Added documentation to mysql.go

* update mysql to be on par with postgres
2017-03-21 20:01:17 +01:00
Travis Reeder
6119f07007 Fixes some route creation and updating bugs. (#526)
* Fixes some route creation and updating bugs.

* Updated README

* Added more info the quickstart

* Updated based on PR comments.

* Fixed based on comments.

* Updated per comments.
2017-02-24 16:04:04 -08:00
Seif Lotfy سيف لطفي
ee398f0d7d Add initial load balancer (#487)
* lb: library for creation of load balancer

* lb: library for creation of load balancer

* Add balance subcommand to fn

* make fnlb its own command

* Update Changelogg

* Add Makefile for fnlb
2017-01-19 03:11:39 +01:00
C Cirello
1dc3145045 functions: upgrade runner to latest (#434)
* functions: upgrade runner

* functions: update to latest runner

Supercedes and fixes #433
2016-12-14 00:10:24 +01:00
C Cirello
4bb32abba8 functions: upgrade runner (#385) 2016-12-05 20:40:08 +01:00
Pedro Nasser
d98d54b1a8 remove unnecessary dep (#323)
Remove unnecessary dependency from glide
2016-11-20 13:05:22 -02:00
C Cirello
6b7886721d Fix supervisor dependency (#167)
Fixes #164
2016-10-14 13:17:34 +02:00
C Cirello
df3d5b48ce Fix race condition during initialization (#163)
Currently, async workers are started before HTTP interface is available
to get their requests. It fixes by ensuring that async workers are
started after HTTP interface is up.

Essentially we are getting rid of an error message during bootstrap:

        ERRO[0000] Could not fetch task error=Get http://127.0.0.1:8080/tasks: dial tcp 127.0.0.1:8080: getsockopt: connection refused
2016-10-13 22:56:34 +02:00
Carlos C
bbdbcbfa94 remove wrok runner version from glide.yaml 2016-10-12 20:05:57 +02:00
Travis Reeder
042bccb2e6 Fixes issues with dependencies. 2016-10-11 18:18:06 -07:00
Pedro Nasser
52f78eb601 fix runner changes (#132)
Fix runner changes
2016-10-07 18:49:16 +02:00
Seif Lotfy سيف لطفي
52cab30056 Change PAYLOAD input to STDIN (#111)
* change to iron-io/runner dependency
* Fix runner dependency
* Change PAYLOAD input to STDIN, fixes #40
2016-10-06 18:44:58 -03:00
Seif Lotfy سيف لطفي
fbcec6bf40 Depend on iron-io/runner instead of iron-io/worker (#124) 2016-10-05 20:42:12 +02:00
Seif Lotfy
54f66c7b09 update glide 2016-09-26 12:59:18 +02:00
Seif Lotfy
b623fc27e4 Initial work on async functions 2016-09-24 10:06:51 +02:00
Henrique Chehad
06294b4b77 updated worker repository ref 2016-09-19 20:41:35 -03:00
Pedro Nasser
f0e03fb7ec update glide and lock 2016-09-12 13:07:18 -03:00
Pedro Nasser
c80ec456ce update glide 2016-09-12 12:50:32 -03:00
Pedro Nasser
6661fd85b2 update glide and glide.lock 2016-09-12 12:43:37 -03:00
Pedro Nasser
7be05cdf37 update iron worker repo 2016-09-12 12:27:59 -03:00
Pedro Nasser
591aafc21b trigger ci 2016-09-12 12:23:27 -03:00
Pedro Nasser
13cb2d5998 trigger ci 2016-09-12 12:17:59 -03:00