mirror of
https://github.com/fnproject/fn.git
synced 2022-10-28 21:29:17 +03:00
61b416a9b5
* adds migrations closes #57 migrations only run if the database is not brand new. brand new databases will contain all the right fields when CREATE TABLE is called, this is for readability mostly more than efficiency (do not want to have to go through all of the database migrations to ascertain what columns a table has). upon startup of a new database, the migrations will be analyzed and the highest version set, so that future migrations will be run. this should also avoid running through all the migrations, which could bork db's easily enough (if the user just exits from impatience, say). otherwise, all migrations that a db has not yet seen will be run against it upon startup, this should be seamless to the user whether they had a db that had 0 migrations run on it before or N. this means users will not have to explicitly run any migrations on their dbs nor see any errors when we upgrade the db (so long as things go well). if migrations do not go so well, users will have to manually repair dbs (this is the intention of the `migrate` library and it seems sane), this should be rare, and I'm unsure myself how best to resolve not having gone through this myself, I would assume it will require running down migrations and then manually updating the migration field; in any case, docs once one of us has to go through this. migrations are written to files and checked into version control, and then use go-bindata to generate those files into go code and compiled in to be consumed by the migrate library (so that we don't have to put migration files on any servers) -- this is also in vcs. this seems to work ok. I don't like having to use the separate go-bindata tool but it wasn't really hard to install and then go generate takes care of the args. adding migrations should be relatively rare anyway, but tried to make it pretty painless. 1 migration to add created_at to the route is done here as an example of how to do migrations, as well as testing these things ;) -- `created_at` will be `0001-01-01T00:00:00.000Z` for any existing routes after a user runs this version. could spend the extra time adding 'today's date to any outstanding records, but that's not really accurate, the main thing is nobody will have to nuke their db with the migrations in place & we don't have any prod clusters really to worry about. all future routes will correctly have `created_at` set, and plan to add other timestamps but wanted to keep this patch as small as possible so only did routes.created_at. there are tests that a spankin new db will work as expected as well as a db after running all down & up migrations works. the latter tests only run on mysql and postgres, since sqlite3 does not like ALTER TABLE DROP COLUMN; up migrations will need to be tested manually for sqlite3 only, but in theory if they are simple and work on postgres and mysql, there is a good likelihood of success; the new migration from this patch works on sqlite3 fine. for now, we need to use `github.com/rdallman/migrate` to move forward, as getting integrated into upstream is proving difficult due to `github.com/go-sql-driver/mysql` being broken on master (yay dependencies). Fortunately for us, we vendor a version of the `mysql` bindings that actually works, thus, we are capable of using the `mattes/migrate` library with success due to that. this also will require go1.9 to use the new `database/sql.Conn` type, CI has been updated accordingly. some doc fixes too from testing.. and of course updated all deps. anyway, whew. this should let us add fields to the db without busting everybody's dbs. open to feedback on better ways, but this was overall pretty simple despite futzing with mysql. * add migrate pkg to deps, update deps use rdallman/migrate until we resolve in mattes land * add README in migrations package * add ref to mattes lib
68 lines
3.4 KiB
Markdown
68 lines
3.4 KiB
Markdown
# FAQ
|
|
|
|
#### How is the code base structured?
|
|
```
|
|
/ package migrate (the heart of everything)
|
|
/cli the CLI wrapper
|
|
/database database driver and sub directories have the actual driver implementations
|
|
/source source driver and sub directories have the actual driver implementations
|
|
```
|
|
|
|
#### Why is there no `source/driver.go:Last()`?
|
|
It's not needed. And unless the source has a "native" way to read a directory in reversed order,
|
|
it might be expensive to do a full directory scan in order to get the last element.
|
|
|
|
#### What is a NilMigration? NilVersion?
|
|
NilMigration defines a migration without a body. NilVersion is defined as const -1.
|
|
|
|
#### What is the difference between uint(version) and int(targetVersion)?
|
|
version refers to an existing migration version coming from a source and therefor can never be negative.
|
|
targetVersion can either be a version OR represent a NilVersion, which equals -1.
|
|
|
|
#### What's the difference between Next/Previous and Up/Down?
|
|
```
|
|
1_first_migration.up.extension next -> 2_second_migration.up.extension ...
|
|
1_first_migration.down.extension <- previous 2_second_migration.down.extension ...
|
|
```
|
|
|
|
#### Why two separate files (up and down) for a migration?
|
|
It makes all of our lives easier. No new markup/syntax to learn for users
|
|
and existing database utility tools continue to work as expected.
|
|
|
|
#### How many migrations can migrate handle?
|
|
Whatever the maximum positive signed integer value is for your platform.
|
|
For 32bit it would be 2,147,483,647 migrations. Migrate only keeps references to
|
|
the currently run and pre-fetched migrations in memory. Please note that some
|
|
source drivers need to do build a full "directory" tree first, which puts some
|
|
heat on the memory consumption.
|
|
|
|
#### Are the table tests in migrate_test.go bloated?
|
|
Yes and no. There are duplicate test cases for sure but they don't hurt here. In fact
|
|
the tests are very visual now and might help new users understand expected behaviors quickly.
|
|
Migrate from version x to y and y is the last migration? Just check out the test for
|
|
that particular case and know what's going on instantly.
|
|
|
|
#### What is Docker being used for?
|
|
Only for testing. See [testing/docker.go](testing/docker.go)
|
|
|
|
#### Why not just use docker-compose?
|
|
It doesn't give us enough runtime control for testing. We want to be able to bring up containers fast
|
|
and whenever we want, not just once at the beginning of all tests.
|
|
|
|
#### Can I maintain my driver in my own repository?
|
|
Yes, technically thats possible. We want to encourage you to contribute your driver to this respository though.
|
|
The driver's functionality is dictated by migrate's interfaces. That means there should really
|
|
just be one driver for a database/ source. We want to prevent a future where several drivers doing the exact same thing,
|
|
just implemented a bit differently, co-exist somewhere on Github. If users have to do research first to find the
|
|
"best" available driver for a database in order to get started, we would have failed as an open source community.
|
|
|
|
#### Can I mix multiple sources during a batch of migrations?
|
|
No.
|
|
|
|
#### What does "dirty" database mean?
|
|
Before a migration runs, each database sets a dirty flag. Execution stops if a migration fails and the dirty state persists,
|
|
which prevents attempts to run more migrations on top of a failed migration. You need to manually fix the error
|
|
and then "force" the expected version.
|
|
|
|
|