phase 1 sqlx migrator (#825)

code is feature complete in the general sense, with minor TODO left.

this is just a patch with 'migratex' and does not use it for fn's migrations
yet, would like to get feedback prior to doing that.

presenting:

A migration library loosely based on pressly/goose and mattes/migrate design,
that does migrations across a smattering of sql databases by only accepting a
`*sqlx.DB`.

why?

* goose didn't support kindly allowing us to rebind transactions based on a
given db to various dialects or offer oracle support
* goose didn't support locking the db (maybe not needed with tx? it's late..
we may want to lock the whole db eventually?)
* goose requires us to do semi-complex migration to it from mattes/migrate
* mattes has stepped down as migrate maintainer and the project is in flux
* mattes/migrate did not allow us to define migrations in go and rebind to
different dialects, an issue since we need to insert ids in our own format and
can't define this in sql
* neither handled context plumbing and risked issues there for various
reasons (deadlock, etc).
* I think I'm forgetting 1 or 2

in the style of goose, this lets us define `*sqlx.Tx` up and down funcs in go
code, but uses mattes' migration table so we don't need to migrate that and
retains its lock behavior with added tx sugar and less errors. most
importantly, this code is terse, leveraging sqlx to support a lot of sql dbs
(unlike mattes) and we control this. there is one useful TODO to handle
migrations failing at startup more gracefully, in prod stuff like that will be
nice to have. open to discussion of putting in a separate library, the
landscape of go sql migrators is... really something.

TODO make test suite and test against sqlite3, pg, mysql [, oracledb] like we
have for our own unit tests. I'm thinking it's faster to wire up through
there and use our bevy of migrations?
This commit is contained in:
Reed Allman
2018-03-12 10:30:58 -07:00
committed by GitHub
parent 00eed1ef73
commit 96aa2a67ae
2 changed files with 421 additions and 0 deletions

View File

@@ -0,0 +1,84 @@
package migratex
import (
"context"
"errors"
"fmt"
"testing"
"github.com/jmoiron/sqlx"
_ "github.com/mattn/go-sqlite3"
)
const testsqlite3 = "file::memory:?mode=memory&cache=shared"
type tm struct{}
func (t *tm) Up(tx *sqlx.Tx) error {
_, err := tx.Exec(`CREATE TABLE IF NOT EXISTS foo (
bar bigint NOT NULL PRIMARY KEY
)`)
return err
}
func (t *tm) Down(tx *sqlx.Tx) error {
_, err := tx.Exec("DROP TABLE foo")
return err
}
func (t *tm) Version() int64 { return 1 }
func TestMigrateUp(t *testing.T) {
x := new(tm)
db, err := sqlx.Open("sqlite3", testsqlite3)
if err != nil {
t.Fatal(err)
}
ctx := context.Background()
err = tx(ctx, db, func(tx *sqlx.Tx) error {
version, dirty, err := Version(ctx, tx)
if version != NilVersion || err != nil || dirty {
return fmt.Errorf("version err: %v %v", err, dirty)
}
if version != NilVersion {
return errors.New("found existing version in db, nuke it")
}
err = Up(ctx, db, []Migration{x})
if err != nil {
return err
}
version, dirty, err = Version(ctx, tx)
if err != nil || dirty {
return fmt.Errorf("version err: %v %v", err, dirty)
}
if version != x.Version() {
return errors.New("version did not update, migration should have ran.")
}
// make sure the table is there.
// TODO find a db agnostic way of doing this.
// query := db.Rebind(`SELECT foo FROM sqlite_master WHERE type = 'table'`)
query := db.Rebind(`SELECT name FROM sqlite_master where type='table' AND name='foo'`)
var result string
err = db.QueryRowContext(ctx, query).Scan(&result)
if err != nil {
return fmt.Errorf("foo check: %v", err)
}
if result != "foo" {
return fmt.Errorf("migration version worked but migration didn't work: %v", result)
}
return nil
})
if err != nil {
t.Fatalf("bad things happened: %v", err)
}
}