the mqs are storing a models.Task, which was not incorporating all the fields
that are in a task.Config. I would very much like to merge these two things,
but expect to do this in a future restructuring as both are used widely and
not cordoned off properly (Config has a channel, stdin, stdout, stderr -- and
isn't just a 'config', so to speak, as Task is).
Since a task.Config is what is used to actually run a container, the result of
the aforementioned deficiency was #193 where tasks are improperly configured
and ran (namely, memory wrong).
async tasks can still not be hot, they will be reverted to default format.
would also like to fix this (also part of restructuring). I actually started
doing this, hence the changes to those files (the surface area of the change
is small and discourages improper future use, so I've left what I've done).
this will:
closes#193closes#195closes#154
removes many unused fields in models.Task, since we have not implemented
retries. priority & delay are left, even though they are not used either,
the main goal of this is to resolve#193 and both these fields are strongly
plumbed into all the mqs, so punting on those two.
the docker one actually sucks to use, as apparent here. had to add a lot of
shit just for it to work, but the heroku registry client has gone stale and is
causing issues with our dependencies. didn't expect this to be that hard, but
it's over now. good thing we aren't even using this code ^_^. we need to at
some point start limiting image sizes and remember this not being fun to
figure out to begin with, so anyway, now this works and we don't have to
wrestle with different versions of docker in deps (hopefully) anymore.
prior to this patch we were allowing 256MB for every function run, just
because that was the default for the docker driver and we were not using the
memory field on any given route configuration. this fixes that, now docker
containers will get the correct memory limit passed into the container from
the route. the default is still 128.
there is also an env var now, `MEMORY_MB` that is set on each function call,
see the linked issue below for rationale.
closes#186
ran the given function code from #186, and now i only see allocations up to
32MB before the function is killed. yay.
notes:
there is no max for memory. for open source fn i'm not sure we want to
cap it, really. in the services repo we probably should add a cap before prod.
since we don't know any given fn server's ram, we can't try to make sure the
setting on any given route is something that can even be run.
remove envconfig & bytefmt
this updates the glide.yaml file to remove the unused deps, but trying to
install fresh is broken atm so i couldn't remove from vendor/, going to fix
separately (next update we just won't get these). also changed the skip dir to
be the cli dir now that its name has changed (related to brokenness).
fix how ram slots were being allocated. integer division is significantly
slower than subtraction.
this changes the behavior of hot containers:
1) we are no longer populating a hot container with all of the env vars from the
first request to start up that hot container. this will only populate the
container with any vars that are defined on the app or route.
2) when env vars are changed on the route or app, we will now start up a new
hot container that contains those changes.
3) fixes a bug where we could have a collision if the image and path name
created one, e.g. `/yo/foo` & `oze/yo:latest` collides with `/yo/fo` and
`ooze/yo:latest` (if all other fields are held constant), since we're
name spacing with app name in theory it would happen to the same user (though
we were relying on a comma delimiter there, not great). now we use NULL bytes
which should be hard to get in through a json api ;) i added a sha1 to keep
the size of the (soon to be very large) map down, i don't expect collisions
but, well, it's a hash function.
a small note that we could add a few things to the hot container that will not
change on a request basis, such as `app_name`, `format` and `route` but it's a
bit pedantic. ultimately, it's confusing imo that we have a different set of
vars in the env and in the request itself for hot, which is unavoidable unless
we choose to omit setting env vars entirely, but it seems to be what the
people want (lmk, people, if otherwise).
