make headers quasi-consistent (#660)

possible breakages:

* `FN_HEADER` on cold are no longer `s/-/_/` -- this is so that cold functions
can rebuild the headers as they were when they came in on the request (fdks,
specifically), there's no guarantee that a reversal `s/_/-/` is the original
header on the request.
* app and route config no longer `s/-/_/` -- it seemed really weird to rewrite
the users config vars on these. should just pass them exactly as is to env.
* headers no longer contain the environment vars (previously, base config; app
config, route config, `FN_PATH`, etc.), these are still available in the
environment.

this gets rid of a lot of the code around headers, specifically the stuff that
shoved everything into headers when constructing a call to begin with. now we
just store the headers separately and add a few things, like FN_CALL_ID to
them, and build a separate 'config' now to store on the call. I thought
'config' was more aptly named, 'env' was confusing, though now 'config' is
exactly what 'base_vars' was, which is only the things being put into the env.
we weren't storing this field in the db, this doesn't break unless there are
messages in a queue from another version, anyway, don't think we're there and
don't expect any breakage for anybody with field name changes.

this makes the configuration stuff pretty straight forward, there's just two
separate buckets of things, and cold just needs to mash them together into the
env, and otherwise hot containers just need to put 'config' in the env, and then
hot format can shove 'headers' in however they'd like. this seems better than
my last idea about making this easier but worse (RIP).

this means:

* headers no longer contain all vars, the set of base vars can only be found
in the environment.
* headers is only the headers from request + call_id, deadline, method, url
* for cold, we simply add the headers to the environment, prepending
`FN_HEADER_` to them, BUT NOT upper casing or `s/-/_/`
* fixes issue where async hot functions would end up with `Fn_header_`
prefixed headers
* removes idea of 'base' vars and 'env'. this was a strange concept. now we just have
'config' which was base vars, and headers, which was base_env+headers; i.e.
they are disjoint now.
* casing for all headers will lean to be `My-Header` style, which should help
with consistency. notable exceptions for cold only are FN_CALL_ID, FN_METHOD,
and FN_REQUEST_URL -- this is simply to avoid breakage, in either hot format
they appear as `Fn_call_id` still.
* removes FN_PARAM stuff
* updated doc with behavior

weird things left:

`Fn_call_id` e.g. isn't a correctly formatted http header, it should likely be
`Fn-Call-Id` but I wanted to live to fight another day on this one, it would
add some breakage.

examples to be posted of each format below

closes #329

api/agent/agent.go

@@ -4,6 +4,7 @@ import (
 	"context"
 	"io"
 	"net/http"
+	"strings"
 	"sync"
 	"time"
 
@@ -513,11 +514,25 @@ func (s *hotSlot) exec(ctx context.Context, call *call) error {
 	// TODO we REALLY need to wait for dispatch to return before conceding our slot
 }
 
+func specialHeader(k string) bool {
+	return k == "Fn_call_id" || k == "Fn_method" || k == "Fn_request_url"
+}
+
 func (a *agent) prepCold(ctx context.Context, call *call, tok ResourceToken, ch chan Slot) {
+	// add additional headers to the config to shove everything into env vars for cold
+	for k, v := range call.Headers {
+		if !specialHeader(k) {
+			k = "FN_HEADER_" + k
+		} else {
+			k = strings.ToUpper(k) // for compat, FN_CALL_ID, etc. in env for cold
+		}
+		call.Config[k] = strings.Join(v, ", ")
+	}
+
 	container := &container{
 		id:      id.New().String(), // XXX we could just let docker generate ids...
 		image:   call.Image,
-		env:     call.EnvVars, // full env
+		env:     map[string]string(call.Config),
 		memory:  call.Memory,
 		timeout: time.Duration(call.Timeout) * time.Second, // this is unnecessary, but in case removal fails...
 		stdin:   call.req.Body,
@@ -577,7 +592,7 @@ func (a *agent) runHot(ctxArg context.Context, call *call, tok ResourceToken) {
 	container := &container{
 		id:     cid, // XXX we could just let docker generate ids...
 		image:  call.Image,
-		env:    call.BaseEnv, // only base env
+		env:    map[string]string(call.Config),
 		memory: call.Memory,
 		stdin:  stdinRead,
 		stdout: stdoutWrite,

api/agent/agent_test.go

@@ -95,14 +95,9 @@ func TestCallConfigurationRequest(t *testing.T) {
 	req.Header.Add("Content-Length", contentLength)
 	req.Header.Add("FN_PATH", "thewrongroute") // ensures that this doesn't leak out, should be overwritten
 
-	// let's assume we got there params from the URL
-	params := make(Params, 0, 2)
-	params = append(params, Param{Key: "YOGURT", Value: "garlic"})
-	params = append(params, Param{Key: "LEGUME", Value: "garbanzo"})
-
 	call, err := a.GetCall(
 		WithWriter(w), // XXX (reed): order matters [for now]
-		FromRequest(appName, path, req, params),
+		FromRequest(appName, path, req),
 	)
 	if err != nil {
 		t.Fatal(err)
@@ -148,7 +143,7 @@ func TestCallConfigurationRequest(t *testing.T) {
 		t.Fatal("GetCall FromRequest should not fill payload, got non-empty payload", model.Payload)
 	}
 
-	expectedBase := map[string]string{
+	expectedConfig := map[string]string{
 		"FN_FORMAT":   format,
 		"FN_APP_NAME": appName,
 		"FN_PATH":     path,
@@ -158,62 +153,29 @@ func TestCallConfigurationRequest(t *testing.T) {
 		"ROUTE_VAR":   "BAR",
 	}
 
-	expectedEnv := make(map[string]string)
+	for k, v := range expectedConfig {
-	for k, v := range expectedBase {
+		if v2 := model.Config[k]; v2 != v {
-		expectedEnv[k] = v
+			t.Fatal("config mismatch", k, v, v2, model.Config)
+		}
+		delete(expectedConfig, k)
 	}
 
-	for k, v := range expectedBase {
+	if len(expectedConfig) > 0 {
-		if v2 := model.BaseEnv[k]; v2 != v {
+		t.Fatal("got extra vars in config set, add me to tests ;)", expectedConfig)
-			t.Fatal("base var mismatch", k, v, v2, model.BaseEnv)
-		}
-		delete(expectedBase, k)
 	}
 
-	if len(expectedBase) > 0 {
-		t.Fatal("got extra vars in base env set, add me to tests ;)", expectedBase)
-	}
-
-	expectedEnv["FN_CALL_ID"] = model.ID
-	expectedEnv["FN_METHOD"] = method
-	expectedEnv["FN_REQUEST_URL"] = url
-
-	// do this before the "real" headers get sucked in cuz they are formatted differently
 	expectedHeaders := make(http.Header)
-	for k, v := range expectedEnv {
+	expectedHeaders.Add("FN_CALL_ID", model.ID)
-		expectedHeaders.Add(k, v)
+	expectedHeaders.Add("FN_METHOD", method)
-	}
+	expectedHeaders.Add("FN_REQUEST_URL", url)
-
-	// from the request headers (look different in env than in req.Header, idk, up to user anger)
-	// req headers down cases things
-	expectedEnv["FN_HEADER_Myrealheader"] = "FOOLORD, FOOPEASANT"
-	expectedEnv["FN_HEADER_Content_Length"] = contentLength
-
-	for k, v := range expectedEnv {
-		if v2 := model.EnvVars[k]; v2 != v {
-			t.Fatal("env var mismatch", k, v, v2, model.EnvVars)
-		}
-		delete(expectedEnv, k)
-	}
-
-	if len(expectedEnv) > 0 {
-		t.Fatal("got extra vars in base env set, add me to tests ;)", expectedBase)
-	}
 
 	expectedHeaders.Add("MYREALHEADER", "FOOLORD")
 	expectedHeaders.Add("MYREALHEADER", "FOOPEASANT")
 	expectedHeaders.Add("Content-Length", contentLength)
 
-	checkExpectedHeaders(t, expectedHeaders, req.Header)
+	checkExpectedHeaders(t, expectedHeaders, model.Headers)
-
-	if w.Header()["Fn_call_id"][0] != model.ID {
-		t.Fatal("response writer should have the call id, or else")
-	}
 
 	// TODO check response writer for route headers
-
-	// TODO idk what param even is or how to get them, but need to test those
-	// TODO we should double check the things we're rewriting defaults of, like type, format, timeout, idle_timeout
 }
 
 func TestCallConfigurationModel(t *testing.T) {
@@ -228,7 +190,7 @@ func TestCallConfigurationModel(t *testing.T) {
 	payload := "payload"
 	typ := "sync"
 	format := "default"
-	env := map[string]string{
+	cfg := models.Config{
 		"FN_FORMAT":   format,
 		"FN_APP_NAME": appName,
 		"FN_PATH":     path,
@@ -236,12 +198,10 @@ func TestCallConfigurationModel(t *testing.T) {
 		"FN_TYPE":     typ,
 		"APP_VAR":     "FOO",
 		"ROUTE_VAR":   "BAR",
-		"DOUBLE_VAR":  "BIZ, BAZ",
 	}
 
 	cm := &models.Call{
-		BaseEnv:     env,
+		Config:      cfg,
-		EnvVars:     env,
 		AppName:     appName,
 		Path:        path,
 		Image:       image,
@@ -266,18 +226,8 @@ func TestCallConfigurationModel(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	// make sure headers seem reasonable
 	req := callI.(*call).req
 
-	// NOTE these are added as is atm, and if the env vars were comma joined
-	// they are not again here comma separated.
-	expectedHeaders := make(http.Header)
-	for k, v := range env {
-		expectedHeaders.Add(k, v)
-	}
-
-	checkExpectedHeaders(t, expectedHeaders, req.Header)
-
 	var b bytes.Buffer
 	io.Copy(&b, req.Body)
 
@@ -300,7 +250,7 @@ func TestAsyncCallHeaders(t *testing.T) {
 	format := "http"
 	contentType := "suberb_type"
 	contentLength := strconv.FormatInt(int64(len(payload)), 10)
-	env := map[string]string{
+	config := map[string]string{
 		"FN_FORMAT":   format,
 		"FN_APP_NAME": appName,
 		"FN_PATH":     path,
@@ -309,14 +259,16 @@ func TestAsyncCallHeaders(t *testing.T) {
 		"APP_VAR":     "FOO",
 		"ROUTE_VAR":   "BAR",
 		"DOUBLE_VAR":  "BIZ, BAZ",
+	}
+	headers := map[string][]string{
 		// FromRequest would insert these from original HTTP request
-		"Fn_header_content_type":   contentType,
+		"Content-Type":   []string{contentType},
-		"Fn_header_content_length": contentLength,
+		"Content-Length": []string{contentLength},
 	}
 
 	cm := &models.Call{
-		BaseEnv:     env,
+		Config:      config,
-		EnvVars:     env,
+		Headers:     headers,
 		AppName:     appName,
 		Path:        path,
 		Image:       image,
@@ -344,14 +296,8 @@ func TestAsyncCallHeaders(t *testing.T) {
 	// make sure headers seem reasonable
 	req := callI.(*call).req
 
-	// NOTE these are added as is atm, and if the env vars were comma joined
-	// they are not again here comma separated.
-	expectedHeaders := make(http.Header)
-	for k, v := range env {
-		expectedHeaders.Add(k, v)
-	}
-
 	// These should be here based on payload length and/or fn_header_* original headers
+	expectedHeaders := make(http.Header)
 	expectedHeaders.Set("Content-Type", contentType)
 	expectedHeaders.Set("Content-Length", strconv.FormatInt(int64(len(payload)), 10))
 
@@ -402,7 +348,7 @@ func TestSubmitError(t *testing.T) {
 	payload := "payload"
 	typ := "sync"
 	format := "default"
-	env := map[string]string{
+	config := map[string]string{
 		"FN_FORMAT":   format,
 		"FN_APP_NAME": appName,
 		"FN_PATH":     path,
@@ -414,8 +360,7 @@ func TestSubmitError(t *testing.T) {
 	}
 
 	cm := &models.Call{
-		BaseEnv:     env,
+		Config:      config,
-		EnvVars:     env,
 		AppName:     appName,
 		Path:        path,
 		Image:       image,

api/agent/call.go

@@ -6,7 +6,6 @@ import (
 	"fmt"
 	"io"
 	"net/http"
-	"strconv"
 	"strings"
 	"time"
 
@@ -63,7 +62,7 @@ func fixupRequestURL(req *http.Request) string {
 	return req.URL.String()
 }
 
-func FromRequest(appName, path string, req *http.Request, params Params) CallOpt {
+func FromRequest(appName, path string, req *http.Request) CallOpt {
 	return func(a *agent, c *call) error {
 		app, err := a.da.GetApp(req.Context(), appName)
 		if err != nil {
@@ -79,62 +78,8 @@ func FromRequest(appName, path string, req *http.Request, params Params) CallOpt
 			route.Format = models.FormatDefault
 		}
 
-		url := fixupRequestURL(req)
 		id := id.New().String()
 
-		// baseVars are the vars on the route & app, not on this specific request [for hot functions]
-		baseVars := make(map[string]string, len(app.Config)+len(route.Config)+3)
-
-		// add app & route config before our standard additions
-		for k, v := range app.Config {
-			k = toEnvName("", k)
-			baseVars[k] = v
-		}
-		for k, v := range route.Config {
-			k = toEnvName("", k)
-			baseVars[k] = v
-		}
-
-		baseVars["FN_FORMAT"] = route.Format
-		baseVars["FN_APP_NAME"] = appName
-		baseVars["FN_PATH"] = route.Path
-		// TODO: might be a good idea to pass in: envVars["FN_BASE_PATH"] = fmt.Sprintf("/r/%s", appName) || "/" if using DNS entries per app
-		baseVars["FN_MEMORY"] = fmt.Sprintf("%d", route.Memory)
-		baseVars["FN_TYPE"] = route.Type
-
-		// envVars contains the full set of env vars, per request + base
-		envVars := make(map[string]string, len(baseVars)+len(params)+len(req.Header)+3)
-
-		for k, v := range baseVars {
-			envVars[k] = v
-		}
-
-		envVars["FN_CALL_ID"] = id
-		envVars["FN_METHOD"] = req.Method
-		envVars["FN_REQUEST_URL"] = url
-
-		headerVars := make(map[string]string, len(req.Header))
-
-		for k, v := range req.Header {
-			if !noOverrideVars(k) { // NOTE if we don't do this, they'll leak in (don't want people relying on this behavior)
-				headerVars[toEnvName("FN_HEADER", k)] = strings.Join(v, ", ")
-			}
-		}
-
-		// add all the env vars we build to the request headers
-		for k, v := range envVars {
-			if noOverrideVars(k) {
-				// overwrite the passed in request headers explicitly with the generated ones
-				req.Header.Set(k, v)
-			} else {
-				req.Header.Add(k, v)
-			}
-		}
-
-		for k, v := range headerVars {
-			envVars[k] = v
-		}
-
 		// TODO this relies on ordering of opts, but tests make sure it works, probably re-plumb/destroy headers
 		// TODO async should probably supply an http.ResponseWriter that records the logs, to attach response headers to
 		if rw, ok := c.w.(http.ResponseWriter); ok {
@@ -147,6 +92,11 @@ func FromRequest(appName, path string, req *http.Request, params Params) CallOpt
 			}
 		}
 
+		// add our per call headers in here
+		req.Header.Set("FN_METHOD", req.Method)
+		req.Header.Set("FN_REQUEST_URL", reqURL(req))
+		req.Header.Set("FN_CALL_ID", id)
+
 		// this ensures that there is an image, path, timeouts, memory, etc are valid.
 		// NOTE: this means assign any changes above into route's fields
 		err = route.Validate()
@@ -167,10 +117,10 @@ func FromRequest(appName, path string, req *http.Request, params Params) CallOpt
 			Timeout:     route.Timeout,
 			IdleTimeout: route.IdleTimeout,
 			Memory:      route.Memory,
-			BaseEnv:     baseVars,
+			Config:      buildConfig(app, route),
-			EnvVars:     envVars,
+			Headers:     req.Header,
 			CreatedAt:   strfmt.DateTime(time.Now()),
-			URL:         url,
+			URL:         reqURL(req),
 			Method:      req.Method,
 		}
 
@@ -179,22 +129,36 @@ func FromRequest(appName, path string, req *http.Request, params Params) CallOpt
 	}
 }
 
-func noOverrideVars(key string) bool {
+func buildConfig(app *models.App, route *models.Route) models.Config {
-	// descrepency in casing b/w req headers and env vars, force matches
+	conf := make(models.Config, 8+len(app.Config)+len(route.Config))
-	return overrideVars[strings.ToUpper(key)]
+	for k, v := range app.Config {
+		conf[k] = v
+	}
+	for k, v := range route.Config {
+		conf[k] = v
 	}
 
-// overrideVars means that the app config, route config or header vars
+	conf["FN_FORMAT"] = route.Format
-// must not overwrite the generated values in call construction.
+	conf["FN_APP_NAME"] = app.Name
-var overrideVars = map[string]bool{
+	conf["FN_PATH"] = route.Path
-	"FN_FORMAT":      true,
+	// TODO: might be a good idea to pass in: "FN_BASE_PATH" = fmt.Sprintf("/r/%s", appName) || "/" if using DNS entries per app
-	"FN_APP_NAME":    true,
+	conf["FN_MEMORY"] = fmt.Sprintf("%d", route.Memory)
-	"FN_PATH":        true,
+	conf["FN_TYPE"] = route.Type
-	"FN_MEMORY":      true,
+	return conf
-	"FN_TYPE":        true,
+}
-	"FN_CALL_ID":     true,
+
-	"FN_METHOD":      true,
+func reqURL(req *http.Request) string {
-	"FN_REQUEST_URL": true,
+	if req.URL.Scheme == "" {
+		if req.TLS == nil {
+			req.URL.Scheme = "http"
+		} else {
+			req.URL.Scheme = "https"
+		}
+	}
+	if req.URL.Host == "" {
+		req.URL.Host = req.Host
+	}
+	return req.URL.String()
 }
 
 // TODO this currently relies on FromRequest having happened before to create the model
@@ -208,31 +172,7 @@ func FromModel(mCall *models.Call) CallOpt {
 		if err != nil {
 			return err
 		}
-
+		req.Header = c.Headers
-		// HACK: only applies to async below, for async we need to restore
-		// content-length and content-type of the original request, which are
-		// derived from Payload and original content-type which now is in
-		// Fn_header_content_type
-		if c.Type == models.TypeAsync {
-			// Hoist original request content type into async request
-			if req.Header.Get("Content-Type") == "" {
-				content_type, ok := c.EnvVars["Fn_header_content_type"]
-				if ok {
-					req.Header.Set("Content-Type", content_type)
-				}
-			}
-
-			// Ensure content-length in async requests for protocol/http DumpRequestTo()
-			if req.ContentLength == -1 || req.Header.Get("Content-Length") == "" {
-				req.ContentLength = int64(len(c.Payload))
-				req.Header.Set("Content-Length", strconv.FormatInt(int64(len(c.Payload)), 10))
-			}
-		}
-
-		for k, v := range c.EnvVars {
-			// TODO if we don't store env as []string headers are messed up
-			req.Header.Set(k, v)
-		}
 
 		c.req = req
 		// TODO anything else really?
@@ -250,7 +190,6 @@ func WithWriter(w io.Writer) CallOpt {
 
 // GetCall builds a Call that can be used to submit jobs to the agent.
 //
-// TODO we could make this package level just moving the cache around. meh.
 // TODO where to put this? async and sync both call this
 func (a *agent) GetCall(opts ...CallOpt) (Call, error) {
 	var c call
@@ -291,7 +230,13 @@ func (a *agent) GetCall(opts ...CallOpt) (Call, error) {
 	c.execDeadline = execDeadline
 
 	execDeadlineStr := strfmt.DateTime(execDeadline).String()
-	c.EnvVars["FN_DEADLINE"] = execDeadlineStr
+
+	// these 2 headers buckets are the same but for posterity!
+	if c.Headers == nil {
+		c.Headers = make(http.Header)
+		c.req.Header = c.Headers
+	}
+	c.Headers.Set("FN_DEADLINE", execDeadlineStr)
 	c.req.Header.Set("FN_DEADLINE", execDeadlineStr)
 
 	return &c, nil
@@ -386,11 +331,3 @@ func (c *call) End(ctx context.Context, errIn error) error {
 
 	return errIn // original error, important for use in sync call returns
 }
-
-func toEnvName(envtype, name string) string {
-	name = strings.Replace(name, "-", "_", -1)
-	if envtype == "" {
-		return name
-	}
-	return fmt.Sprintf("%s_%s", envtype, name)
-}

api/agent/protocol/http.go

@@ -21,13 +21,14 @@ type HTTPProtocol struct {
 
 func (p *HTTPProtocol) IsStreamable() bool { return true }
 
-// this is just an http.Handler really
 // TODO handle req.Context better with io.Copy. io.Copy could push us
 // over the timeout.
-// TODO maybe we should take io.Writer, io.Reader but then we have to
-// dump the request to a buffer again :(
 func (h *HTTPProtocol) Dispatch(ctx context.Context, ci CallInfo, w io.Writer) error {
-	err := DumpRequestTo(h.in, ci) // TODO timeout
+	req := ci.Request()
+
+	req.RequestURI = ci.RequestURL() // force set to this, for DumpRequestTo to use
+
+	err := DumpRequestTo(h.in, req) // TODO timeout
 	if err != nil {
 		return err
 	}
@@ -70,17 +71,32 @@ func (h *HTTPProtocol) Dispatch(ctx context.Context, ci CallInfo, w io.Writer) e
 // the body in the process.
 //
 // TODO we should support h2!
-func DumpRequestTo(w io.Writer, ci CallInfo) error {
+func DumpRequestTo(w io.Writer, req *http.Request) error {
+	// By default, print out the unmodified req.RequestURI, which
+	// is always set for incoming server requests. But because we
+	// previously used req.URL.RequestURI and the docs weren't
+	// always so clear about when to use DumpRequest vs
+	// DumpRequestOut, fall back to the old way if the caller
+	// provides a non-server Request.
 
-	req := ci.Request()
+	reqURI := req.RequestURI
-	reqURI := ci.RequestURL()
+	if reqURI == "" {
+		reqURI = req.URL.RequestURI()
+	}
 
 	fmt.Fprintf(w, "%s %s HTTP/%d.%d\r\n", valueOrDefault(req.Method, "GET"),
 		reqURI, req.ProtoMajor, req.ProtoMinor)
 
-	absRequestURI := strings.HasPrefix(reqURI, "http://") || strings.HasPrefix(reqURI, "https://")
+	absRequestURI := strings.HasPrefix(req.RequestURI, "http://") || strings.HasPrefix(req.RequestURI, "https://")
-	if !absRequestURI && req.URL.Host != "" {
+	if !absRequestURI {
-		fmt.Fprintf(w, "Host: %s\r\n", req.URL.Host)
+		host := req.Host
+		if host == "" && req.URL != nil {
+			host = req.URL.Host
+		}
+
+		if host != "" {
+			fmt.Fprintf(w, "Host: %s\r\n", host)
+		}
 	}
 
 	chunked := len(req.TransferEncoding) > 0 && req.TransferEncoding[0] == "chunked"

api/agent/protocol/json.go

@@ -18,6 +18,7 @@ type jsonio struct {
 
 // CallRequestHTTP for the protocol that was used by the end user to call this function. We only have HTTP right now.
 type CallRequestHTTP struct {
+	// TODO request method ?
 	Type       string      `json:"type"`
 	RequestURL string      `json:"request_url"`
 	Headers    http.Header `json:"headers"`

api/agent/slots.go

@@ -326,14 +326,14 @@ func getSlotQueueKey(call *call) string {
 	fmt.Fprint(hash, call.Format, "\x00")
 
 	// we have to sort these before printing, yay. TODO do better
-	keys := make([]string, 0, len(call.BaseEnv))
+	keys := make([]string, 0, len(call.Config))
-	for k := range call.BaseEnv {
+	for k := range call.Config {
 		keys = append(keys, k)
 	}
 
 	sort.Strings(keys)
 	for _, k := range keys {
-		fmt.Fprint(hash, k, "\x00", call.BaseEnv[k], "\x00")
+		fmt.Fprint(hash, k, "\x00", call.Config[k], "\x00")
 	}
 
 	var buf [sha1.Size]byte

api/models/call.go

@@ -1,6 +1,8 @@
 package models
 
 import (
+	"net/http"
+
 	"github.com/fnproject/fn/api/agent/drivers"
 	"github.com/go-openapi/strfmt"
 )
@@ -116,11 +118,11 @@ type Call struct {
 	// Memory is the amount of RAM this call is allocated.
 	Memory uint64 `json:"memory,omitempty" db:"-"`
 
-	// BaseEnv are the env vars for hot containers, not request specific.
+	// Config is the set of configuration variables for the call
-	BaseEnv map[string]string `json:"base_env,omitempty" db:"-"`
+	Config Config `json:"config,omitempty" db:"-"`
 
-	// Env vars for the call. Comes from the ones set on the Route.
+	// Headers are headers from the request that created this call
-	EnvVars map[string]string `json:"env_vars,omitempty" db:"-"`
+	Headers http.Header `json:"headers,omitempty" db:"-"`
 
 	// Time when call completed, whether it was successul or failed. Always in UTC.
 	CompletedAt strfmt.DateTime `json:"completed_at,omitempty" db:"completed_at"`

api/server/calls_test.go

@@ -23,20 +23,6 @@ func TestCallGet(t *testing.T) {
 		ID:      id.New().String(),
 		AppName: "myapp",
 		Path:    "/thisisatest",
-		Image:   "fnproject/hello",
-		// Delay: 0,
-		Type:   "sync",
-		Format: "default",
-		// Payload: TODO,
-		Priority:    new(int32), // TODO this is crucial, apparently
-		Timeout:     30,
-		IdleTimeout: 30,
-		Memory:      256,
-		BaseEnv:     map[string]string{"YO": "DAWG"},
-		EnvVars:     map[string]string{"YO": "DAWG"},
-		CreatedAt:   strfmt.DateTime(time.Now()),
-		URL:         "http://localhost:8080/r/myapp/thisisatest",
-		Method:      "GET",
 	}
 
 	rnr, cancel := testRunner(t)
@@ -90,20 +76,6 @@ func TestCallList(t *testing.T) {
 		ID:      id.New().String(),
 		AppName: "myapp",
 		Path:    "/thisisatest",
-		Image:   "fnproject/hello",
-		// Delay: 0,
-		Type:   "sync",
-		Format: "default",
-		// Payload: TODO,
-		Priority:    new(int32), // TODO this is crucial, apparently
-		Timeout:     30,
-		IdleTimeout: 30,
-		Memory:      256,
-		BaseEnv:     map[string]string{"YO": "DAWG"},
-		EnvVars:     map[string]string{"YO": "DAWG"},
-		CreatedAt:   strfmt.DateTime(time.Now()),
-		URL:         "http://localhost:8080/r/myapp/thisisatest",
-		Method:      "GET",
 	}
 	c2 := *call
 	c3 := *call

api/server/runner.go

@@ -50,16 +50,6 @@ func (s *Server) handleFunctionCall2(c *gin.Context) error {
 	return s.serve(c, a, path.Clean(p))
 }
 
-// convert gin.Params to agent.Params to avoid introducing gin
-// dependency to agent
-func parseParams(params gin.Params) agent.Params {
-	out := make(agent.Params, 0, len(params))
-	for _, val := range params {
-		out = append(out, agent.Param{Key: val.Key, Value: val.Value})
-	}
-	return out
-}
-
 // TODO it would be nice if we could make this have nothing to do with the gin.Context but meh
 // TODO make async store an *http.Request? would be sexy until we have different api format...
 func (s *Server) serve(c *gin.Context, appName, path string) error {
@@ -67,7 +57,7 @@ func (s *Server) serve(c *gin.Context, appName, path string) error {
 	// strip params, etc.
 	call, err := s.agent.GetCall(
 		agent.WithWriter(c.Writer), // XXX (reed): order matters [for now]
-		agent.FromRequest(appName, path, c.Request, parseParams(c.Params)),
+		agent.FromRequest(appName, path, c.Request),
 	)
 	if err != nil {
 		return err

docs/function-format.md

@@ -58,9 +58,26 @@ Content-Length: 11
 hello world
 ```
 
-Request HTTP headers contain Default format environment variables listed in [Inputs](writing.md)
+The header keys and values will be populated with information about the
+function call such as the request URL and query parameters, in addition to any
+headers sent in the request to invoke the function itself. The additional
+headers are:
 
-`Content-Length` is determined by the [Content-Length](https://tools.ietf.org/html/rfc7230#section-3.3.3) header, which is mandatory both for input and output. It is used by Functions to know when stop writing to STDIN and reading from STDOUT.
+* `Fn_deadline` - RFC3339 time stamp of the expiration (deadline) date of function execution.
+* `Fn_request_url` - the full URL for the request ([parsing example](https://github.com/fnproject/fn/tree/master/examples/tutorial/params))
+* `Fn_call_id` - a unique ID for each function execution.
+* `Fn_method` - the HTTP method used to invoke
+* `$X` - the HTTP headers that were set for this request, exactly as they were sent in the request.
+
+HTTP Headers will not be populated with app config, route config or any of the
+following, that may be found in the environment instead:
+
+* `FN_APP_NAME`
+* `FN_PATH`
+* `FN_METHOD`
+* `FN_FORMAT`
+* `FN_MEMORY`
+* `FN_TYPE`
 
 #### Pros/Cons
 

docs/writing.md

@@ -27,20 +27,42 @@ db.update(return_struct)
 Inputs are provided through standard input and environment variables. We'll just talk about the default input format here, but you can find others [here](function-format.md).
 To read in the function body, just read from STDIN.
 
-You will also have access to a set of environment variables.
+You will also have access to a set of environment variables, independent of
+the function's format:
 
-* `FN_REQUEST_URL` - the full URL for the request ([parsing example](https://github.com/fnproject/fn/tree/master/examples/tutorial/params))
 * `FN_APP_NAME` - the name of the application that matched this route, eg: `myapp`
 * `FN_PATH` - the matched route, eg: `/hello`
 * `FN_METHOD` - the HTTP method for the request, eg: `GET` or `POST`
-* `FN_CALL_ID` - a unique ID for each function execution.
-* `FN_DEADLINE` - RFC3339 time stamp of the expiration (deadline) date of function execution.
 * `FN_FORMAT` - a string representing one of the [function formats](function-format.md), currently either `default` or `http`. Default is `default`.
 * `FN_MEMORY` - a number representing the amount of memory available to the call, in MB
 * `FN_TYPE` - the type for this call, currently 'sync' or 'async'
+
+Dependent upon the function's format, additional variables that change on a
+per invocation basis will be in a certain location.
+
+For `default` format, these will be in environment variables as well:
+
+* `FN_DEADLINE` - RFC3339 time stamp of the expiration (deadline) date of function execution.
+* `FN_REQUEST_URL` - the full URL for the request ([parsing example](https://github.com/fnproject/fn/tree/master/examples/tutorial/params))
+* `FN_CALL_ID` - a unique ID for each function execution.
+* `FN_METHOD` - http method used to invoke this function
 * `FN_HEADER_$X` - the HTTP headers that were set for this request. Replace $X with the upper cased name of the header and replace dashes in the header with underscores.
-  * `$X` - any [configuration values](https://github.com/fnproject/cli/blob/master/README.md#application-level-configuration) you've set
+  * `$X` - $X is the header that came in the http request that invoked this function.
-  for the Application or the Route. Replace X with the upper cased name of the config variable you set. Ex: `minio_secret=secret` will be exposed via MINIO_SECRET env var.
+
+For `http` format these will be in http headers:
+
+* `Fn_deadline` - RFC3339 time stamp of the expiration (deadline) date of function execution.
+* `Fn_request_url` - the full URL for the request ([parsing example](https://github.com/fnproject/fn/tree/master/examples/tutorial/params))
+* `Fn_call_id` - a unique ID for each function execution.
+* `Fn_method` - the HTTP method used to invoke
+* `$X` - the HTTP headers that were set for this request, exactly as they were sent in the request.
+
+For `json` format, these will be fields in the json object (see
+[format](functions-format.md)):
+
+* `call_id`
+* `protocol: { "headers": { "$X": [ "$Y" ] } }` where `$X:$Y` is each http
+  header exactly as it was sent in the request
 
 Warning: these may change before release.