URL rewriter and $tyk_context.headers_HEADERNAME not working?

bitsofinfo · January 26, 2018, 10:54pm

Net of this is that I need the Host header in an Url rewriter

Going off this doc: (with 1.5/2.5 quick start docker compose)

https://tyk.io/docs/concepts/context-variables/

As headers are already exposed to context data, you can also access any header from context variables by using:

    $tyk_context.headers_HEADERNAME

Attempting the following for an API def

                    "url_rewrites": [
                        {
                            "path": "/.*",
                            "method": "POST",
                            "match_pattern": "/.*",
                            "rewrite_to": "https://$tyk_context.headers_HOST/dog",
                            "triggers": []
                        }
                    ],

I’ve tried

$tyk_context.headers_HOST
$tyk_context.headers_Host
$tyk_context.headers_Content-Type
etc
etc

Nothing seems to work.

I have “context variables” enabled for this api

"enable_context_vars": true,

Logs state:

2018-01-26T22:48:50.717731554Z time="Jan 26 22:48:50" level=error msg="http: proxy error: dial tcp: lookup $tyk_context.headers_HOST: no such host"......

But if I do (as a test, not for reality):

                    "url_rewrites": [
                        {
                            "path": "/.*",
                            "method": "POST",
                            "match_pattern": "/.*",
                            "rewrite_to": "https://$tyk_context.remote_addr/dog",
                            "triggers": []
                        }
                    ],

This works fine. i,e. var is being interpolated

2018-01-26T22:52:30.040202154Z time="Jan 26 22:52:30" level=error msg="http: proxy error: dial tcp 172.20.0.1:443: getsockopt: connection refused".....

Doc is wrong or I’m missing something?

bitsofinfo · January 26, 2018, 10:57pm

If I do just $tyk_context.headers

I get

2018-01-26T22:57:03.658504507Z time="Jan 26 22:57:03" level=error msg="Context variable type is not supported: map[string][]string"

bitsofinfo · January 26, 2018, 11:05pm

Found this: tyk/mw_context_vars.go at 28757d933269739d78a7ecee1c6dfa831afb386b · TykTechnologies/tyk · GitHub

Tried out of curiosity: $tyk_context.headers_Content_type

Got:

2018-01-26T23:04:26.784510257Z time="Jan 26 23:04:26" level=error msg="http: proxy error: dial tcp: address $tyk_context.headers_Content_type]:443: unexpected ']'

So I next tried out of curiosity: (casing diff)

$tyk_context.headers_Content_Type

I get:

2018-01-26T23:02:47.460678942Z 2018/01/26 23:02:47 http: panic serving 172.20.0.1:33614: runtime error: invalid memory address or nil pointer dereference
2018-01-26T23:02:47.460724138Z goroutine 667 [running]:
2018-01-26T23:02:47.460741656Z net/http.(*conn).serve.func1(0xc4205086e0)
2018-01-26T23:02:47.460747456Z 	/usr/local/go/src/net/http/server.go:1697 +0xd0
2018-01-26T23:02:47.460752813Z panic(0xd3f5c0, 0x151a200)
2018-01-26T23:02:47.460757490Z 	/usr/local/go/src/runtime/panic.go:491 +0x283
2018-01-26T23:02:47.460772075Z main.(*URLRewriteMiddleware).CheckHostRewrite(0xc420813500, 0xc4204361c0, 0x19, 0xc4204cac30, 0x29, 0xc420199200)
2018-01-26T23:02:47.460778496Z 	/src/github.com/TykTechnologies/tyk/mw_url_rewrite.go:285 +0x6d
2018-01-26T23:02:47.460783990Z main.(*URLRewriteMiddleware).ProcessRequest(0xc420813500, 0x145eb40, 0xc420371260, 0xc420199200, 0x0, 0x0, 0x1d, 0x412506, 0xc4209be910)
2018-01-26T23:02:47.460805142Z 	/src/github.com/TykTechnologies/tyk/mw_url_rewrite.go:308 +0x24d
2018-01-26T23:02:47.460810768Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460815160Z 	/src/github.com/TykTechnologies/tyk/middleware.go:81 +0x5a3
2018-01-26T23:02:47.460819463Z net/http.HandlerFunc.ServeHTTP(0xc420982e00, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460855490Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.460862281Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460866671Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.460870932Z net/http.HandlerFunc.ServeHTTP(0xc420982e40, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460875209Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.460879435Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460883571Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.460887858Z net/http.HandlerFunc.ServeHTTP(0xc420982e80, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460892177Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.460903437Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460907983Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.460912351Z net/http.HandlerFunc.ServeHTTP(0xc420982ec0, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460916777Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.460921137Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460925336Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.460929738Z net/http.HandlerFunc.ServeHTTP(0xc420982f00, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460934011Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.460940636Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460945210Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.460949550Z net/http.HandlerFunc.ServeHTTP(0xc420982f40, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460953907Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.460958110Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460962470Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.460966823Z net/http.HandlerFunc.ServeHTTP(0xc420982f80, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460971132Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.460982638Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460987290Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.460991635Z net/http.HandlerFunc.ServeHTTP(0xc420982fc0, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.460995890Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.461000135Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.461004357Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.461008644Z net/http.HandlerFunc.ServeHTTP(0xc420983000, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.461013889Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.461018089Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.461022256Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.461026540Z net/http.HandlerFunc.ServeHTTP(0xc420983040, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.461030872Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.461041053Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.461046195Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.461050454Z net/http.HandlerFunc.ServeHTTP(0xc420983080, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.461054758Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.461058865Z main.createMiddleware.func1.1(0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.461063069Z 	/src/github.com/TykTechnologies/tyk/middleware.go:95 +0x76d
2018-01-26T23:02:47.461067878Z net/http.HandlerFunc.ServeHTTP(0xc4209830c0, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.461072467Z 	/usr/local/go/src/net/http/server.go:1918 +0x44
2018-01-26T23:02:47.461076845Z github.com/TykTechnologies/tyk/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc4201da360, 0x145eb40, 0xc420371260, 0xc420199200)
2018-01-26T23:02:47.461081462Z 	/src/github.com/TykTechnologies/tyk/vendor/github.com/gorilla/mux/mux.go:150 +0xed
2018-01-26T23:02:47.461086820Z main.mainHandler.ServeHTTP(0x145eb40, 0xc420371260, 0xc420199000)
2018-01-26T23:02:47.461091173Z 	/src/github.com/TykTechnologies/tyk/main.go:1299 +0x73
2018-01-26T23:02:47.461095495Z main.(*mainHandler).ServeHTTP(0x15657c8, 0x145eb40, 0xc420371260, 0xc420199000)
2018-01-26T23:02:47.461099733Z 	<autogenerated>:1 +0x52
2018-01-26T23:02:47.461104056Z net/http.serverHandler.ServeHTTP(0xc4201652b0, 0x145eb40, 0xc420371260, 0xc420199000)
2018-01-26T23:02:47.461161572Z 	/usr/local/go/src/net/http/server.go:2619 +0xb4
2018-01-26T23:02:47.461167655Z net/http.(*conn).serve(0xc4205086e0, 0x145fb40, 0xc420924240)
2018-01-26T23:02:47.461172248Z 	/usr/local/go/src/net/http/server.go:1801 +0x71d
2018-01-26T23:02:47.461188987Z created by net/http.(*Server).Serve
2018-01-26T23:02:47.461194335Z 	/usr/local/go/src/net/http/server.go:2720 +0x288

bitsofinfo · January 26, 2018, 11:15pm

the only way I can get this to work for Host is to get the Host header in JSVM middleware and set it to another header name (like “Myhost”) then this works

$tyk_context.headers_Myhost

Martin · January 27, 2018, 12:23am

It would be $tyk_context.headers_Host - if that isn’t working it might be worth jamming the $tyk_context.headers map into a header so you can take a look at what is available to the proxy as it’s processinng the request.

bitsofinfo · January 27, 2018, 2:24am

Host is available now in middleware plugin, but is not when doing $tyk_context.headers_Host, so I just set my own header in middlware called Myhost (equal to the Host available in middleware) and then $tyk_context.headers_Myhost works

Martin · January 30, 2018, 4:55am

Thanks for the report - perhaps the middleware isn’t rebuilding the request correctly when it creates the outbound version, we’ll investigate internally.

bitsofinfo · January 30, 2018, 10:46pm

Also headers with more than 2 “-” dashes in them don’t seem to work

Reading this, dashes should be replaced with underscores: tyk/mw_context_vars.go at 28757d933269739d78a7ecee1c6dfa831afb386b · TykTechnologies/tyk · GitHub

Given header X-My-Proxyto-Url in a middleware plugin: this DOES NOT work in an url rewrite $tyk_context.headers_X_My_Proxyto_Url

Given header X-My-ProxytoUrl in a middleware plugin: this DOES WORK in an url rewrite $tyk_context.headers_X_My_ProxytoUrl

Martin · February 7, 2018, 9:46pm

We’re investigating this now - it’s quite odd behaviour.

bitsofinfo · February 20, 2018, 3:01pm

Any news on this? We’re sort of in a holding pattern on some development pending for a release in about 20 days. URLRewriter does not support full URI via context variable, panic or 400 bad request · Issue #1453 · TykTechnologies/tyk · GitHub