Tyk-Gateway/Virtual Endpoint: Request headers vs. Response headers inconsistency?

beyerj · February 27, 2019, 6:45am

Hi,

I am currently evaluating tyk-gateway for a project.

The idea is simple: The frontend (browser) connects to a backend (rest) via tyk-gateway, and tyk-gateway should be the mediator. I set up a virtual endpoint with JS and basically can connect to the backend and send back the response body to the frontend, but now I realized that there seems to be a difference between the Headers structures of request objects and response objects in the JS API.

If I am not mistaken, the Headers seem to be of the following structure:

in request from frontend (response function parameter): map[string][]string
in request to backend (TykMakeHttpRequest): map[string]string
in response from backend (TykMakeHttpRequest): map[string][]string
in response to frontend (TykJsResponse): map[string]string

Thus, questions arise: How can I simply transfer all headers from backend response to my response to frontend? How should I specify multiple headers like Set-Cookie in my response to frontend? I was not able to find documentation about this, and the example also does not discuss this.

Many thanks in advance!

Josh · February 27, 2019, 9:36am

Hi

In all cases the data structures are map[string][]string in the Go backend - you can have a look at mw_js_plugin.go for example which contains some of the data structures.

Have you tried simply copying them across from your response you recieve inside the JS to the response you constuct and pass to TykJsResponse?

beyerj · February 27, 2019, 12:00pm

Hi Josh,

in fact I tried, and it does not work.

var backendRequest = {
	"Method": request.Method,
	"Body": request.Body,
	"Headers": request.Headers,
	"Domain": backendService.Meta.TargetDomain,
	"Resource": request.URL.replace(config.config_data.service_base_path, ""),
	"FormData": request.Params
};
var backendResponse = TykMakeHttpRequest(JSON.stringify(backendRequest));

(while request is the variable from the response function parameter - the request from the frontend to tyk-gateway)

leads to:

level=error msg="JSVM: Failed to deserialise HTTP Request object"

Also,

if I say:

"Headers": {
		"Set-Cookie": "ABC=xyz"
	},

it works. And if I say

"Headers": {
		"Set-Cookie": ["ABC=xyz"]
	},

I get the same error as with copying the Headers object:

level=error msg="JSVM: Failed to deserialise HTTP Request object"

Just to make sure, this is tyk-gateway 2.7.6 from the packages repo at https://packagecloud.io/tyk/tyk-gateway/ubuntu/ on ubuntu bionic.

The example above is getting request header from frontend to the request to backend, the same problem is with getting response headers from backend to response headers to frontend.

PS: I just had a look at https://github.com/TykTechnologies/tyk/blob/master/mw_js_plugin.go :

type TykJSHttpRequest struct {
Method   string
Body     string
Headers  map[string]string
Domain   string
Resource string
FormData map[string]string
}

type TykJSHttpResponse struct {
Code    int
Body    string
Headers map[string][]string

// Make this compatible with BatchReplyUnit
CodeComp    int                 `json:"code"`
BodyComp    string              `json:"body"`
HeadersComp map[string][]string `json:"headers"`
}

beyerj · March 1, 2019, 5:08am

Mhh, no answers? Wondering what I should do now. File a bug on github? I am not sure how “connected” this community forum and the people on this project on github really are.
For me, this is kind of a blocker right now.

Thanks

Yaara · March 6, 2019, 9:58pm

Hi,
Sorry no one could respond earlier. We are swamped at the moment but if you are doing a poc please feel free to contact us directly.

I haven’t checked it myself but have you tried to range over the array and set every header one by one?
I’m almost certain there’s an open ticket for it (have you checked our issues on GH?)
Will ask someone to have a look at this.

beyerj · March 7, 2019, 7:15am

Hi Yaara,

thanks for your answer and taking your time

I had a look at Github then, and did find nothing.
In the mean time, from thoroughly reading the documentation and examples, I realized this is currently probably by design, even if not expressively.

Your answer now lead me back to Github, and I found an enhancement request which I did not see when looking for a bug regarding header incosistency:

github.com/TykTechnologies/tyk

Add "append/addHeaders" functionality to JSVM request object

opened 04:41PM - 06 Mar 18 UTC

bitsofinfo

bug enhancement customer request Priority: Medium

**Do you want to request a *feature* or report a *bug*?** feature/bug **Wh…at is the current behavior?** If I need to set multiple headers in a JSVM plugin based on conditional logic, every call to "request.setHeaders" overwrites the last one. This is especially detrimental if I have a chain of plugins that use request headers to exchange arguments downstream, or headers set by plugin A, that need to be consumed by plugin C (the existence of plugin B that sets headers, wipes out plugin's A headers)... i.e. A(set x)->B(set y)->C(needs x, but missing cause B wiped it, B has no knowledge of A nor should it) ``` request.setHeaders({x:y,a:b}) request.setHeaders({c:d}) Yields headers = "c=d" ``` **What is the expected behavior?** Would just like an "appendHeaders" or "addHeaders" function instead ``` request.addHeaders({x:y,a:b}) request.addHeaders({c:d}) Yields headers = "c=d, x=y, a=b" ``` **Which versions of Tyk affected by this issue? Did this work in previous versions of Tyk?** All current

This is probably the one you were thinking of, as the stated problem is exactly the one I mentioned, only the expected solution is different - I expected consistent header structures in requests and responses, so we could simply copy whole objects over.

Regarding your suggestion to set the headers one by one - tried that, and it only results in one header - I believe the last one won, but I can’t tell for sure anymore.

Regarding our evaluation: the current architecture draft shows that we might not have to add multiple headers somewhere, so this might not be a blocker anymore for now (although I thought otherwise). But thank you very much for your offer to contact you directly this is much appreciated.

I will bookmark the Github issue to see where this is going.

Thanks again!

Yaara · March 7, 2019, 12:54pm

HI,
When you say tried that, and it only results in one header - do you mean one header in case it’s the same header, right?
So you want the ability to have:

x-header-xyz: “my xyz header value”
x-header-xyz: “my xyz header with another value”
but currently you only get the last one “my xyz header with another value”, Is that correct?

beyerj · March 7, 2019, 2:00pm

Hi.

Exactly. Most common case would probably be the “Cookie:” or “Set-Cookie:” Header.
If I want to provide two Cookies to the response to the frontend/user, the browser would expect two headers with the same name “Set-Cookie:”.

Set-Cookie: ABC=abc;path=/
Set-Cookie: XYZ=xyz;path=/

But currently it is not possible to provide that with tyk (at least that I know of).

I am not sure anymore if I currently would get the first or the last one, but it is definitely only one. It probably was the last one. Otherwise, yes, that’s what I want. See also above

To make it easier/simpler to understand, what I want is basically consistency between all request and response objects in JSVM to have headers defined as
map[string][]string.

Thanks