I think there might be a bug in TYK regarding middleware scripts. When the request that is handled in the middleware script contains binary data it seems that the data in the request gets corrupted.
We have a backend API that we wish to publish using TYK. One endpoint of the backend enables user to upload images. The image data is transferred to backend using a PUT HTTP request. Everything works fine until I add a middleware script to the endpoint. The script is the one I took from your documentation and it actually does nothing but some logging: https://tyk.io/tyk-documentation/customise-tyk/javascript-middleware/middleware-scripting-guide/. As soon as the script is added I see already in the Log browser of the TYK Dasboard that the body of the request gets changed. Also backend gives an error response because it is not abled to process the image.
It seems that somehow the binary body of the request gets changed when the request is passed to the middleware script. Is this a known bug? When could I expect to get a fix for it?
The content length is also affected. I add here the beginning ok and not ok requests. The data is copy pasted from the TYK Log Viewer. There you can clearly see the changes to the body.
------ OK REQUEST -----
PUT /apiserver/v1/user/profileimage HTTP/1.1
Host: sally-iam-apimanagement.amersportsdigital.com
User-Agent: okhttp/3.4.1
Connection: close
Content-Length: 79812
Accept-Encoding: gzip
Authorization: Bearer eyAidHlwIjogIkpXVCIsICJraWQiOiAiUDEzc2N0ZXIzZ3dQK2ZveHBLM0ZTVHdRVlFZPSIsICJhbGciOiAiUlMyNTYiIH0.eyAiYXRfaGFzaCI6ICJ3YmhzZUFxMVVVMkxkcTUtNjlfdllnIiwgInN1YiI6ICI4NTUzZmRhNS1hMTA3LTRkNzktOGIzYi1hNDgzNjY2NzA1ODEiLCAiYXVkaXRUcmFja2luZ0lkIjogIjhjODQ5M2FmLTA3YzEtNDY4OS1hOGRiLTY5ZTc2N2YzM2E1Mi0yNjYiLCAiaXNzIjogImh0dHBzOi8vc2FsbHktaWFtLWZyb250ZW5kLTAxLmFtZXJzcG9ydHNkaWdpdGFsLmNvbTo0NDMvc3NvL29hdXRoMi9TVCIsICJ0b2tlbk5hbWUiOiAiaWRfdG9rZW4iLCAibm9uY2UiOiAiYW1lcjEyMzQiLCAiYXVkIjogInN0LWNsaWVudCIsICJvcmcuZm9yZ2Vyb2NrLm9wZW5pZGNvbm5lY3Qub3BzIjogIjdkN2QxYzBjLTAwYjEtNDEzNi1hMDNjLWYzOWZmYjdhNjBmZCIsICJhenAiOiAic3QtY2xpZW50IiwgImF1dGhfdGltZSI6IDE0NzY2ODgyOTAsICJyZWFsbSI6ICIvU1QiLCAiZXhwIjogMTQ3NjY5MTg5MCwgInRva2VuVHlwZSI6ICJKV1RUb2tlbiIsICJpYXQiOiAxNDc2Njg4MjkwIH0.q1iGV8jD18Cznnidf86fDV3kx3iL1nQ0HfGL91MdZgQFh4wBaxRk3uPvMe2QR_-bwbNSUobjuo6hzsTHMosuNk7HQd6otfxmF1hjQmudEsLT-BCd7vN76kdMWFuz-HQV9dPJdRwzqfDyBq1YI6LW1NaGURZHdOmqd9sO4ihaehPNx2LTCqGTTPFcB4Xm4rgDagmh6mICnIZqEOxxZfkrpfOIeN1G-yr1i5ErN0p5J6SBp6LUEoY3TQX6QXI7pOATXi0vNGdXLO2GO_Sh2OwR2eVsd-d7HbdhlcgtEUKkmUNMJheH-qeEOHlGQxE-_eZI4ZIjvqlz79AMuNO95OAmOw
Connection: close
Content-Type: application/octet-stream
Sttauthorization: csnqmh7vf1fqrft0aje6ut6s45s6j5au
X-Forwarded-For: 109.70.165.70
X-Forwarded-Proto: https
X-Real-Ip: 109.70.165.70
I am also pretty sure I found the cause of the problem. In plugins.go line 102 you set the body using string(originalBody). If the body is binary data it gets some string enocoding (probably UTF-8) here. And then you use the same data to the new request body in plugins.go line 163 that is now broken because it is encoded to string along the way.
We have issued a patch for this. The JS middleware can set a IgnoreBody flag, if this flag is set to true, any body transformation will be ignored. You may still set headers or perform additional operations.
In the context of your test request, this means that the original body will be used, and no encoding step will be performed. Sample code:
var samplePreProcessMiddleware = new TykJS.TykMiddleware.NewMiddleware({})
samplePreProcessMiddleware.NewProcessRequest(function(request, session) {
request.IgnoreBody = true
return samplePreProcessMiddleware.ReturnData(request, {})
})
Thank you for a quick fix. We are using TYK via the Docker images. For development environment we use the tyk_quickstart and for production we will use the tyk-hybrid-docker. Could you still verify which of the docker images now contain fix. At least the tykio/tyk-gateway says that it is last updated a month ago so it does not contain the fix? The tykio/tyk-hybrid-docker was updated yesterday. Does it now contain the fix?
The fix is currently only available in hybrid - as this is the easiest platform for us to patch and release. The docker images or the QuickStart do not contain the fix, these are a little behind our overall patch/release cycle.
Our standard deb/yum versions also do not contain the fix yet.