Http: proxy error: context canceled

bitsofinfo 2018-07-19 05:41:26 UTC #1

We see these a fair amount “http: proxy error: context canceled”

This is a golang net lib error, appears to be due to the client prematurely severing the connection prior to the tyk proxy being complete in proxying the request?

Could this be related to a simple timeout?

Any settings in tyk we could tweak to possibly reduce these?

bitsofinfo 2018-07-19 05:46:49 UTC #2

if “proxy_default_timeout” is NOT set, then is this infinite by default?

bitsofinfo 2018-07-19 09:29:04 UTC #3

We’re just seeing a fair amount of these during certain times and cannot track down what it is.

a) client makes call to simple service via Tyk. This backend service who when called directly and not via tyk responds consistently in < 100ms

b) the plugin calls TykGetKeyData (to lookup a session), if session is valid (in this case it is) it proceeds to proxy the request.

c) all of the log statements generated by the plugins for this request all share “Jul 19 09:09:26” (1 second). So I assume the plugins are executing in < 1 second

d) yet the last log statement (from tyk) shows the client timing out… .(the client has a 30s timeout) “http: proxy error: context canceled”. The timestamp on this tyk log statement is the same “Jul 19 09:09:26”

e) again the backend service is responding fine.

A bit lost as to what is causing this, its definitely something in tyk but not sure where to look as the logs are not very helpful

I can reproduce this in a manual call, its like one in every 30 requests has this issue. The backend responds fine, but when I go through tyk this happens every so often

bitsofinfo 2018-07-24 07:00:55 UTC #4

anyone have any thoughts on this?

matiasb 2018-07-30 01:56:24 UTC #5

Hi @bitsofinfo, can you clarify the type of plugins in use?

bitsofinfo 2018-07-30 07:04:38 UTC #6

We are using JSVM plugins.

bitsofinfo 2018-07-30 07:19:33 UTC #7

again for the failed request, you can see the plugin executing (we have logs) in < 1s, and the “context cancelled” statement occurs after executing and within the same time period.

bitsofinfo 2018-07-30 09:24:11 UTC #8

any tips on how to debug this further its becoming a concern

shabeerm 2018-09-12 04:53:23 UTC #9

Hi TYK Team,

We upgraded our gateway & dashboard to the latest TYK versions (gateway to 2.7.0 and dashboard to 1.7.0) last week.

We are also experiencing similar error for some of the API calls.

time=“Sep 12 03:17:05” level=error msg=“http: proxy error: context canceled” api_id=5faa5f3c51XXXXXXXXXXXXXXXXXX9514 org_id=YYYYYYYYYYYYYYYYYYY server_name=zzzzzzzzzz.yyyyyyyyy.net user_id="****e6cc" user_ip=“172.21.XXX.YYY, 10.76.XXX.YYY” user_name=

Could you please let us know the root cause and solution to this issue ?

Thanks

shabeerm 2018-09-25 04:38:48 UTC #10

Hi TYK Team,

Any updates on this ?

We are getting these frequently in our production systems.

Regards,
Shabeer