Response transform not working in Hybrid Gateway

Branch/Environment/Version

  • Branch/Version: Master:4.0.3
  • Environment: Hybrid

Describe the bug
REST to SOAP transformation is not working in the hybrid gateway, we are using blob based template to transform requests from JSON to XML that is working and but the response from the SOAP service XML is not converted back to JSON in the response transformer. only when we update the API multiple times without any changes from the dashboard this is working in the hybrid gateway.

Hi @ibuar,

Sorry for the issue with this.

Are you still experiencing it?
Are you using Tyk cloud? for the Dashboard and Hybrid gateway?

Following this guide, I’m not able to reproduce your issue.

Hi @Ubong ,
Yes still facing the issue.

we are using the Tyk Cloud Hybrid.
Only hybrid gateway with the Redis running in our data center.
When we update the API config with request and response transformation using the blob base64 in the Tyk operator, only the request is transformed to XML and the response is not transformed back to JSON.

After that when we click update API button multiple times using the dashboard the hybrid gateway is working with req and res transformation.

is this a sync issue? is there a way to debug the hybrid gateway to list all APIs and configs?

Thanks
Mohamed Ibrahim AR

Hi,

Please, with log_level set to “debug” on the hybrid gateway, send its logs starting from when the API config is updated via Tyk operator up till it successfully transforms response. ie

Tyk operator update → API call (failed response transform) → Dashboard API Update button multiple clicks → API call (successful response transform)

Hi @Ubong

Tyk operator is able to successfully create or update APIs with the transform plugin.
I can able to see the config in the dashboard as well.

It is either the gateway is not pulling the config from the control plane or the transformation is not working on the gateway.

Thanks

Sorry I wasn’t very clear.

I’m asking for the hybrid gateway’s logs. Where these logs will be depends on your setup. Please see here and here for its typical locations.
Please set the gateway’s log level to “debug” and restart it before getting the logs.

The logs will help debug the issue and figure out if and/or why it’s not pulling the config from the control plane like you suspect. The transformation ought to work on the gateway.

Hybrid Gateway logs
time=“Sep 05 09:39:39” level=debug msg=Started api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” mw=VersionCheck org_id=63070d0df5d53500016aa725 origin=$IP path=“/attachments/submit” ts=1662370779129136330

time=“Sep 05 09:39:39” level=debug msg=Finished api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” code=200 mw=VersionCheck ns=130310 org_id=63070d0df5d53500016aa725 origin=$IP path=“/attachments/submit”

time=“Sep 05 09:39:39” level=debug msg=Started api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” mw=RateCheckMW org_id=63070d0df5d53500016aa725 origin=$IP path=“/attachments/submit” ts=1662370779129336170

time=“Sep 05 09:39:39” level=debug msg=Finished api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” code=200 mw=RateCheckMW ns=36890 org_id=63070d0df5d53500016aa725 origin=$IP path=“/attachments/submit”

time=“Sep 05 09:39:39” level=debug msg=Started api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” mw=MiddlewareContextVars org_id=63070d0df5d53500016aa725 origin=$IP path=“/attachments/submit” ts=1662370779129528160

time=“Sep 05 09:39:39” level=debug msg=Finished api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” code=200 mw=MiddlewareContextVars ns=88499 org_id=63070d0df5d53500016aa725 origin=$IP path=“/attachments/submit”

time=“Sep 05 09:39:39” level=debug msg=Started api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” mw=TransformMiddleware org_id=63070d0df5d53500016aa725 origin=$IP path=“/attachments/submit” ts=1662370779129647990

time=“Sep 05 09:39:39” level=debug msg=Finished api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” code=200 mw=TransformMiddleware ns=149510 org_id=63070d0df5d53500016aa725 origin=$IP path=“/attachments/submit”

time=“Sep 05 09:39:39” level=debug msg=Started api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” mw=TransformHeaders org_id=63070d0df5d53500016aa725 origin=$IP path=“/attachments/submit” ts=1662370779129830040

time=“Sep 05 09:39:39” level=debug msg=Finished api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” code=200 mw=TransformHeaders ns=62810 org_id=63070d0df5d53500016aa725 origin=$IP path=“/attachments/submit”

time=“Sep 05 09:39:39” level=debug msg=“Started proxy”

time=“Sep 05 09:39:39” level=debug msg=“Stripping proxy listen path: /attachments/submit”

time=“Sep 05 09:39:39” level=debug msg=“Upstream path is: /”

time=“Sep 05 09:39:39” level=debug msg=Started api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” mw=ReverseProxy org_id=63070d0df5d53500016aa725 ts=1662370779129989620

time=“Sep 05 09:39:39” level=debug msg=“Upstream request URL: /” api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” mw=ReverseProxy org_id=63070d0df5d53500016aa725

time=“Sep 05 09:39:39” level=debug msg=“Outbound request URL: $PROXY_URL” api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” mw=ReverseProxy org_id=63070d0df5d53500016aa725

time=“Sep 05 09:39:39” level=debug msg=Finished api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” mw=ReverseProxy ns=90892312 org_id=63070d0df5d53500016aa725

time=“Sep 05 09:39:39” level=debug msg=“Upstream request took (ms): 91.014922”

time=“Sep 05 09:39:39” level=debug msg=“Checking: 63070d0df5d53500016aa725” api_id=ZGVmYXVsdC9zdWJtaXQtYXR0YWNobWVudC1yZXN0 api_name=“API_NAME” org_id=63070d0df5d53500016aa725

time=“Sep 05 09:39:39” level=debug msg=“Done proxy”

Hi @Ubong

The Hybrid gateway can able to pull the configs because the proxying is working and also request transformation is also working, only the response transformation is not working. but I can see the response transformation enabled in the dashboard. the response is returned as it is from the upstream without the transformation.

Thanks

Hi @Ubong

Found the issue, when setting response transform we need to also set the response_processors for response_body_transform & header_injector.

when we update the API on the dashboard, will it be set by the dashboard API?
because we are using the dashboard API for the Tyk operator as well, it should work from the Tyk operator too.

Thanks