Transformation and Routing in TYK

Support
, ,
1

I am trying to do transformation and routing in TYK so I decided to write javascript codes in the Virtual Endpoint plugin. In the code, I tried to call an upstream API but while testing on Postman; I got the below error: “error during virtual endpoint execution. contact Administrator for more details”. Then I checked the gateway log for the error details, the error found was “JS middleware timed out after 5seconds”.
In the process of fixing the error, I added hard_timeout in the definition file but the error still persist.
Kindly assist me on how to resolve this error.
Thanks.

2

You can change timeout by setting jsvm_timeout option to bigger value in tyk.conf

3

Thanks for your response Leon. I have added jsvm_timeout to tyk.conf file but it is no longer showing that timeout error. The error I have now is: “Failed to run JS middleware”, though I have jsvm enabled in the tyk.conf file. And I am using virtual endpoint for the javascript code. Below is the javascript code I am using to call the upstream api:

function myAttempt(request, session, config) {
newRequest = {
“Method”: “POST”,
“Body”: request.Body,
“Domain”: “https://api.fcmb.com/fcmb/test/v1/wallet/Customerwallet/new”,

};
//response = TykMakeHttpRequest(newRequest);
response = TykMakeHttpRequest(JSON.stringify(newRequest));
usableResponse = JSON.parse(response);
var bodyObject = JSON.parse(usableResponse.Body);

//return TykJsResponse(bodyObject, session.meta_data)
return TykJsResponse(response, session.meta_data);
}

Is there anything I have done wrong. Kindly assist.
Thanks

4

Try smth like this, to convert response object to TyjJSResponse format:

var responseObject = {
            Body: usableResponse.Body,
            Code: usableResponse.Code,
            Headers: {}
        }

        for (var k in usableResponse.Headers) {
          if (usableResponse.Headers.hasOwnProperty(k)) {
            responseObject.Headers[k] = usableResponse.Headers[k]
          }
        }

        return TykJsResponse(responseObject, {})
5

Thanks Leon. I have added the code to the virtual endpoint plugin but the error: “Failed to run JS middleware” still persist. Below is a part of my tyk.conf file; kindly check if I have done something wrong:

“hostname”: “”,
“enable_custom_domains”: true,
“proxy_enable_http2”: true,
“enable_jsvm”: true,
“jsvm_timeout”: 120,
“oauth_redirect_uri_separator”: “;”,
“coprocess_options”: {
“enable_coprocess”: true,
“coprocess_grpc_server”: “”,
“python_path_prefix”: “/opt/tyk-gateway”
},
“enable_bundle_downloader”: true,
“bundle_base_url”: “http://.host.example.com:8080”,

Also, below is the modified javascript in the virtual endpoint plugin:
function myAttempt(request, session, config) {
newRequest = {
“Method”: “POST”,
“Body”: request.Body,
“Headers”: {“x-ibm-client-id”:“94ce312c-4aab-4365-ad2a-aae945990518”},
“Domain”: “https://api.fcmb.com/fcmb/test/v1/wallet/Customerwallet/new”,

};
//response = TykMakeHttpRequest(newRequest);
response = TykMakeHttpRequest(JSON.stringify(newRequest));
usableResponse = JSON.parse(response);
//var bodyObject = JSON.parse(usableResponse.Body);

var responseObject = {
Body: usableResponse.Body,
Code: usableResponse.Code,
Headers: {}
}

    for (var k in usableResponse.Headers) {
      if (usableResponse.Headers.hasOwnProperty(k)) {
        responseObject.Headers[k] = usableResponse.Headers[k]
      }
    }

    return TykJsResponse(responseObject, {})

//return TykJsResponse(bodyObject, session.meta_data)
//return TykJsResponse(response, session.meta_data);
}

Thanks.

6

You should see more errors in Tyk logs, when you get syntax errors or similar.

7

Regarding Tyk log, do you mean gateway log. On the gateway log, there is only one error which is- “failed to run JS middleware”. Or is there any other log asides the gateway log. Also is it that I need to instantiate the JS middleware in the javascript code embedded in the virtual plugin. Kindly assist.

8

Try setting “log_level”: “debug” in tyk.conf, additionally note that error itself can be displayed during API load as well, not actual usage.

9

I have realised that I gave you code with a typo, here is a working VP code:

function myAttempt(request, session, config) {
    newRequest = {
        "Method": "POST",
        "Body": request.Body,
        "Headers": {
            "x-ibm-client-id": "94ce312c-4aab-4365-ad2a-aae945990518"
        },
        "Domain": "https://api.fcmb.com/fcmb/test/v1/wallet/Customerwallet/new",

    };

    response = TykMakeHttpRequest(JSON.stringify(newRequest));
    usableResponse = JSON.parse(response);

    var responseObject = {
        Body: usableResponse.Body,
        Code: usableResponse.Code,
        Headers: {}
    }

    for (var k in usableResponse.Headers) {
        if (usableResponse.Headers.hasOwnProperty(k)) {
            responseObject.Headers[k] = usableResponse.Headers[k][0]
        }
    }

    return TykJsResponse(responseObject, {})
}
10

Thanks very much Leon. The code now works in the virtual endpoint plugin.

11

Dear Leon,

So far, I have been able to develop an api that route between two different endpoints using TYK. I just observed now that the JSON payload on Postman is not hitting the upstream API , thus it is not sending the expected response…

And in other to fix that I parsed request.Body to JSON and then stringify before sending to the upstream API. But the response I got on postman shows that it is not seeing the JSON payload.

Below is my updated virtual endpoint plugin code and the JSON payload:
VP code:
function myAttempt(request, session, config) {
var myReq = JSON.parse(request.Body);
var Req = JSON.stringify(myReq);

newRequest = {

    "Method": "POST",

   // "Body": request.Body,
    
    "Body": Req,
    "Headers": {

        "x-ibm-client-id": "94ce312c-4aab-4365-ad2a-aae945990518"

    },

    "Domain": "https://api.fcmb.com/fcmb/test/v1/wallet/Customerwallet/new",



};



response = TykMakeHttpRequest(JSON.stringify(newRequest));

usableResponse = JSON.parse(response);



var responseObject = {

    Body: usableResponse.Body,

    Code: usableResponse.Code,

    Headers: {}

}

for (var k in usableResponse.Headers) {

    if (usableResponse.Headers.hasOwnProperty(k)) {

        responseObject.Headers[k] = usableResponse.Headers[k][0]

    }

}

return TykJsResponse(responseObject, {})

}

Below is the payload:
{
“customerId”: “0100000005”,
“schemeCode”: “02”,
“walletStatus”: true,
“createdBy”: “oyinda”,
“accountName”: “oyinda”
}
The response gotten on Postman is ‘customerId cannot be empty or null’; of which it has a value in the payload.
The expected response should show that wallet has been created successfully.

Thanks for your prompt support.

12

Feel free to use “log” and “rawlog” functions inside virtual endpoint for the debugging purpose.
like this: “log(JSON.stringify(Req))”

However I don’t quite get why you do:

var myReq = JSON.parse(request.Body);
var Req = JSON.stringify(myReq);

Since it is essentially the same as using request.Body directly (except that it does additional JSON validation).

Hope it helps!

13

You are right. It is the same as request.Body since the data type is string. Thus, newRequest object takes a payload in string form which is passed to Body as shown
“Body”: request.Body.
Then newRequest object is parsed to string to make http request to the upstream api as shown below:
response = TykMakeHttpRequest(JSON.stringify(newRequest)); which is later parsed to JSON object as shown:
usableResponse = JSON.parse(response);
However, I dont know why the payload is not getting to the upstream api as it is seeing empty payload.
I tried using Debugging on api designer, it shows failed to load JS middleware. The response I got on Postman is not the expected response.
Kindly assist.

14

Dear Leon,

While debugging, I noticed that the payload is hitting the upstream api in the below format; after newRequest object variable has been stringify for TykMakeHttpRequest function.
“{“customerId”: “0100000005”, “schemeCode”: “02”, “walletStatus”: true, “createdBy”: “oyinda”, “accountName”: “oyinda”}”

However expected format to hit the upstream api is JSON object as shown:
{“customerId”: “0100000005”, “schemeCode”: “02”, “walletStatus”: true, “createdBy”: “oyinda”, “accountName”: “oyinda”}

But I tried using the TykMakeHttpRequest function without stringify the request object variable (newRequest) because of the JSON payload but it threw an error. Is it possible to use TykMakeHttpRequest without stringifying the request object passed into the function ? or is there an alternative function which sends object to the third party api without stringify, as the third party api I’m routing to expects an object not a string.

Thanks

15

I just tested the following VP endpoint:

function myAttempt(request, session, config) {
    newRequest = {
        "Method": "POST",
        "Body": request.Body,
        "Domain": "https://httpbin.org/post",
    };

    response = TykMakeHttpRequest(JSON.stringify(newRequest));
    usableResponse = JSON.parse(response);

    var responseObject = {
        Body: usableResponse.Body,
        Code: usableResponse.Code,
        Headers: {}
    }

    for (var k in usableResponse.Headers) {
        if (usableResponse.Headers.hasOwnProperty(k)) {
            responseObject.Headers[k] = usableResponse.Headers[k][0]
        }
    }

    return TykJsResponse(responseObject, {})
}

With the following command:

curl -X POST https://<dashboard>/myapi -H "Content-Type: application/json" --data "{\"Status\":\"Ok\"}"

And httpbin returns the following:

{
  "args": {}, 
  "data": "{\"Status\":\"Ok\"}", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip", 
    "Auth-Sig": "Signature keyId=\"asd\",algorithm=\"hmac-sha256\",headers=\"(request-target) accept x-real-ip x-scheme content-length content-type user-agent x-forwarded-for x-forwarded-port x-forwarded-proto connection authorization date\",signature=\"I7L8J1M0W325ku0U4iQ%2BoJEhFPWr7YY6A4uFMihvwts%3D\"", 
    "Authorization": "Signature", 
    "Content-Length": "15", 
    "Content-Type": "application/json", 
    "Date": "Tue, 11 Feb 2020 13:45:37 UTC", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.54.0", 
    "X-Amzn-Trace-Id": "Root=1-5e42b001-b3bbd1ba7e8612a4f2b1955a", 
    "X-Scheme": "https"
  }, 
  "json": {
    "Status": "Ok"
  }, 
  "method": "POST", 
  "origin": "93.185.27.194, 93.185.27.194, 18.223.195.89, 172.31.25.233, 13.59.220.123", 
  "url": "http://httpbin.org/anything"
}

However if i do not send Content-Type header, it gets interpreted as a string.
So maybe this is the same issue you have: try adding content-type header set to application/json.

16

Thanks Leon. I set Content-Type in the header in the virtual endpoint plugin. And it was very helpful.

17

Dear Leon,

I am trying to map parameter in a JSON payload to another parameter in a different JSON payload using virtual endpoint plugin. I have tried using body transform plugin but I don’t know if it works along with virtual endpoint plugin as am getting this error on postman: Bad request. Below is the mapping done with body transform.

image
image634×577 7.34 KB

Kindly assist.

Thanks

18

Body transform will not work with VP.
So you need to embed all this logic to the VP itself.

19

I tried mapping accountToDebit with sourceAccount in virtual endpoint with the code snippet below

var payload = {};
payload.sourceAccount = JSON.parse(request.Body).accountToDebit;
payload.narration = JSON.parse(request.Body).purpose;
newRequest = {

    "Method": "POST",

    "Body": payload }

but It is giving this error: Error during virtual endpoint execution.
Kindly assist.

20

Try like this:

parsedBody = JSON.parse(request.Body)
var payload = {};
payload.sourceAccount = parsedBody.accountToDebit;
payload.narration = parsedBody.purpose;
newRequest = {
    "Method": "POST",
    "Body": payload
}

Additionally gateway debug logs, should show you more info, like why exactly execution failed.