Tyk Pump - Full Response and Request data

Hey Everyone,

I’m trying to set up Tyk Pump to handle analytics, but can’t seem to see the full response body and request data.

Example config :

tyk.conf

{
  "log_level": "debug" ,
  "listen_port": 8080,
  "secret": "352d20ee67be67f6340b4c0605b044b7",
  "template_path": "/opt/tyk-gateway/templates",
  "storage": {
    "type": "redis",
    "host": "tyk-redis",
    "port": 6379
  },
  "enable_analytics": true,
  "analytics_config": {
    "type": "",
    "enable_detailed_recording" :true
  },
  "http_server_options":{
    "use_ssl" : true,
    "ssl_insecure_skip_verify" : false,
    "ssl_certificates":[
      "certificates/example.pem"
    ]
  },
  "health_check": {
    "enable_health_checks": false,
    "health_check_value_timeouts": 60
  }
}

pump.conf

{
  "analytics_storage_config": {
    "type": "redis",
    "host": "tyk-redis",
    "port": 6379
  },
  "log_level":"debug",
  "purge_delay": 5,
  "pumps": {
    "csv": {
      "type": "csv",
      "meta": {
        "csv_dir": "./"
      }
    }
  },
  "dont_purge_uptime_data": true
}

api definition example

{
    "name": "service",
    "use_keyless": true,
    "api_id": "service",
    "org_id": "1",
    "use_mutual_tls_auth" : true,
    "client_certificates" : [
        "certificates/example.pem"
    ],
    "version_data": {
        "not_versioned": true,
        "versions": {
            "Default": {
                "name": "Default",
                "expires": "",
                "use_extended_paths": true,
                "extended_paths": {
                    "ignored": [],
                    "white_list": [],
                    "black_list": []
                }
            }
        }
    },
    "proxy": {
        "listen_path": "/service",
        "target_url": "https://fakestoreapi.com/products",
        "strip_listen_path": true
    },
    "active": true
}

For the raw request and raw response in the above csv, when I decode the base64, all I can see is the headers for the response and request , not the response body.

Part of the body from an example call to gateway : https://localhost:8080/service

[
    {
        "id": 1,
        "title": "Fjallraven - Foldsack No. 1 Backpack, Fits 15 Laptops",
        "price": 109.95,
        "description": "Your perfect pack for everyday use and walks in the forest. Stash your laptop (up to 15 inches) in the padded sleeve, your everyday",
        "category": "men's clothing",
        "image": "https://fakestoreapi.com/img/81fPKd-2AYL._AC_SL1500_.jpg",
        "rating": {
            "rate": 3.9,
            "count": 120
        }
    },
    {
        "id": 2,
        "title": "Mens Casual Premium Slim Fit T-Shirts ",
        "price": 22.3,
        "description": "Slim-fitting style, contrast raglan long sleeve, three-button henley placket, light weight & soft fabric for breathable and comfortable wearing. And Solid stitched shirts with round neck made for durability and a great fit for casual fashion wear and diehard baseball fans. The Henley style round neckline includes a three-button placket.",
        "category": "men's clothing",
        "image": "https://fakestoreapi.com/img/71-3HjGNDUL._AC_SY879._SX._UX._SY._UY_.jpg",
        "rating": {
            "rate": 4.1,
            "count": 259
        }
    },

Is there something missing from the config or is this not possible to view ?

Hi @WrightKD,

Welcome to the community :tada:

These configs look fine. Please confirm your Gateway and Pump versions.

Hey, thanks!

It was definitely a version issue as I was using the latest tag for the docker images.

New Updated versions :

  • tykio/tyk-gateway:v5.0rc1
  • tykio/tyk-pump-docker-pub:v1.8

I can now see that there is extra data in the analytics records but it seems to be encrypted/encoded. Did remove all ssl config as well to test , but still the same encoded data.Is this due to the Content-Encoding , if so can tyk decode on the fly ?

Also tried setting the raw decoded :
image

Hey @WrightKD,

I’ve reproduced and yes, this is due to the Content-Encoding. You’ll need to use use a Response or Analytics plugin to decode/decompress the response.

See a sample response plugin here.

Hi @WrightKD,

Just updating that I’ve opened an internal ticket with the team, to make Tyk to decode this on the fly, without the need for a plugin. For the record, the reference is TT-8892.

Cheers,

1 Like

Hey @Ubong , awsome!

I’ll implement the plugin on my side in the meantime.

Thanks

1 Like