Using Pump with CE Gateway to export to Prometheus

Hi there, I’m trying to use Pump with the CE Gateway to export gateway metrics to Prometheus. Everything is running in docker and I’m using images for the gateway & pump from dockerhub.

I’ve set “enable_analytics”: true in my tyk.conf, and have a pump.conf as follows:
{
“analytics_storage_type”: “redis”,
“analytics_storage_config”: {
“type”: “redis”,
“host”: “redis”,
“port”: 6379,
“hosts”: null,
“username”: “”,
“password”: “”,
“database”: 0,
“optimisation_max_idle”: 2000,
“optimisation_max_active”: 4000,
“enable_cluster”: false
},
“purge_delay”: 1,
“pumps”: {
“csv”: {
“type”: “csv”,
“meta”: {
“csv_dir”: “./”
}
},
“prometheus”: {
“type”: “prometheus”,
“meta”: {
“listen_address”: “:9090”,
“path”: “/metrics”
}
},
},
“dont_purge_uptime_data”: false
}

For now I’m volume mounting this as /opt/tyk-pump/pump.conf

the redis settings match whats in the tyk.conf.

I can see the /metrics endpoint exposed by the pump, but I don’t see any of the counters / histograms that the pump repo says I should expect to see… i.e.

  • tyk_http_status{code, api}
  • tyk_http_status_per_path{code, api, path, method}
  • tyk_http_status_per_key{code, key}
  • tyk_http_status_per_oauth_client{code, client_id}
  • tyk_latency{type, api}

Any suggestions on what I need to do to get them to appear?

Thanks in advance,

James

Further to my question, I got this working by configuring “dont_purge_uptime_data”: true

However, the tyk_http_status_per_key{code, key} counters are all reporting 0 for all keys, including those which are being actively used - any ideas why this is so? Missing config for the gateway or pump?

Thanks,

James

I am still trying to decide if Tyk is the right api-gateway solution for my company, since prometheus is the core component of our metrics stack, i am interested in knowing how did you solve the issue?

Managed to expose the tyk metrics with the following configuration.

Defined a new job for tyk in prometheus.yml:

  • job_name: ‘tyk’
    static_configs:
    • targets: [‘tyk_pump:9191’]

Added the prometheus configuration to the pump.conf:
“prometheus”: {
“type”: “prometheus”,
“meta”: {
“listen_address”: “:9191”,
“path”: “/metrics”
}
}

The prometheus targets page (http : // localhost:9090/targets) shows the tyk target running on http : // tyk_pump:9191/metrics and the tyk counters / histograms are available.

It takes few seconds for tyk_pump to connect to prometheus and shows on the page http : // localhost:9090/targets that the tyk job is up.
I’m using tyk_pump on docker that’s why the hostname is tyk_pump.