Using Pump with CE Gateway to export to Prometheus

Support Installation and Deployment
, ,
1

Hi there, I’m trying to use Pump with the CE Gateway to export gateway metrics to Prometheus. Everything is running in docker and I’m using images for the gateway & pump from dockerhub.

I’ve set “enable_analytics”: true in my tyk.conf, and have a pump.conf as follows:
{
“analytics_storage_type”: “redis”,
“analytics_storage_config”: {
“type”: “redis”,
“host”: “redis”,
“port”: 6379,
“hosts”: null,
“username”: “”,
“password”: “”,
“database”: 0,
“optimisation_max_idle”: 2000,
“optimisation_max_active”: 4000,
“enable_cluster”: false
},
“purge_delay”: 1,
“pumps”: {
“csv”: {
“type”: “csv”,
“meta”: {
“csv_dir”: “./”
}
},
“prometheus”: {
“type”: “prometheus”,
“meta”: {
“listen_address”: “:9090”,
“path”: “/metrics”
}
},
},
“dont_purge_uptime_data”: false
}

For now I’m volume mounting this as /opt/tyk-pump/pump.conf

the redis settings match whats in the tyk.conf.

I can see the /metrics endpoint exposed by the pump, but I don’t see any of the counters / histograms that the pump repo says I should expect to see… i.e.

  • tyk_http_status{code, api}
  • tyk_http_status_per_path{code, api, path, method}
  • tyk_http_status_per_key{code, key}
  • tyk_http_status_per_oauth_client{code, client_id}
  • tyk_latency{type, api}

Any suggestions on what I need to do to get them to appear?

Thanks in advance,

James

2

Further to my question, I got this working by configuring “dont_purge_uptime_data”: true

However, the tyk_http_status_per_key{code, key} counters are all reporting 0 for all keys, including those which are being actively used - any ideas why this is so? Missing config for the gateway or pump?

Thanks,

James

3

I am still trying to decide if Tyk is the right api-gateway solution for my company, since prometheus is the core component of our metrics stack, i am interested in knowing how did you solve the issue?

4

Managed to expose the tyk metrics with the following configuration.

Defined a new job for tyk in prometheus.yml:

  • job_name: ‘tyk’
    static_configs:
    • targets: [‘tyk_pump:9191’]

Added the prometheus configuration to the pump.conf:
“prometheus”: {
“type”: “prometheus”,
“meta”: {
“listen_address”: “:9191”,
“path”: “/metrics”
}
}

The prometheus targets page (http : // localhost:9090/targets) shows the tyk target running on http : // tyk_pump:9191/metrics and the tyk counters / histograms are available.

It takes few seconds for tyk_pump to connect to prometheus and shows on the page http : // localhost:9090/targets that the tyk job is up.
I’m using tyk_pump on docker that’s why the hostname is tyk_pump.

1 Like