Tyk CE Pump with Prometheus in Kubernetes

Support Installation and Deployment
, , ,
1

Hello,

I am deploying Tyk CE using Headless helm chart but it seems it is looking all the time by default onto MongoDB for Pump
the below is my Pump configMap and also the error that i have :

Also as described here in this issue the pump doesn't pump to Prometheus · Issue #164 · TykTechnologies/tyk-helm-chart · GitHub i have added all related environment variables under “Pump” section but same issue

{
      "log_level": "debug",
      "analytics_storage_type": "redis",
      "analytics_storage_config": {
          "type": "redis",
          "hosts": null,
          "username": "",
          "optimisation_max_idle": 2000,
          "optimisation_max_active": 4000
      },
      "pumps": {
          "prometheus": {
            "type": "prometheus",
            "meta": {
                "listen_address": "prometheus-server.default.svc.cluster.local:80",
                "path": "/metrics",
                "custom_metrics": [
                  {
                   "name": "tyk_http_requests_total",
                   "description": "Total of API requests",
                   "metric_type": "counter",
                   "labels": ["response_code","api_name","method","api_key","alias","path"]
                  },
                  {
                   "name": "tyk_http_latency",
                   "description": "Latency of API requests",
                   "metric_type": "histogram",
                   "labels": ["type","response_code","api_name","method","api_key","alias","path"]
                  }
                ]
            }
          }
      },
      "dont_purge_uptime_data": false,
      "purge_delay": 2,
      "uptime_pump_config": {
          "collection_name": "tyk_uptime_analytics_headless"
      }
    }

and i am getting this log:

time="Mar  9 18:23:33" level=info msg="## Tyk Analytics Pump, 1.5.1 ##"
time="Mar  9 18:23:33" level=debug msg="Connecting to redis cluster"
time="Mar  9 18:23:33" level=debug msg="Creating new Redis connection pool"
time="Mar  9 18:23:33" level=info msg="--> [REDIS] Creating single-node client"
time="Mar  9 18:23:33" level=debug msg="[STORE] SET Raw key is: pump"
time="Mar  9 18:23:33" level=debug msg="Input key was: version-check-pump"
time="Mar  9 18:23:33" level=debug msg="[STORE] Setting key: version-check-pump"
time="Mar  9 18:23:33" level=debug msg="Input key was: version-check-pump"
time="Mar  9 18:23:33" level=info msg="Serving health check endpoint at http://localhost:8083/health ..."
time="Mar  9 18:23:33" level=debug msg="Checking MongoDB Aggregate Pump env variables with prefix TYK_PMP_PUMPS_MONGOAGGREGATE_META"````
2

Hi @AlyRagab,

Your logs show Pump v1.5.1. It needs to be at least v1.6 to use Prometheus as stated here.
Would recommend v1.7.0, the latest available version.

3

Hi @AlyRagab,

We are currently working on new Helm Charts here that are designed to be much more flexible and user-friendly than before. While this is a significant change, we plan to develop these charts progressively and will only deprecate the original Helm Chart when the new one has feature parity.

We are pleased to inform you that the support for Tyk CE gateway with pump is quite complete, and you can try it out here: tyk-oss. We’ll keep you posted with any updates, and we’ll share the official release of the new Helm Charts in the community forum as soon as they’re ready.

1 Like
4

Thanks @carol i will check it, but the question now is, Does this Helm chart create a Service resource for the Pump deployment in order to update Prometheus regarding it with target section ?

5

Thanks @Ubong , Actually i tried with latest version of Pump and it was not working too and without any error in the logs, I think it is communication issue with Prometheus as Pump deployment does not have a Service resource, is not it ?

6

Hi @AlyRagab,

Can you try setting "dont_purge_uptime_data": true, in pump? By default that tries to connect to Mongo and could be silent

7

Hi @AlyRagab,

No it does not expose Pump as a service.

If you run Prometheus using prometheus operator, you can set tyk-pump.pump.prometheusPump.prometheusOperator.enabled to true. It creates a PodMonitor resource, and prometheus will then scrap from pump.

Otherwise, please configure the prometheus target as :9090 in prometheus config.

8

I got it working by the other helm chart (headless) and i created a service resource for Pump
Now prometheus can get the Pump as a target but i have issue in Grafana Dashboard does not show anything