Pump and Dashboard unable to connect with MongoDB 4.2.23 Enterprise Atlas cluster (TLS issue?)

armujahid · December 5, 2022, 8:56am

Detailed steps to reproduce are mentioned here

Mongo pumps: Atlas connection issue (mongo TLS issue)

opened 09:05AM - 05 Dec 22 UTC

closed 07:34AM - 08 Dec 22 UTC

**Issue:** Unable to connect with MongoDB 4.2.23 Enterprise Atlas cluster. *…*Steps to reproduce:** 1) Create mongo 4.2 cluster on https://cloud.mongodb.com 2) You will get connection string like `mongodb+srv://<username>:<password>@something.xyz.mongodb.net/?retryWrites=true&w=majority` .Remove query params since pump throws error with `?retryWrites=true&w=majority`. Final connection string is `mongodb+srv://<username>:<password>@something.xyz.mongodb.net/tyk_analytics` that will be used in pump config 3) Clone https://github.com/TykTechnologies/tyk-pro-docker-demo 4) Optional: Update pump and dashboard version (Issue is also reproducible in latest versions) 5) Use `pump.mongo.env` to enable TLS (set `_MONGOUSESSL` and `_MONGOSSLINSECURESKIPVERIFY` for each pump. Although 2nd one is not required) ``` # Outdated variables in docs that doesn't work https://tyk.io/docs/tyk-pump/tyk-pump-configuration/tyk-pump-environment-variables/ # Docs should be updated TYK_PMP_PUMPS_MONGO_META_MONGOUSESSL=true TYK_PMP_PUMPS_MONGO_META_MONGOSSLINSECURESKIPVERIFY=true TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOUSESSL=true TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOSSLINSECURESKIPVERIFY=true TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOUSESSL=true TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOSSLINSECURESKIPVERIFY=true TYK_PMP_PUMPS_MAIN_TYPE=mongo TYK_PMP_PUMPS_MAIN_META_COLLECTIONNAME=tyk_analytics TYK_PMP_PUMPS_MAIN_META_MONGOURL=mongodb+srv://<username>:<password>@something.xyz.mongodb.net/tyk_analytics # added by me TYK_PMP_PUMPS_MAIN_META_MONGOUSESSL=true TYK_PMP_PUMPS_MAIN_META_MONGOSSLINSECURESKIPVERIFY=true TYK_PMP_PUMPS_MAIN_META_MAXINSERTBATCHSIZEBYTES=80000 TYK_PMP_PUMPS_MAIN_META_MAXDOCUMENTSIZEBYTES=20112 TYK_PMP_PUMPS_MAINAGG_TYPE=mongo-pump-aggregate TYK_PMP_PUMPS_MAINAGG_META_MONGOURL=mongodb+srv://<username>:<password>@something.xyz.mongodb.net/tyk_analytics # added by me TYK_PMP_PUMPS_MAINAGG_META_MONGOUSESSL=true TYK_PMP_PUMPS_MAINAGG_META_MONGOSSLINSECURESKIPVERIFY=true TYK_PMP_PUMPS_MAINAGG_META_USEMIXEDCOLLECTION=true TYK_PMP_PUMPS_MAINAGG_META_TRACKALLPATHS=true TYK_PMP_UPTIMEPUMPCONFIG_COLLECTIONNAME=tyk_uptime_analytics TYK_PMP_UPTIMEPUMPCONFIG_MONGOURL=mongodb+srv://<username>:<password>@something.xyz.mongodb.net/tyk_analytics # added by me TYK_PMP_UPTIMEPUMPCONFIG_MONGOUSESSL=true TYK_PMP_UPTIMEPUMPCONFIG_MONGOSSLINSECURESKIPVERIFY=true TYK_PMP_UPTIMEPUMPCONFIG_MAXINSERTBATCHSIZEBYTES=500000 TYK_PMP_UPTIMEPUMPCONFIG_MAXDOCUMENTSIZEBYTES=200000 ``` **Pump 1.7.0 output:** ``` time="Dec 06 13:37:12" level=info msg="## Tyk Analytics Pump, 1.7.0 ##" prefix=main time="Dec 06 13:37:12" level=info msg="--> [REDIS] Creating cluster client" time="Dec 06 13:37:12" level=info msg="Serving health check endpoint at http://localhost:8083/health ..." prefix=server ``` Below line is only logged if i use `mongodb://` instead of using `mongodb+srv://` ``` time="Dec 06 13:37:12" level=info msg=Init collection_name= prefix=mongo-pump url= ``` **Pump 1.6.0 output:** ``` time="Dec 5 06:55:18" level=info msg="## Tyk Analytics Pump, 1.6.0 ##" time="Dec 5 06:55:18" level=info msg="--> [REDIS] Creating cluster client" time="Dec 5 06:55:18" level=info msg="Serving health check endpoint at http://localhost:8083/health ..." ``` **Pump 1.5.1 output:** ``` time="Dec 5 07:18:02" level=info msg="## Tyk Analytics Pump, 1.5.1 ##" time="Dec 5 07:18:02" level=info msg="--> [REDIS] Creating cluster client" time="Dec 5 07:18:02" level=info msg="Serving health check endpoint at http://localhost:8083/health ..." time="Dec 5 07:18:02" level=info msg=Init collection_name= url= ``` **Additional Info:** Mongo supported versions are MongoDB 3.x to 4.4.x as per https://tyk.io/docs/planning-for-production/database-settings/mongodb/#supported-versions https://github.com/TykTechnologies/tyk-pump/pull/24 added TLS support for mongo but I think relevant environment variables are not working Also facing similar issue with Tyk dashboard (even after setting `TYK_DB_MONGOUSESSL=true`) ``` time="Dec 05 07:21:47" level=info msg="Tyk Analytics Dashboard 4.2.3" time="Dec 05 07:21:47" level=info msg="Copyright Tyk Technologies Ltd 2019" time="Dec 05 07:21:47" level=info msg="https://www.tyk.io" time="Dec 05 07:21:47" level=info msg="Using /opt/tyk-dashboard/tyk_analytics.conf for configuration" time="Dec 05 07:21:47" level=info msg="Listening on port: 3000" time="Dec 05 07:21:47" level=warning msg="Default admin_secret `12345` should be changed for production use." time="Dec 05 07:21:47" level=info msg="Initializing Main storage with `mongo` database" time="Dec 05 07:21:47" level=info msg="Connecting to MongoDB: [something.xyz.mongodb.net]" time="2022-12-05T07:21:57Z" level=fatal msg="Could not connect to config store" error="no reachable servers" ``` P.S database is reachable since I can use `mongosh "<connection string>"` to connect to that exact DB

I am using GitHub - TykTechnologies/tyk-pro-docker-demo: Tyk Pro demo using docker and docker compose, supercedes tyk_quickstart and locally it works with MongoDB 4.2.23 Community

  tyk-mongo:
    image: mongo:4.2
    container_name: tyk-mongo

    ports:
      - "27017:27017"

    volumes:
      - db-data:/data/db

    networks:
      - tyk

Now I have a MongoDB 4.2.23 Enterprise Atlas cluster created at https://cloud.mongodb.com/
I am unable to make pump and dashboard connect to that mongo cluster (even after setting relevant USESSL and optional SSLINSECURESKIPVERIFY variables as mentioned below.

pump.mongo.env

# Outdated variables in docs that doesn't work https://tyk.io/docs/tyk-pump/tyk-pump-configuration/tyk-pump-environment-variables/
# Docs should be updated
TYK_PMP_PUMPS_MONGO_META_MONGOUSESSL=true
TYK_PMP_PUMPS_MONGO_META_MONGOSSLINSECURESKIPVERIFY=true
TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOUSESSL=true
TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOSSLINSECURESKIPVERIFY=true
TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOUSESSL=true
TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOSSLINSECURESKIPVERIFY=true

TYK_PMP_PUMPS_MAIN_TYPE=mongo
TYK_PMP_PUMPS_MAIN_META_COLLECTIONNAME=tyk_analytics
TYK_PMP_PUMPS_MAIN_META_MONGOURL=mongodb+srv://<username>:<password>@something.xyz.mongodb.net/tyk_analytics
# added by me
TYK_PMP_PUMPS_MAIN_META_MONGOUSESSL=true
TYK_PMP_PUMPS_MAIN_META_MONGOSSLINSECURESKIPVERIFY=true

TYK_PMP_PUMPS_MAIN_META_MAXINSERTBATCHSIZEBYTES=80000
TYK_PMP_PUMPS_MAIN_META_MAXDOCUMENTSIZEBYTES=20112

TYK_PMP_PUMPS_MAINAGG_TYPE=mongo-pump-aggregate
TYK_PMP_PUMPS_MAINAGG_META_MONGOURL=mongodb+srv://<username>:<password>@something.xyz.mongodb.net/tyk_analytics
# added by me
TYK_PMP_PUMPS_MAINAGG_META_MONGOUSESSL=true
TYK_PMP_PUMPS_MAINAGG_META_MONGOSSLINSECURESKIPVERIFY=true

TYK_PMP_PUMPS_MAINAGG_META_USEMIXEDCOLLECTION=true
TYK_PMP_PUMPS_MAINAGG_META_TRACKALLPATHS=true

TYK_PMP_UPTIMEPUMPCONFIG_COLLECTIONNAME=tyk_uptime_analytics
TYK_PMP_UPTIMEPUMPCONFIG_MONGOURL=mongodb+srv://<username>:<password>@something.xyz.mongodb.net/tyk_analytics
# added by me
TYK_PMP_UPTIMEPUMPCONFIG_MONGOUSESSL=true
TYK_PMP_UPTIMEPUMPCONFIG_MONGOSSLINSECURESKIPVERIFY=true

TYK_PMP_UPTIMEPUMPCONFIG_MAXINSERTBATCHSIZEBYTES=500000
TYK_PMP_UPTIMEPUMPCONFIG_MAXDOCUMENTSIZEBYTES=200000

dashboard env:

...
TYK_DB_MONGOUSESSL=true

Olu · December 7, 2022, 11:39pm

Hi @armujahid,

And thanks for sharing this. Can you try the connection using an old connection string from version 3.4 or earlier?

This stack overflow question will give a hint

armujahid · December 8, 2022, 6:39am

@Olu Connection string of this form mongodb://<username>:<password>@some-cluster-01-shard-00-00.abcde.mongodb.net:27017,some-cluster-01-shard-00-01.abcde.mongodb.net:27017,some-cluster-01-shard-00-02.abcde.mongodb.net:27017/tyk_analytics?replicaSet=atlas-fghij-shard-0&authSource=admin is working perfectly fine with pump If I set USESSL as

TYK_PMP_PUMPS_MAIN_META_MONGOUSESSL=true
TYK_PMP_PUMPS_MAINAGG_META_MONGOUSESSL=true
TYK_PMP_UPTIMEPUMPCONFIG_MONGOUSESSL=true

Issue is dashboard is still not working probably because TYK_DB_MONGOUSESSL=true as mentioned in docs is outdated/incorrect (like outdated pump variables mentioned above). Any idea how to make dashboard work with that same connection string with TLS?

Currently I am getting this error

time="2022-12-08T08:12:12Z" level=fatal msg="Could not connect to config store" error="no reachable servers"

Dashboard env

TYK_DB_LICENSEKEY=<license>
TYK_DB_STORAGE_MAIN_TYPE=mongo
TYK_DB_STORAGE_MAIN_CONNECTIONSTRING=<same string that works in pump>
TYK_DB_MONGOUSESSL=true # doesn't seem to be working although it's mentioned in docs
TYK_DB_STORAGE_MAIN_MONGOUSESSL=true # also doesn't work
...

Olu · December 12, 2022, 5:45pm

I think your environment variables are wrong. The right values are

TYK_DB_MONGOURL
TYK_DB_MONGOSSLINSECURESKIPVERIFY
TYK_DB_MONGOUSESSL

You could also try setting it via the config file

armujahid · December 13, 2022, 5:56am

Thanks. TYK_DB_MONGOURL is working perfectly fine with TYK_DB_MONGOUSESSL after removing TYK_DB_STORAGE_MAIN_ variables. Dashboard is starting although I couldn’t verify complete functionality since my license has been expired. I will try to renew that. I was following this demo
tyk-pro-docker-demo/docker-compose.yml at master · TykTechnologies/tyk-pro-docker-demo · GitHub that created this confusion.