@Gabriel_Pu Looking at your redis manifests, the standout difference with what am running - using bitnami redis - is that you have only the one headless service. The bitnami setup has 2, a headless service and a ‘normal’ ClusterIP service.
Not really clear on the distinction, my understanding is the headless service is used by the redis statefulset for service discovery while the ClusterIP service provides direct access to each pod. The gateway connects to the cluster through the latter
Hi again,
I have not forgotten about you, but I have been a little bit busy this days.
Some update here, I have uninstalled my previous redis-cluster installation, and deployed a new one based con bitnami helm chart redis-cluster 9.1.3 · bitnami/bitnami . For some weird reason, I couldn’t install an old version of redis so I am in v7.0.5.
With this installation, as @NgariNdungu addressed, there is a pair of services so I am pointing to the ClusterIP one from tyk, directly using its IP.
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redisbit-redis-cluster ClusterIP 10.110.65.176 <none> 6379/TCP 7d1h
redisbit-redis-cluster-headless ClusterIP None <none> 6379/TCP,16379/TCP 7d1h
tyk-gateway NodePort 10.96.96.48 <none> 80:32500/TCP 7d5h
Both applications, tyk and redis cluster, runs on same namespace. I also have found that I have connection between tyk and redis service IP using telnet (ping does fail).
b9c-8qntv:/opt/tyk-gateway# ping 10.110.65.176
PING 10.110.65.176 (10.110.65.176) 56(84) bytes of data.
^C
--- 10.110.65.176 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4000ms
root@tyk-gateway-65df8fdb9c-8qntv:/opt/tyk-gateway# telnet 10.110.65.176 6379
Trying 10.110.65.176...
Connected to 10.110.65.176.
Escape character is '^]'.
quit
+OK
Connection closed by foreign host.
After all, Tyk keeps giving me the same error…
I have made a lot of tests, and I think I have tryed that before. In my recent installation, its not easy to downgrade, so I will keep that as a last resort when out of ideas.
Tyk logs. Why is “Creating new redis connection” twice? I think its there since using gateway 4.2.1 version
time="Oct 07 11:45:30" level=debug msg="Using /opt/tyk-gateway/tyk.conf for configuration" prefix=main
time="Oct 07 11:45:30" level=info msg="Tyk API Gateway 4.2.1" prefix=main
time="Oct 07 11:45:30" level=warning msg="Insecure configuration allowed" config.allow_insecure_configs=true prefix=checkup
time="Oct 07 11:45:30" level=warning msg="Default secret should be changed for production." config.secret=352d20ee67be67f6340b4c0605b044b7 prefix=checkup
time="Oct 07 11:45:30" level=error msg="Could not set version in versionStore" error="storage: Redis is either down or was not configured" prefix=main
time="Oct 07 11:45:30" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Oct 07 11:45:30" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Oct 07 11:45:30" level=info msg="Starting Poller" prefix=host-check-mgr
time="Oct 07 11:45:30" level=debug msg="---> Initialising checker" prefix=host-check-mgr
time="Oct 07 11:45:30" level=debug msg="[HOST CHECKER] Config:TriggerLimit: 3"
time="Oct 07 11:45:30" level=debug msg="[HOST CHECKER] Config:Timeout: ~10"
time="Oct 07 11:45:30" level=debug msg="[HOST CHECKER] Config:WorkerPool: 2"
time="Oct 07 11:45:30" level=debug msg="[HOST CHECKER] Init complete"
time="Oct 07 11:45:30" level=debug msg="---> Starting checker" prefix=host-check-mgr
time="Oct 07 11:45:30" level=debug msg="[HOST CHECKER] Starting..."
time="Oct 07 11:45:30" level=debug msg="[HOST CHECKER] Check loop started..."
time="Oct 07 11:45:30" level=debug msg="[HOST CHECKER] Host reporter started..."
time="Oct 07 11:45:30" level=debug msg="---> Checker started." prefix=host-check-mgr
time="Oct 07 11:45:30" level=debug msg="Notifier will not work in hybrid mode" prefix=main
time="Oct 07 11:45:30" level=info msg="PIDFile location set to: /var/run/tyk/tyk-gateway.pid" prefix=main
time="Oct 07 11:45:30" level=warning msg="The control_api_port should be changed for production" prefix=main
time="Oct 07 11:45:30" level=debug msg="Initialising default org store" prefix=main
time="Oct 07 11:45:30" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
time="Oct 07 11:45:30" level=debug msg="Creating new Redis connection pool"
time="Oct 07 11:45:30" level=info msg="--> [REDIS] Creating cluster client"
time="Oct 07 11:45:30" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
time="Oct 07 11:45:30" level=debug msg="Loaded API Endpoints" prefix=main
time="Oct 07 11:45:30" level=info msg="--> Standard listener (http)" port=":8081" prefix=main
time="Oct 07 11:45:30" level=warning msg="Starting HTTP server on:[::]:8081" prefix=main
time="Oct 07 11:45:30" level=info msg="Initialising distributed rate limiter" prefix=main
time="Oct 07 11:45:30" level=debug msg="DRL: Setting node ID: solo-53893398-9204-4ec2-a3cc-667f44ac4590|tyk-gateway-65df8fdb9c-8qntv"
time="Oct 07 11:45:30" level=info msg="Tyk Gateway started (4.2.1)" prefix=main
time="Oct 07 11:45:30" level=info msg="--> Listening on address: (open interface)" prefix=main
time="Oct 07 11:45:30" level=info msg="--> Listening on port: 8081" prefix=main
time="Oct 07 11:45:30" level=info msg="--> PID: 1" prefix=main
time="Oct 07 11:45:30" level=info msg="Loading policies" prefix=main
time="Oct 07 11:45:30" level=info msg="Starting gateway rate limiter notifications..."
time="Oct 07 11:45:30" level=info msg="Policies found (1 total):" prefix=main
time="Oct 07 11:45:30" level=debug msg=" - default" prefix=main
time="Oct 07 11:45:30" level=info msg="Loading API Specification from /opt/tyk/apps/app_sample.json"
time="Oct 07 11:45:30" level=debug msg="Checking for transform paths..."
time="Oct 07 11:45:30" level=debug msg="Checking for transform paths..."
time="Oct 07 11:45:30" level=info msg="Detected 1 APIs" prefix=main
time="Oct 07 11:45:30" level=info msg="Loading API configurations." prefix=main
time="Oct 07 11:45:30" level=info msg="Tracking hostname" api_name="Tyk Test API" domain="(no host)" prefix=main
time="Oct 07 11:45:30" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
time="Oct 07 11:45:30" level=debug msg="Loaded API Endpoints" prefix=main
time="Oct 07 11:45:30" level=info msg="API bind on custom port:0" prefix=main
time="Oct 07 11:45:30" level=debug msg="Initializing API" api_id=1 api_name="Tyk Test API" org_id=default
time="Oct 07 11:45:30" level=debug msg="Batch requests enabled for API" prefix=main
time="Oct 07 11:45:30" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=VersionCheck org_id=default
time="Oct 07 11:45:30" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=RateCheckMW org_id=default
time="Oct 07 11:45:30" level=info msg="Checking security policy: Token" api_id=1 api_name="Tyk Test API" org_id=default
time="Oct 07 11:45:30" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=AuthKey org_id=default
time="Oct 07 11:45:30" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=KeyExpired org_id=default
time="Oct 07 11:45:30" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=AccessRightsCheck org_id=default
time="Oct 07 11:45:30" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=GranularAccessMiddleware org_id=default
time="Oct 07 11:45:30" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=RateLimitAndQuotaCheck org_id=default
time="Oct 07 11:45:30" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=VersionCheck org_id=default
time="Oct 07 11:45:30" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=KeyExpired org_id=default
time="Oct 07 11:45:30" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=AccessRightsCheck org_id=default
time="Oct 07 11:45:30" level=debug msg="Rate limit endpoint is: /tyk-api-test/tyk/rate-limits" api_id=1 api_name="Tyk Test API" org_id=default
time="Oct 07 11:45:30" level=debug msg="Setting Listen Path: /tyk-api-test/" api_id=1 api_name="Tyk Test API" org_id=default
time="Oct 07 11:45:30" level=info msg="API Loaded" api_id=1 api_name="Tyk Test API" org_id=default prefix=gateway server_name=-- user_id=-- user_ip=--
time="Oct 07 11:45:30" level=debug msg="Checker host list" prefix=main
time="Oct 07 11:45:30" level=info msg="Loading uptime tests..." prefix=host-check-mgr
time="Oct 07 11:45:30" level=debug msg="--- Setting tracking list up" prefix=host-check-mgr
time="Oct 07 11:45:30" level=debug msg="Reset initiated" prefix=host-check-mgr
time="Oct 07 11:45:30" level=debug msg="[HOST CHECKER] Checker reset queued!"
time="Oct 07 11:45:30" level=debug msg="Checker host Done" prefix=main
time="Oct 07 11:45:30" level=info msg="Initialised API Definitions" prefix=main
time="Oct 07 11:45:30" level=info msg="API reload complete" prefix=main
time="Oct 07 11:45:31" level=debug msg="Creating new Redis connection pool"
time="Oct 07 11:45:31" level=info msg="--> [REDIS] Creating cluster client"
time="Oct 07 11:45:31" level=debug msg="Creating new Redis connection pool"
time="Oct 07 11:45:31" level=info msg="--> [REDIS] Creating cluster client"
time="Oct 07 11:45:40" level=error msg="Redis health check failed" error="storage: Redis is either down or was not configured" liveness-check=true prefix=main
time="Oct 07 11:45:40" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Oct 07 11:45:40" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Oct 07 11:45:40" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
time="Oct 07 11:45:41" level=debug msg="[HOST CHECKER] Host list reset"
time="Oct 07 11:45:50" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Oct 07 11:45:50" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Oct 07 11:45:50" level=error msg="Redis health check failed" error="storage: Redis is either down or was not configured" liveness-check=true prefix=main
time="Oct 07 11:45:50" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
time="Oct 07 11:46:00" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Oct 07 11:46:00" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Oct 07 11:46:00" level=error msg="Redis health check failed" error="storage: Redis is either down or was not configured" liveness-check=true prefix=main
time="Oct 07 11:46:00" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
time="Oct 07 11:46:10" level=error msg="Redis health check failed" error="storage: Redis is either down or was not configured" liveness-check=true prefix=main
time="Oct 07 11:46:10" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Oct 07 11:46:10" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Oct 07 11:46:10" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
time="Oct 07 11:46:20" level=error msg="Redis health check failed" error="storage: Redis is either down or was not configured" liveness-check=true prefix=main
time="Oct 07 11:46:20" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Oct 07 11:46:20" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Oct 07 11:46:20" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
time="Oct 07 11:46:30" level=error msg="Redis health check failed" error="storage: Redis is either down or was not configured" liveness-check=true prefix=main
time="Oct 07 11:46:30" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Oct 07 11:46:30" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Oct 07 11:46:30" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
time="Oct 07 11:46:40" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Oct 07 11:46:40" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Oct 07 11:46:40" level=error msg="Redis health check failed" error="storage: Redis is either down or was not configured" liveness-check=true prefix=main
time="Oct 07 11:46:40" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
I cleaned up my config file a little bit, it should be the same as this, except port and storage.
apiVersion: v1
data:
tyk.conf: |
{
"listen_port": 8081,
"secret": "352d20ee67be67f6340b4c0605b044b7",
"template_path": "/opt/tyk/templates",
"tyk_js_path": "/opt/tyk/js/tyk.js",
"middleware_path": "/opt/tyk/middleware",
"use_db_app_configs": false,
"app_path": "/opt/tyk/apps/",
"storage": {
"type": "redis",
"enable_cluster": true,
"addrs": [
"10.110.65.176:6379"
]
,
"database": 0,
"optimisation_max_idle": 2000,
"optimisation_max_active": 4000,
"username": "",
"password": "",
"use_ssl": false
},
"enable_analytics": false,
"optimisations_use_async_session_write": true,
"enable_non_transactional_rate_limiter": true,
"enable_sentinel_rate_limiter": false,
"enable_redis_rolling_limiter": false,
"allow_master_keys": false,
"policies": {
"policy_source": "file",
"policy_record_name": "/opt/tyk/policies/policies.json"
},
"hash_keys": true,
"close_connections": false,
"http_server_options": {
"enable_websockets": true
},
"allow_insecure_configs": true,
"coprocess_options": {
"enable_coprocess": true,
"coprocess_grpc_server": ""
},
"enable_bundle_downloader": true,
"bundle_base_url": "",
"global_session_lifetime": 100,
"force_global_session_lifetime": false,
"max_idle_connections_per_host": 500
}
kind: ConfigMap
metadata:
name: tyk-gateway-conf
Thanks for your efforts
Hello @Gabriel_Pu I am working on something that could potentially help us here. I will reach back once its done.
1 Like
@Gabriel_Pu I think I found the issue. The helm chart version your are running is 8.2.4 which is Redis 6. If you downgrade to 7.6.4 it will work ootb.
Also your Redis cluster URLs are incorrect.
redis-cluster-0.redis-cluster.redis.svc.cluster.local:6379 wont resolve since there is no service with that name. You just need a single entry which in your case would be redisbit-redis-cluster.redis.svc.cluster.local:6379
One last thing, what I was working on is here. Its a repo of shell commands that can help you get started with Tyk.
You can install Tyk gateway with a redis cluster with this simple command ./poc.sh -r redis-cluster tyk-gateway
Thanks @zaid !
I finally managed to install the bitnami/redis-cluster version you suggested. I was previously using values.yml option image.tag as suggested in the documentation redis-cluster 8.4.3 · bitnami/bitnami
However, cleaning previous installation and this command did the trick
$ helm install redisbit -f values.yml bitnami/redis-cluster --version v7.6.4 --namespace tyk
and installed Redis 6.2.7
On the other hand, I modified tyk configmap to point to redis-cluster service as this (I had same results configuring the six instances separately):
...
"storage": {
"type": "redis",
"enable_cluster": true,
"addrs": [
"redisbit-redis-cluster.tyk.svc.cluster.local:6379"
]
,
"database": 0,
"optimisation_max_idle": 2000,
"optimisation_max_active": 4000,
"username": "",
"password": "",
"use_ssl": false
}
...
After deploying tyk-gateway, the error has changed!:
time="Oct 10 12:14:41" level=debug msg="Using /opt/tyk-gateway/tyk.conf for configuration" prefix=main
time="Oct 10 12:14:41" level=info msg="Tyk API Gateway 4.2.1" prefix=main
time="Oct 10 12:14:41" level=warning msg="Insecure configuration allowed" config.allow_insecure_configs=true prefix=checkup
time="Oct 10 12:14:41" level=warning msg="Default secret should be changed for production." config.secret=352d20ee67be67f6340b4c0605b044b7 prefix=checkup
time="Oct 10 12:14:41" level=error msg="Could not set version in versionStore" error="storage: Redis is either down or was not configured" prefix=main
time="Oct 10 12:14:41" level=debug msg="Notifier will not work in hybrid mode" prefix=main
time="Oct 10 12:14:41" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Oct 10 12:14:41" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Oct 10 12:14:41" level=info msg="Starting Poller" prefix=host-check-mgr
time="Oct 10 12:14:41" level=debug msg="---> Initialising checker" prefix=host-check-mgr
time="Oct 10 12:14:41" level=debug msg="[HOST CHECKER] Config:TriggerLimit: 3"
time="Oct 10 12:14:41" level=debug msg="[HOST CHECKER] Config:Timeout: ~10"
time="Oct 10 12:14:41" level=debug msg="[HOST CHECKER] Config:WorkerPool: 2"
time="Oct 10 12:14:41" level=debug msg="[HOST CHECKER] Init complete"
time="Oct 10 12:14:41" level=debug msg="---> Starting checker" prefix=host-check-mgr
time="Oct 10 12:14:41" level=debug msg="[HOST CHECKER] Starting..."
time="Oct 10 12:14:41" level=debug msg="[HOST CHECKER] Check loop started..."
time="Oct 10 12:14:41" level=debug msg="[HOST CHECKER] Host reporter started..."
time="Oct 10 12:14:41" level=debug msg="---> Checker started." prefix=host-check-mgr
time="Oct 10 12:14:41" level=info msg="PIDFile location set to: /var/run/tyk/tyk-gateway.pid" prefix=main
time="Oct 10 12:14:41" level=warning msg="The control_api_port should be changed for production" prefix=main
time="Oct 10 12:14:41" level=debug msg="Initialising default org store" prefix=main
time="Oct 10 12:14:41" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
time="Oct 10 12:14:41" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
time="Oct 10 12:14:41" level=debug msg="Creating new Redis connection pool"
time="Oct 10 12:14:41" level=info msg="--> [REDIS] Creating cluster client"
time="Oct 10 12:14:41" level=debug msg="Loaded API Endpoints" prefix=main
time="Oct 10 12:14:41" level=info msg="--> Standard listener (http)" port=":8080" prefix=main
time="Oct 10 12:14:41" level=warning msg="Starting HTTP server on:[::]:8080" prefix=main
time="Oct 10 12:14:41" level=info msg="Initialising distributed rate limiter" prefix=main
time="Oct 10 12:14:41" level=debug msg="DRL: Setting node ID: solo-2519ac5b-06eb-4b27-8da1-788d4fefd872|tyk-gateway-76fc9686bb-297wb"
time="Oct 10 12:14:41" level=info msg="Tyk Gateway started (4.2.1)" prefix=main
time="Oct 10 12:14:41" level=info msg="--> Listening on address: (open interface)" prefix=main
time="Oct 10 12:14:41" level=info msg="--> Listening on port: 8080" prefix=main
time="Oct 10 12:14:41" level=info msg="--> PID: 1" prefix=main
time="Oct 10 12:14:41" level=info msg="Loading policies" prefix=main
time="Oct 10 12:14:41" level=info msg="Policies found (1 total):" prefix=main
time="Oct 10 12:14:41" level=debug msg=" - default" prefix=main
time="Oct 10 12:14:41" level=info msg="Loading API Specification from /opt/tyk/apps/app_sample.json"
time="Oct 10 12:14:41" level=debug msg="Checking for transform paths..."
time="Oct 10 12:14:41" level=debug msg="Checking for transform paths..."
time="Oct 10 12:14:41" level=info msg="Detected 1 APIs" prefix=main
time="Oct 10 12:14:41" level=info msg="Loading API configurations." prefix=main
time="Oct 10 12:14:41" level=info msg="Tracking hostname" api_name="Tyk Test API" domain="(no host)" prefix=main
time="Oct 10 12:14:41" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
time="Oct 10 12:14:41" level=debug msg="Loaded API Endpoints" prefix=main
time="Oct 10 12:14:41" level=info msg="API bind on custom port:0" prefix=main
time="Oct 10 12:14:41" level=debug msg="Initializing API" api_id=1 api_name="Tyk Test API" org_id=default
time="Oct 10 12:14:41" level=debug msg="Batch requests enabled for API" prefix=main
time="Oct 10 12:14:41" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=VersionCheck org_id=default
time="Oct 10 12:14:41" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=RateCheckMW org_id=default
time="Oct 10 12:14:41" level=info msg="Checking security policy: Token" api_id=1 api_name="Tyk Test API" org_id=default
time="Oct 10 12:14:41" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=AuthKey org_id=default
time="Oct 10 12:14:41" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=KeyExpired org_id=default
time="Oct 10 12:14:41" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=AccessRightsCheck org_id=default
time="Oct 10 12:14:41" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=GranularAccessMiddleware org_id=default
time="Oct 10 12:14:41" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=RateLimitAndQuotaCheck org_id=default
time="Oct 10 12:14:41" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=VersionCheck org_id=default
time="Oct 10 12:14:41" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=KeyExpired org_id=default
time="Oct 10 12:14:41" level=debug msg=Init api_id=1 api_name="Tyk Test API" mw=AccessRightsCheck org_id=default
time="Oct 10 12:14:41" level=debug msg="Rate limit endpoint is: /tyk-api-test/tyk/rate-limits" api_id=1 api_name="Tyk Test API" org_id=default
time="Oct 10 12:14:41" level=debug msg="Setting Listen Path: /tyk-api-test/" api_id=1 api_name="Tyk Test API" org_id=default
time="Oct 10 12:14:41" level=info msg="API Loaded" api_id=1 api_name="Tyk Test API" org_id=default prefix=gateway server_name=-- user_id=-- user_ip=--
time="Oct 10 12:14:41" level=debug msg="Checker host list" prefix=main
time="Oct 10 12:14:41" level=info msg="Loading uptime tests..." prefix=host-check-mgr
time="Oct 10 12:14:41" level=debug msg="--- Setting tracking list up" prefix=host-check-mgr
time="Oct 10 12:14:41" level=debug msg="Reset initiated" prefix=host-check-mgr
time="Oct 10 12:14:41" level=debug msg="[HOST CHECKER] Checker reset queued!"
time="Oct 10 12:14:41" level=debug msg="Checker host Done" prefix=main
time="Oct 10 12:14:41" level=info msg="Initialised API Definitions" prefix=main
time="Oct 10 12:14:41" level=info msg="API reload complete" prefix=main
time="Oct 10 12:14:41" level=info msg="Starting gateway rate limiter notifications..."
time="Oct 10 12:14:41" level=debug msg="Creating new Redis connection pool"
time="Oct 10 12:14:41" level=info msg="--> [REDIS] Creating cluster client"
time="Oct 10 12:14:41" level=debug msg="Creating new Redis connection pool"
time="Oct 10 12:14:41" level=info msg="--> [REDIS] Creating cluster client"
time="Oct 10 12:14:50" level=debug msg="[HOST CHECKER] Host list reset"
time="Oct 10 12:14:51" level=debug msg="Error trying to get value:redis: nil"
time="Oct 10 12:14:51" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Oct 10 12:15:01" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:15:11" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:15:21" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:15:31" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:15:41" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:15:51" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:16:01" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:16:11" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:16:21" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:16:31" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:16:41" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Oct 10 12:16:51" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
(and againg, and again…)
Well, I am not really sure if this is an error, as I am on debug level. However I tryed to create a custom key following this API Key Management just to check whats going on:
root@dnsutils:/# curl 10.244.2.57:8080/hello
{"status":"pass","version":"4.2.1","description":"Tyk GW","details":{"redis":{"status":"pass","componentType":"datastore","time":"2022-10-10T12:36:41Z"}}}
root@dnsutils:/# curl 10.244.2.57:8080/api/keys/mycustomkey -H 'x-tyk-authorization: 352d20ee67be67f6340b4c0605b044b7' -H 'Content-Type: application/json' -d @token.json
Not Foundroot
I will try to dig into this…
Thank you very much
Hello @Gabriel_Pu, glad this worked. I did not see the “I am master” logs when setting things up using the my script Maybe you are doing something different? Maybe you can use the script to create the environment for you then you can export that environment in yaml files?
@Gabriel_Pu disregard what I said above. Those logs are absolutely normally. In fact, we have a ticket to silence them as they are confusing and distracting 
1 Like
Hello, I’m having persistence issues with Tyk Headless Gateway v4.3.0 and v5.0.0
Tyk Headless using tyk-headless 0.14.0 · tyk/tyk-helm
Redis Cluster redis-cluster 8.6.2 · bitnami/bitnami
On startup I get the error level=error msg="Could not set version in versionStore" error="storage: Redis is either down or was not configured" prefix=main
Previously I was seeing the error Connection to Redis failed, reconnect in 10s that about continuously but after switching from bitnami/redis to `bitnami/redis-cluster, I only see the error above on startup. Unfortunately, persistence doesn’t seem to be working. On restart of the headless gateway deployment, all configurations are lost.
Below are the logs from the gateway on from startup. I have confirmed and tested all the redis cluster endpoints.
time="May 30 14:58:07" level=info msg="Tyk API Gateway 5.0.0" prefix=main
time="May 30 14:58:07" level=warning msg="Insecure configuration allowed" config.allow_insecure_configs=true prefix=checkup
time="May 30 14:58:07" level=error msg="Could not set version in versionStore" error="storage: Redis is either down or was not configured" prefix=main
time="May 30 14:58:07" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="May 30 14:58:07" level=info msg="Starting Poller" prefix=host-check-mgr
time="May 30 14:58:07" level=info msg="PIDFile location set to: /mnt/tyk-gateway/tyk.pid" prefix=main
time="May 30 14:58:07" level=warning msg="The control_api_port should be changed for production" prefix=main
time="May 30 14:58:07" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
time="May 30 14:58:07" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
time="May 30 14:58:07" level=info msg="--> [REDIS] Creating cluster client"
time="May 30 14:58:07" level=info msg="--> Standard listener (http)" port=":8080" prefix=main
time="May 30 14:58:07" level=warning msg="Starting HTTP server on:[::]:8080" prefix=main
time="May 30 14:58:07" level=info msg="Initialising distributed rate limiter" prefix=main
time="May 30 14:58:07" level=info msg="Tyk Gateway started (5.0.0)" prefix=main
time="May 30 14:58:07" level=info msg="--> Listening on address: (open interface)" prefix=main
time="May 30 14:58:07" level=info msg="--> Listening on port: 8080" prefix=main
time="May 30 14:58:07" level=info msg="--> PID: 1" prefix=main
time="May 30 14:58:07" level=info msg="Loading policies" prefix=main
time="May 30 14:58:07" level=error msg="Couldn't unmarshal policies: EOF" prefix=policy
time="May 30 14:58:07" level=info msg="Policies found (0 total):" prefix=main
time="May 30 14:58:07" level=info msg="Detected 0 APIs" prefix=main
time="May 30 14:58:07" level=warning msg="No API Definitions found, not reloading" prefix=main
time="May 30 14:58:07" level=info msg="Starting gateway rate limiter notifications..."
time="May 30 14:58:07" level=info msg="--> [REDIS] Creating cluster client"
time="May 30 14:58:07" level=info msg="--> [REDIS] Creating cluster client"