Unable to Connect Standalone Redis with Tyk Gateway

Hi Team,
We are setting up data plane in our on-prem instance and are using Gateway 5.0 version and Standalone redis at Backend.

When Tyk Gateway is trying to connect with Standalone Redis its not able to connect to same and healthcheck for the Redis shown Down.

We are using Default Tyk.conf file and same is being overriden using the Envirnment Variables in Tyk G/W Deployment files.
{code}
apiVersion: apps/v1
kind: Deployment
metadata:
name: gateway-tyk-data-plane-tyk-gateway
namespace: ‘20772’
labels:
app: gateway-tyk-data-plane-tyk-gateway
spec:
replicas: 1
minReadySeconds: 5
strategy:
# indicate which strategy we want for rolling update
type: RollingUpdate
rollingUpdate:
maxSurge: 2
maxUnavailable: 1
selector:
matchLabels:
app: gateway-tyk-data-plane-tyk-gateway
template:
metadata:
labels:
app: gateway-tyk-data-plane-tyk-gateway
spec:
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
containers:
- name: gateway-tyk-gateway
image: “tyk/tyk-gateway:001”
imagePullPolicy: IfNotPresent
env:
- name: TYK_GW_DBAPPCONFOPTIONS_NODEISSEGMENTED
value : “false”
- name: TYK_GW_LISTENPORT
value: “8080”

        # Legacy support for Redis Cluster driver. Driver dropped in v3.0.0.
        - name: REDIGOCLUSTER_SHARDCOUNT
          value: "128"
        - name: TYK_GW_STORAGE_TYPE
          value: "redis"
        - name: TYK_GW_STORAGE_HOST
          value: "localhost"
        - name: TYK_GW_STORAGE_ADDRS
          value: "redis-service.local"
        - name: TYK_GW_STORAGE_ENABLECLUSTER
          value: "false"


        - name: TYK_GW_STORAGE_DATABASE
          value: "0"
        - name: TYK_GW_STORAGE_PASSWORD
          valueFrom:
            secretKeyRef:
              name:  secrets-tyk-data-plane-tyk-gateway
              key: redisPass
        - name: TYK_GW_STORAGE_USESSL
          value: "false"
        - name: TYK_GW_SECRET
          valueFrom:
            secretKeyRef:
              name:  secrets-tyk-data-plane-tyk-gateway
              key: APISecret
        - name: TYK_GW_NODESECRET
          valueFrom:
            secretKeyRef:
              name:  secrets-tyk-data-plane-tyk-gateway
              key: APISecret

        - name: TYK_GW_POLICIES_ALLOWEXPLICITPOLICYID
          value: "true"
        - name: TYK_GW_HTTPSERVEROPTIONS_USESSL
          value: "false"
        - name: TYK_GW_TEMPLATEPATH
          value: "/opt/tyk-gateway/templates"
        - name: TYK_GW_TYKJSPATH
          value: "/opt/tyk-gateway/js/tyk.js"
        - name: TYK_GW_MIDDLEWAREPATH
          value: "/mnt/tyk-gateway/middleware"
        - name: TYK_GW_APPPATH
          value: "/mnt/tyk-gateway/apps"
        - name: TYK_GW_POLICIES_POLICYPATH
          value: "/mnt/tyk-gateway/policies"
        - name: TYK_GW_STORAGE_MAXIDLE
          value: "1000"
        - name: TYK_GW_OPTIMISATIONSUSEASYNCSESSIONWRITE
          value: "true"
        - name: TYK_GW_ENABLENONTRANSACTIONALRATELIMITER
          value: "true"
        - name: TYK_GW_POLICIES_POLICYSOURCE
          value: "rpc"
        - name: TYK_GW_ANALYTICSCONFIG_TYPE
          value: ""
        - name: TYK_GW_SLAVEOPTIONS_USERPC
          value: "true"
        - name: TYK_GW_AUTHOVERRIDE_FORCEAUTHPROVIDER
          value: "true"
        - name: TYK_GW_AUTHOVERRIDE_AUTHPROVIDER_STORAGEENGINE
          value: "rpc"
        - name: TYK_GW_POLICIES_POLICYRECORDNAME
          value: "tyk_policies"
        - name: TYK_GW_HASHKEYS
          value: "true"
        - name: TYK_GW_HASHKEYFUNCTION
          value: "murmur128"
        - name: TYK_GW_HTTPSERVEROPTIONS_ENABLEWEBSOCKETS
          value: "true"
        - name: TYK_GW_HTTPSERVEROPTIONS_MINVERSION
          value: "771"
        - name: TYK_GW_HTTPSERVEROPTIONS_CERTIFICATES
          value: '[{"domain_name": "*", "cert_file": "/etc/certs/tls.crt", "key_file": "/etc/certs/tls.key"}]'
        - name: TYK_GW_ALLOWINSECURECONFIGS
          value: "true"
        - name: TYK_GW_COPROCESSOPTIONS_ENABLECOPROCESS
          value: "true"
        - name: TYK_GW_GLOBALSESSIONLIFETIME
          value: "100"
        - name: TYK_GW_MAXIDLECONNSPERHOST
          value: "500"
        - name: TYK_GW_ENABLECUSTOMDOMAINS
          value: "true"
        - name: TYK_GW_PIDFILELOCATION
          value: "/mnt/tyk-gateway/tyk.pid"
        - name: TYK_GW_ENABLEANALYTICS
          value: "true"
        - name: TYK_GW_PROXYSSLINSECURESKIPVERIFY
          value: "true"
        - name: TYK_LOGLEVEL
          value: "debug"
        - name: TYK_GW_SLAVEOPTIONS_RPCKEY
          valueFrom:
            secretKeyRef:
              key: orgId
              name: secrets-tyk-data-plane
        - name: TYK_GW_SLAVEOPTIONS_APIKEY
          valueFrom:
            secretKeyRef:
              key: userApiKey
              name: secrets-tyk-data-plane
        - name: TYK_GW_SECRET
          valueFrom:
            secretKeyRef:
              key: APISecret
              name: secrets-tyk-data-plane
        - name: TYK_GW_SLAVEOPTIONS_GROUPID
          valueFrom:
            secretKeyRef:
              key: groupID
              name: secrets-tyk-data-plane
        - name: TYK_GW_SLAVEOPTIONS_CONNECTIONSTRING
          valueFrom:
            secretKeyRef:
              key: rpcConnString
              name: secrets-tyk-data-plane
        - name: TYK_GW_SLAVEOPTIONS_USESSL
          valueFrom:
            configMapKeyRef:
              key: rpcUseSSL
              name: config-tyk-data-plane
        - name: TYK_GW_SLAVEOPTIONS_SSLINSECURESKIPVERIFY
          valueFrom:
            configMapKeyRef:
              key: rpcSslInsecureSkipVerify
              name: config-tyk-data-plane
      command: ["/opt/tyk-gateway/tyk","--conf=/opt/tyk-gateway/tyk.conf"]
      workingDir: /opt/tyk-gateway
      ports:
        - containerPort: 8080
      resources:
        {}
      volumeMounts:
        - name: tyk-dp-gateway-conf
          mountPath: /mnt/tyk-gateway
        - name: tyk-scratch
          mountPath: /etc/tyk-gateway
      livenessProbe:
        httpGet:
          scheme: "HTTP"
          path: /hello
          port: 8080
        initialDelaySeconds: 5
        periodSeconds: 2
        timeoutSeconds: 3
        failureThreshold: 2
      readinessProbe:
        httpGet:
          scheme: "HTTP"
          path: /hello
          port: 8080
        initialDelaySeconds: 1
        periodSeconds: 10
        timeoutSeconds: 3
        failureThreshold: 3
  volumes:
    - configMap:
        defaultMode: 420
        items:
          - key: tyk_dp.conf
            path: tyk.conf
        name: config-tyk-data-plane
      name: tyk-scratch
    - name: tyk-dp-gateway-conf
      emptyDir: {}

{code}

Logs:
{code}
time=“Sep 11 18:21:21” level=debug msg=“Using /opt/tyk-gateway/tyk.conf for configuration” prefix=main
time=“Sep 11 18:21:21” level=info msg=“Tyk API Gateway 5.0.0” prefix=main
time=“Sep 11 18:21:21” level=warning msg=“Insecure configuration allowed” config.allow_insecure_configs=true prefix=checkup
AnalyticsConfig.PoolSize unset. Defaulting to number of available CPUs" prefix=checkup runtime.NumCPU=8
time=“Sep 11 18:21:21” level=warning msg=“AnalyticsConfig.RecordsBufferSize < minimum - Overriding” minRecordsBufferSize=1000 prefix=checkup
time=“Sep 11 18:21:21” level=warning msg=“AnalyticsConfig.StorageExpirationTime is 0, defaulting to 60s” prefix=checkup storageExpirationTime=0
time=“Sep 11 18:21:21” level=error msg=“Could not set version in versionStore” error=“storage: Redis is either down or was not configured” prefix=main
time=“Sep 11 18:21:21” level=debug msg=“Setting up analytics DB connection” prefix=main
time=“Sep 11 18:21:21” level=debug msg=“Analytics pool worker buffer size” workerBufferSize=125
time=“Sep 11 18:21:21” level=debug msg=“Using serializer msgpack for analytics \n”
time=“Sep 11 18:21:21” level=debug msg=“No Primary instance found, assuming control” prefix=host-check-mgr
time=“Sep 11 18:21:21” level=error msg=“cannot set key in pollerCacheKey” error=“storage: Redis is either down or was not configured”
ti

{code}

It appears some Redis configs are not set appropriately.
The address field takes a host and port separated by colon e.g. redis-master.tyk.svc.cluster.local:6379. If you’ve set this, then you don’t need to set the port and host environment variables.

You can find more information about gateway configs and environment variables in our docs

Feel free to let us know how it goes afterwards

Hi
In last couple of days I was able to fix that but one thing which I am currently TYK_GW_Storage_Address env variable was updated with port number as well nd I was able to connect to redis for few seconds and same connection started breaking again.

On doing curl in Tyk gateway for curl localhost:8080/hello was able to see response that redis container was up .l, but after certain amount of time when I was running the same command to check the status for redis container was marked as down.

Any suggestions what is might be causing unusual behaviour.

That’s weird behavior. The gateway sends periodic signals to Redis to verify the connection is intact.

I see you have some RPC configs there. Are you trying to set up a hybrid cloud gateway?. Scratch that, I see you mention you setting up a data plane.

What does the output field of the Redis object in the hello endpoint show?

The symptom suggests Redis may be crashing and restarting. If not, then you might have to troubleshoot your network connectivity or debug things from the side of Redis.

Do let us know of any updates.