How to setup redis-server 3.0.7 for Tyk 1.9.1 on Ubuntu 14.04.3 LTS


#1

I get the following errors when trying to start my redis instance using tyk :

ERRO[0000] Could not EXPIRE key: could not complete command
ERRO[0005] Multi command failed: could not complete command

I did set the following parameters in redis.conf in both the master and the slave :
port 6379 - cluster-enabled yes - cluster-config-file nodes.conf - cluster-node-timeout 5000 - appendonly yes

I’ve also tried to set maxmemory-policy to any parameter but it doesn’t change anything.

Here is the storage section of my tyk.conf :

“storage”: {
“type”: “redis”,
“enable_cluster”: true,
“hosts” : {
“127.0.0.1”: “6379”,
“192.168.50.41”: “6379”
},
“username”: “”,
“password”: “”,
“database”: 0,
“optimisation_max_idle”: 500,
“optimisation_max_active”: 800
},

Here is a part of the content of my tyk_analytics.conf file :

“enable_cluster”: true,
“redis_hosts”: {
“127.0.0.1”: “6379”,
“192.168.50.42”: “6379”,
},
“redis_password”: “”,
“force_api_defaults”: false,
“notify_on_change”: true,
“license_owner”: “Your Name”,
“hash_keys”: true,
“email_backend”: {},
“hide_listen_path”: false,
“use_sentry”: false,
“sentry_code”: “”,
“sentry_js_code”: “”,
“show_org_id”: true,
“enable_duplicate_slugs”: true,
“host_config” : {
“override_hostname”: “tyk2”,
“disable_org_slug_prefix”: true,
“enable_host_names”: true,
“hostname”: “192.168.50.41”,
“portal_domains”: {},
“portal_root_path”: “/portal”
},

The local IP in the “redis_hosts” section of the master is the slave IP and same for the other way around.
Anyone who has already configured redis-server with cluster mode for Tyk?

Running Redis server v=3.0.7 on Ubuntu 14.04.3 LTS and Tyk v1.9


#2

Have you set up a Redis cluster or just a redis master/slave setup with some kind of failover? If you have master/slave with failover, then you need to handle failover with something like HAProxy, ou redis driver won’t do it for you. If you are using a cluster, it should be fine.


#3

The parameter slaveof doesn’t work if you have “enable_cluster”: true

I followed the redis cluster guide as much as possible. I get stuck in the section where it says to create different directories with different ports and to put one redis.conf inside each directories. I obviously don’t have to do this since I start my redis instance using the tyk.conf gateway.

My problem is when I run ./tyk --conf=tyk.conf , then I get the following errors :

[email protected]:/opt/tyk-gateway# ./tyk --conf=tyk.conf
INFO[0000] Hostname set:
INFO[0000] --> Using clustered mode
INFO[0000] Connection dropped, connecting…
INFO[0000] --> Using clustered mode
ERRO[0000] Could not EXPIRE key: could not complete command
INFO[0000] [HOST CHECK MANAGER] Starting Poller
INFO[0000] Setting up Server
INFO[0000] --> Standard listener (http)
INFO[0000] Detected 0 APIs
INFO[0000] Loading uptime tests…
INFO[0000] Loaded 0 policies
INFO[0000] Gateway started (v1.9.1.1)
INFO[0000] --> Listening on port: 8080
ERRO[0005] Multi command failed: could not complete command
ERRO[0005] Multi command failed: could not complete command
ERRO[0010] Multi command failed: could not complete command
ERRO[0010] Multi command failed: could not complete command
ERRO[0010] Could not EXPIRE key: could not complete command

So I haven’t continued the Redis tutorial which say to install redis-trib and run “gem install redis” to create the cluster. I was bothered by the errors I get simply trying to run the gateway.

Thank you for the fast reply. Greatly appreciated.


#4

Huh? Tyk doesn’t start Redis for you, you need to set that up yourself, since it has so many ways of being run (dev / prod / master+slave+hot-failover / cluster), Tykwon’t make that architecture decision for you.

That’s the bit when you create the actual cluster. You haven’t installed a cluster in that case, instead, you have a half-installed redis master/slave relationship (maybe). So the cluster flag in tyk.conf will not work since Tyk will try to speak to the redis DB as f it were a cluster, not as if it were a single instance.

If you can’t set up a Redis cluster using the guide, maybe just use a single instance? Or, if you must use a cluster, but don’t want to go through setting one up, maybe use a pre-built docker-based redis cluster, it’s much easier to get started and debug with?


#5

My goal is to have two different physical servers running on different locations to back up each other incase of failover. I’ve been told that the docker method was not the right approach and I’ve been trying to install redis using cluster mode.

I’ve been following this guide up to the steps where they tell you to run different redis instances to then build your cluster afterward running “gem install redis”.

I understand that Tyk won’t run a Redis instance for me. This is where it gets confusing for me in the tutorial. It says to run different Redis instances to build the cluster. Does this mean I need to to run one instance on each server and create the cluster with them?

Any clarifications would be appreciated,

thanks.

EDIT: I’ve installed Ruby, and ran gem install ruby and then tried to associate my first node with the second. I get the following error :

[email protected]:~/redis-3.0.7/src$ ./redis-trib.rb create --replicas 1 192.168.50.41:6379 192.168.50.42:6379

Creating cluster
*** ERROR: Invalid configuration for cluster creation.
*** Redis Cluster requires at least 3 master nodes.
*** This is not possible with 2 nodes and 1 replicas per node.
*** At least 6 nodes are required.

Is there any way to have two different physical servers running and synchronizing in case of a fail over or I must absolutely have 3 different ones (3 masters/3replicas - 6 nodes using cluster mode)?


#6

Hi,

A Redis cluster requires at least three hosts to reach quorum. It’s in the redis clustering docs I think.

If you really want a safe failover, why not use Redis master/slave with HAProxy hot-failover?

Since a cluster, in order to be truly “safe” will need multiple instances.

Cheers,
Maritn