Imported Google Group message.
Date:Wednesday, 25 March 2015 11:38:33 UTC.
How odd - to be honest I'm not that familiar with HaProxy either, I assume you are having redis load balanced behind haproxy in case the master node goes down and then the slave can still push signals. This makes sense for the Tyk gateway nodes, they maintain a pool (and try to reconnect) with a downed Redis host and so they should recover quickly if the redis master goes down.
However, for the host manager, since it isn't reconnecting at the moment, the pubsub connection would be lost anyway (I believe haproxy would sever the connection on failure and force client reconnect) there's an option redispatch, but I'm not sure if it would re-create the tcp connection with a slave and renegotiate a pubsub channel on the behalf of the client, so the host manager would fail even if the long-running connection worked.
Now the host_manager is nothing more than a pubsub / REST fanout, it receives the restart signal from Redis and then uses the REST api to enumerate the tyk nodes on the host (so it should be installed alongside your tyk gateway, the gateway will then pull new configurations from your DB, however it's connection pool should refresh if the connection fails.
That means it would be quite safe to just restart the host_manager process on failure using a script until we've got some reconnection code in place.
Some assumptions there about your setup, but it might work. Could you raise an issue in our GH repo to cover host manager reconnects?
- show quoted text -