Redis crashes unexpectedly and i lose all the oauth clients generated...any help


#1

Imported Google Group message. Original thread at: https://groups.google.com/forum/#!topic/tyk-community-support/SH5uvWasPOo Import Date: 2016-01-19 21:33:48 +0000.
Sender:[email protected].
Date:Monday, 7 December 2015 12:52:52 UTC.

Hi,

I have setup apis and generated oauth clients using tyk dashboard , everything works fine . But something happening overnight and losing all my clients and when i check redis log , it says received shutdown signal and some warning about memory limit . Please point me towards a viable fix .

Thanks
Kalyan


#2

Imported Google Group message.
Sender:Martin Buhr.
Date:Monday, 7 December 2015 13:14:28 UTC.

Hi Kalyan,

Sounds like you deployed redis on a small box with no persistence settings… Redis is an in-memory store, it does not write to disk by default, unless you specifically tell it to (and then you have issues around disk space management). The reason is speed, since it’s an in-memory store, it’s potentially volatile but incredibly fast. We get round this in Tyk cloud through multiple master/slave multi-AZ redis instances that are snapshotted regularly and have hot-failover.

This is a well documented “issue” with redis: http://stackoverflow.com/questions/8585275/my-redis-server-crashed-and-i-lost-all-my-data-because-its-all-in-memory - it has pro/s and cons but since tyk is all about speed we optimise for that and work around the persistence issue in other ways.

Your DB probably crashed because it ran out of RAM, so you’ll need to manage that - either by adding more RAM to the instance, or by ensuring that Redis doesn’t fill up too fast, the only thing that would be causing that is a high analytics purge value.

Basically - your issue here is with incorrect setup and management of Redis :slight_smile:

Cheers,
Martin