Service Interruption - Hybrid users : Action Required

1

Hi All

We had a brief service interruption today, resulting in a short term outage for many customers.
Service was promptly restored for all users, but those using Hybrid Architecture need to take action

Hybrid Users Only:
Please re-generate your API Key in your dashboard in order for the connection to work (“System Management” -> “Users” -> “Edit User” -> “Reset Key”). An issue with our redis cluster caused RPC keys to expire, which has knocked out hybrid clients. We’re available to support if you have any queries.

We’re currently investigating further and putting mechanisms in place to prevent this from happening in future.

Please accept our sincere apologies for the inconvenience and let us know if you need any help in the meantime.

Regards

James

2

Update on this outage

Tyk Cloud had an issue with it’s key cache, which caused temporary write failures for new tokens and new registrations for Hybrid clients, it also temporarily affected cloud login.

The outage also caused us to temporarily work from an empty cache, which caused existing tokens to fail as they passed through our gateway.

Actions taken:

  • Key cache was fully restored to normal operation
  • All services that were using the key cache are now updated and back to normal

We will be continuing to review this situation so it does not happen again.

Please direct any issues to our support desk.

Many thanks,
Martin