Can no longer authenticate with Tyk Dashboard API Access Credentials after restarting services

Support
, ,
1

We’re running into an issue where, when we cycle tyk gateway and dashboard and related services, our automation can NO LONGER authenticate to the Dashboard API using the authorization header. Doesn’t matter which Dashboard user access credential we use, including the dashboard admin account. They all get a HTTP 401.

However, the Portal website itself continues to function in every way (which appears to leverages the same API - the difference between these is that the Dashboard web site seems to use cookies instead of an auth header). I’ve tried copying the access token out of the Dashboard User profile and issue direct curl requests to the Dashboard API and get 401 auth failures as well.

Everything works well until we cycle the services (redis, mongo, tyk-gateway, tyk-dashboard) then all access to the Dashboard API fails. I’ve combed thorough the logs for redis, mongo, tyk-gateway, and tyk-dashboard and all the services and I don’t see anything pointing to an issue - except when trying to authenticate to the Dashboard API using the authentication header - otherwise the logs seem to indicate everything is running fine and as expected.

The tyk dashboard log seems to indicate that redis does not have the information needed to authenticate the Portal Users’ API key. These are the only two log entries generated from all services when I attempt the Dashboard API call:

tyk-dashboard_1  | time="Aug 23 22:32:40" level=error msg="Session key not found" error="redigo: nil returned"
tyk-dashboard_1  | time="Aug 23 22:32:40" level=error msg="Could not get session" error="redigo: nil returned"

The authorization header hidden below is straight out of the main dashboard admin account - from what I can tell, it doesn’t matter which Dashboard API endpoint i call - they all return a 401.

curl http://tyk-dashboard:3000/api/users/5b7f28ddf7770ca51b453782 -H "authorization: *********************************************"

401 Unauthorized
Access-Control-Allow-Credentials: true
Cache-Control: no-store, no-cache, private
Content-Security-Policy: default-src * data: blob: 'unsafe-inline' 'unsafe-eval'; script-src * data: blob: 'unsafe-inline' 'unsafe-eval'; connect-src * 'unsafe-inline'; img-src * data: blob: 'unsafe-inline'; frame-src *; style-src * 'unsafe-inline';
Content-Type: text/plain; charset=utf-8
Strict-Transport-Security: max-age=63072000; includeSubDomains
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Date: Fri, 24 Aug 2018 12:52:47 GMT
Content-Length: 58
None
{
     "Message": "Not authorised",
    "Meta": null,
    "Status": "Error"
 }

We’re using the following docker images:
tykio/tyk-gateway:v2.7.0
tykio/tyk-dashboard:v1.7.0
mongo:3.2
redis

Any help would be greatly appreciated.

Thanks,
Jason

2

Hello!

Can you clarify what you mean by cycle?

It feels like your redis configured without persistence, and during its restart dashboard session data gets lost.

Can you confirm that this is your case?

Thank you.

3

we are using docker compose and we’re doing a restart

docker-compose -f docker-compose.yml -f docker-local.yml restart

4

Ah - I thought about that and then read this and think I misunderstood it.
https://tyk.io/docs/frequently-asked-questions/redis-persistence-using-containers/

it isn’t configured for persistence because it would very quickly get very large - I took that to mean it’s not necessary.

5

Yes, sorry tht FAQ refers to a particular setup and isn’t super clear. You want persistence on restart :slight_smile:

6

Excellent. Can do! Thank you.

7

So what happens if I lose redis? How do we get the keys back in there?

8

You should regularry backup your redis, or use cloud provider like AWS ElastiCache, which will do it for you. You can read about redis backups here: https://redis.io/docs/management/persistence/ at “Backing up Redis data” and “Disaster recovery” sections.

Cheers.