Location of Developer API keys

Support Setting up Tyk
1

Imported Google Group message. Original thread at: Redirecting to Google Groups Import Date: 2016-01-19 21:14:16 +0000.
Sender:[email protected].
Date:Wednesday, 27 May 2015 09:48:02 UTC+1.

I’m wondering how developer API keys are generated/stored. They don’t seem to be placed in the tyk_keys table, so I’m interested in how Tyk itself determines a developer’s access to an API.

With the example key ‘55657c6b6cf82c2de20000017478bdb2dfb64d3e6620693cd57cd524’ I can see the first part is the organization ID ‘55657c6b6cf82c2de2000001’, but the latter portion doesn’t seem to match anything.

My reason for asking is essentially because there’s no API to recover the keys once they’ve been created. We’d prefer to allow them to be retrievable in the future, so information about how they’re generated / what they’re composed of would be extremely useful.

2

Imported Google Group message.
Sender:Martin Buhr.
Date:Thursday, 28 May 2015 11:09:50 UTC+1.

Hi Rhys,

Essentially we’re using the portal backend, but not the frontend, so for example we currently automate building up the catalogue, policies, developer lists, etc. The benefit on piggybacking on the existing portal infrastructure is that we retain visibility from an admin perspective through the Portal Management features of the Dashboard.

You could, theoretically do a lot of this stuff yourself, the developers section has an endpoint (undocumented though, you’d need to reverse engineer it based on what the dashboard is up to when it creates a developer record).

You would need to manually add the API ID -> Key references in a follow-up developer PUT request

The only key part that’s missing from the Dashboard’s API is Developer API key creation, which was why I was asking for more details about how this process happens. Are you recommending that we simply abandon Tyk’s portal entirely and do this whole process ourselves?

There’s no API for this (at least that’s documented), everything happens via Key Requests, and those are created using form posts (see option 2)

Option 1 New flow for you to do it manually:
Pre-requisites: You have an API Definition, and policy per API set up already
Create the API Key using the Advanced Management API, make sure that it is populated with the basic data from the policy object so it gets created properly,a lso make sure to assign the policy ID
Create a developer using the dashboard API (it’s not documented, but you should be able to get the gist from watching what the dashboard is up to)
You might be able to ut the API ID -> Key map directly into the developer object on create, otherwise you may need to do it on a subsequent PUT
Unfortunately the create API key doesn’t return an ID so you’d need to get the developer list and find the email address manually so you can lookup the user later
Save whatever data you need so you can hook back in
Alternatively you could use hijack the forms on the portal and simply POST programatically to those endpoints for both the Developer Signup and Key Request, but you’d need an http client that can handle cookies so you retain a session, you would need to mock out this journey:

  1. Sign up
  2. Log in as user
  3. Submit key request form (asuming key requests are approved automatically, this will handle key creation)
  4. Lookup the user in the dashbaord and get their key so you can give it to them

Option 2 might be more palatable since you don’t need to meddle with the internal machinery, a lot happnes when a key request is approved.

We will, in the longer term, be expanding the key requests API to handle create and edits, but for the purposes of the portal it wasn;t seen as necessary since the point of the portal was to take control over this process completely.

Hope that helps.

Thanks,
Martin

3

Imported Google Group message.
Sender:[email protected].
Date:Wednesday, 27 May 2015 09:57:42 UTC+1.

Hi Rhys,

Tyk stores all keys in Redis, as keys are managed by the gateway, not the dashboard.

If you have hashed keys installation (default), keys are generated, exposed for communication to the end user and then hashed and stored, never to be retrieved in their original format again, this is a security precaution, a otherwise all keys would be exposed if your redis DB were breached, whereas the hashes are useless.

If you are running an unhashed installation, then those keys will be in redis and in the format you described and can be retrieved using the Tyk REST API and the Advanced Managmeent API, though keys are just that - random tokens, keeping an allocation against some kind of identity obejct would be up to the the implementer.

The latter half of the token is the random element, keys are segmented by OrgID and then by a UUID to keep them unique.

If you are using the portal, then it does the identity -> key mapping for you, in the developer record there is a map between the API ID and the developer’s key which can be retrieved in an unhashed installation, in a hashed installation it stores the hash, but at least you have a record of who owns what key. There is an API for this (the dashboard app uses it to get the data), but it isn’t documented. If you take a peek at the network traffic when you open a developer record in the dashboard you’ll see the request, it’s a straightforward GET.

If you want to be able to retrieve keys you’ll need to set your Tyk installation into unhashed mode (needs to be done both in the tyk.con and the tyk_analytics.conf), this isn’t reversable, you can;t convert from one mode to the other without bizarre side-effects, just a heads up.

Thanks,
Martin

4

Imported Google Group message.
Sender:Martin Buhr.
Date:Thursday, 28 May 2015 10:11:08 UTC+1.

Hi Martin,

Thanks for the information. Unhashed mode seems to be exactly what I was looking for. We’re not currently using the portal provided by the dashboard as we have our own web app that we’re planning on hooking into Tyk.

In terms of generating these keys, from investigating a bit further the workflow internally seems to be to create a key associated to a specific organization, which is then stored in redis as apikey-{api key}, with the full details in the body of that entry. This key is then added to the developer’s api_keys map, with the key being the API ID, and the value being the API Key.

Based on this information, I should be able to automate generating developer API keys by first sending off a key request, then updating the developer object with the resulting key ID. Are there any downsides to this approach? I haven’t found any evidence that keys generated by an admin user are associated to that user, so in theory this approach should work.

5

Imported Google Group message.
Sender:[email protected].
Date:Thursday, 28 May 2015 10:36:31 UTC+1.

Hi Rhys,

Answers below:

Thanks for the information. Unhashed mode seems to be exactly what I was looking for. We’re not currently using the portal provided by the dashboard as we have our own web app that we’re planning on hooking into Tyk.

Note that the portal is the thing which handles all the developer records etc. Tyk Gateway does not have any conept of a developer, the portl/dashboard adds this abstraction as an integration feature, it basically does everything via the tyk REST API and stores data, that’s it.

In terms of generating these keys, from investigating a bit further the workflow internally seems to be to create a key associated to a specific organization, which is then stored in redis as apikey-{api key}, with the full details in the body of that entry. This key is then added to the developer’s api_keys map, with the key being the API ID, and the value being the API Key.

Correct, except that there is no api_keys map if you are not using the developer portal. Also, the record that is mapped in the portal for API ID is arbitrary, as a key can grant access to multiple upstream APIs, we’re thinking of re-jigging this data structure to make it more sensible. Again, don;t rely on the developer record unless youa re using the portal.

Based on this information, I should be able to automate generating developer API keys by first sending off a key request, then updating the developer object with the resulting key ID. Are there any downsides to this approach? I haven’t found any evidence that keys generated by an admin user are associated to that user, so in theory this approach should work.

Assuming that the developer object in question here is your own, stored in your cusotm portal, then yes, this is what the Tyk portal does, essentially:

  1. Grab a key template (basics of a key, some base settings need to be present such a an initial access rights rule etc.)
  2. Set the apply_policy field to the key policy to be used to manage this class of key - this is super handy and we reocmmend doing this so that you can control access rules, quotas, rate limits etc across groups of keys
  3. Send the create key request to Tyk with this session object
  4. Tyk replies with the KeyID,
  5. Store this in your local developer record - instead of storing against the API ID, we suggest storing against the Policy ID, this way you can buklk manage these keys and their access rules, as well as managing the policies themselves with the Tyk REST API, makes integration / tiering easier.

In tyk, tokens are treated as ephemeral, as they may be revoked or destroyed at any time, keeping your DB version in sync with the Tyk gateway can cause lookup errors, just be aware of that. There are settings to handle session lifetimes and key expiry times both in the key temlate and globally, see the docs for the details on this.

Hope that helps.

Cheers,
Martin

1 Like
6

Imported Google Group message.
Sender:Martin Buhr.
Date:Thursday, 28 May 2015 10:43:24 UTC+1.

Hi Martin,

That might be an issue then. We were hoping to directly use Tyk’s concept of Developers i.e. what you get via the portal with our own application. Our app isn’t simple enough to be embedded in a portal page template, so we can’t be directly served from Tyk unfortunately.

Essentially we’re using the portal backend, but not the frontend, so for example we currently automate building up the catalogue, policies, developer lists, etc. The benefit on piggybacking on the existing portal infrastructure is that we retain visibility from an admin perspective through the Portal Management features of the Dashboard.

The only key part that’s missing from the Dashboard’s API is Developer API key creation, which was why I was asking for more details about how this process happens. Are you recommending that we simply abandon Tyk’s portal entirely and do this whole process ourselves?

Many thanks,
Rhys.

7

Imported Google Group message.
Sender:[email protected].
Date:Thursday, 28 May 2015 11:09:50 UTC+1.

Hi Rhys,

Essentially we’re using the portal backend, but not the frontend, so for example we currently automate building up the catalogue, policies, developer lists, etc. The benefit on piggybacking on the existing portal infrastructure is that we retain visibility from an admin perspective through the Portal Management features of the Dashboard.

You could, theoretically do a lot of this stuff yourself, the developers section has an endpoint (undocumented though, you’d need to reverse engineer it based on what the dashboard is up to when it creates a developer record).

You would need to manually add the API ID -> Key references in a follow-up developer PUT request

The only key part that’s missing from the Dashboard’s API is Developer API key creation, which was why I was asking for more details about how this process happens. Are you recommending that we simply abandon Tyk’s portal entirely and do this whole process ourselves?

There’s no API for this (at least that’s documented), everything happens via Key Requests, and those are created using form posts (see option 2)

Option 1 New flow for you to do it manually:
Pre-requisites: You have an API Definition, and policy per API set up already
Create the API Key using the Advanced Management API, make sure that it is populated with the basic data from the policy object so it gets created properly,a lso make sure to assign the policy ID
Create a developer using the dashboard API (it’s not documented, but you should be able to get the gist from watching what the dashboard is up to)
You might be able to ut the API ID -> Key map directly into the developer object on create, otherwise you may need to do it on a subsequent PUT
Unfortunately the create API key doesn’t return an ID so you’d need to get the developer list and find the email address manually so you can lookup the user later
Save whatever data you need so you can hook back in
Alternatively you could use hijack the forms on the portal and simply POST programatically to those endpoints for both the Developer Signup and Key Request, but you’d need an http client that can handle cookies so you retain a session, you would need to mock out this journey:

  1. Sign up
  2. Log in as user
  3. Submit key request form (asuming key requests are approved automatically, this will handle key creation)
  4. Lookup the user in the dashbaord and get their key so you can give it to them

Option 2 might be more palatable since you don’t need to meddle with the internal machinery, a lot happnes when a key request is approved.

We will, in the longer term, be expanding the key requests API to handle create and edits, but for the purposes of the portal it wasn;t seen as necessary since the point of the portal was to take control over this process completely.

Hope that helps.

Thanks,
Martin

8

Imported Google Group message.
Sender:Martin Buhr.
Date:Thursday, 28 May 2015 13:46:13 UTC+1.

Hi Martin,

That makes sense, we’ll probably go with the latter approach and implement the server half of the portal ourselves, and just reference Tyk keys and policies via the admin API. That seems to be the simplest and most reliable approach.

Thanks for the advice.

Rhys.

9