Use version in target URL

Imported Google Group message. Original thread at: Redirecting to Google Groups Import Date: 2016-01-19 21:18:07 +0000.
Sender:Gavin Williams.
Date:Friday, 10 July 2015 17:14:07 UTC+1.

Afternoon all

I’m currently evaluating Tyk for use as an API gateway on a new Microservice platform that we are building.

The microservices are all being built using Dropwizard, which seems by default to have the ‘version’ as the first part of the server-side path, e.g.: http://dropwizard-node:8080/v1/api

I’m currently trying to work out how to map this logic into Tyk.

I’ve defined the API, which has generated an ID of ‘714007cd6f1347906a9218a0a7ad2213’, setup the target URL of the API to ‘http://dropwizard-node:8080’, and also enabled the ‘URL or Form’ version manager.
However if I submit a request via tyk, I get the following error:
curl -i http://tyk.local:8080/714007cd6f1347906a9218a0a7ad2213/v1/api
HTTP/1.1 403 Forbidden
Content-Type: application/json
X-Generator: tyk.io
Date: Fri, 10 Jul 2015 16:07:29 GMT
Content-Length: 48

{
“error”: “Version information not found”
}%

If I specify a URL encoded ‘version=v1’ value, it works:
curl -i http://tyk.local:8080/714007cd6f1347906a9218a0a7ad2213/v1/api\?version\=v1
HTTP/1.1 200 OK
Content-Type: application/json
Date: Fri, 10 Jul 2015 16:08:59 GMT
Vary: Accept-Encoding
X-Ratelimit-Limit: 0
X-Ratelimit-Remaining: 0
X-Ratelimit-Reset: 0
Content-Length: 83
{“meta”:{“totalCount”:0,“offset”:0,“limit”:100},“catalogues”:[],“notifications”:[]}

However I’m now left with two ‘v1’ annotations, which doesn’t look great. So is there a way that tyk can pull the ‘/v1’ from the URL, and match that to the defined version?

Or alternatively, is there a way that tyk can insert the version into the target URL?
As if I try to remove the ‘/v1’ from the request, I get a 404 from the backend, as it’s just request ‘http://dropwizard-node:8080/api’…
curl -i http://tyk.local:8080/714007cd6f1347906a9218a0a7ad2213/catalogues\?version\=v1
HTTP/1.1 404 Not Found
Cache-Control: must-revalidate,no-cache,no-store
Content-Length: 299
Content-Type: text/html; charset=ISO-8859-1
Date: Fri, 10 Jul 2015 16:13:13 GMT
X-Ratelimit-Limit: 0
X-Ratelimit-Remaining: 0
X-Ratelimit-Reset: 0

Let me know if I haven’t been clear on anything above, or if I’ve fundamentally missed anything…

Regards
Gavin

Imported Google Group message.
Sender:Martin Buhr.
Date:Friday, 10 July 2015 17:43:58 UTC+1.

Hi Gavin,

Currently Tyk does not support pulling a version number out of the URL, when we first investigated this feature we found its not considered best practice since it changes the resource location every time a new version is released, so we only support parameter or header based versioning for now.

Since then we’ve found that people use path based versioning anyway, so we’ve added it to our roadmap for the next version (best laid plans).

Currently the only thing you can do is set up multiple APIs, then set their listen paths to contain the version number.

When you create keys, you simply give them access to both APIs.

This kind of defeats the purpose of versioning on an API basis though.

There may be something you can do with the dynamic JavaScript middleware, as it gives you access to the request object (in particular the path), so you could inject the v1/ in a post-processing middleware function (https://tyk.io/v1.7/plugins/middleware-scripting/)

(Obviously you can do it the other way around and inject the version header from the URL in a preprocessing function, but post processing might actually be less resource intensive and introduce less latency.)
Hope that helps. As I said we’ll be looking at this in the next version.

Cheers,
Martin

Imported Google Group message.
Sender:Gavin Williams.
Date:Monday, 13 July 2015 10:52:51 UTC+1.

Martin

Cheers for the quick reply…

Yeh, I’m inclined to agree that having the version as the root of the resource URL doesn’t make for a great delivery experience…

I’ll have a go and see if I can work up a Post-processing middleware function…

Gav

  • show quoted text -

Imported Google Group message.
Sender:Gavin Williams.
Date:Monday, 13 July 2015 12:31:13 UTC+1.

OK, it looks like either I’m doing something wrong, or something isn’t working…

I’ve defined an initial testing JS file[1] just to confirm that I can get the middleware POST processor to fire…

However even though the debug logs[2] suggest that the JS has been loaded, I’m not seeing any output from my logging :S

As far as I can tell, the custom_middleware options is set correctly in the API[3], and the file is obviously being loaded as I’m seeing “INFO[0000] JSVM LOG: insertVersion middleware initialised” in the tyk logs…

Any ideas?

Cheers
Gavin

[1] Tyk Middleware testing... · GitHub
[2] Tyk Middleware testing... · GitHub
[3] Tyk Middleware testing... · GitHub

  • show quoted text -

Imported Google Group message.
Sender:Martin Buhr.
Date:Monday, 13 July 2015 12:57:26 UTC+1.

Hi Gavin,

It’s not running because your API is keyless, open APIs don’t currently have dynamic middleware support.

The reason for this is because open Apis have no session context (no token to bind to).

If you make the API use an auth token then the middleware will run.

We’re looking to add JSVM support to open APIs in future versions.

Let me know if that helps at all.

Cheers,
Martin

  • show quoted text -

  • show quoted text -


You received this message because you are subscribed to the Google Groups “Tyk Community Support” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web, visit https://groups.google.com/d/msgid/tyk-community-support/7ffc073f-d259-40d4-837c-7e6a066510b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Imported Google Group message.
Sender:Martin Buhr.
Date:Monday, 13 July 2015 13:57:33 UTC+1.

Hi Gavin,

This feature is now in our master branch, it’s silly not to be able to use dynamic middleware on Open APIs, so we’ve made the changes as necessary - will be in next release.

Cheers,
Martin

Imported Google Group message.
Sender:Gavin Williams.
Date:Monday, 13 July 2015 14:02:38 UTC+1.

Wow, that was quick :slight_smile:

I’ll see if I can get the API setup for a key-based access, and confirm back…

Cheers
Gavin

  • show quoted text -

Imported Google Group message.
Sender:Martin Buhr.
Date:Monday, 13 July 2015 14:04:27 UTC+1.

Hi Gavin,

It turns out it was trivial to change, there’s some contraints to the middleware code (i.e. you reply back with empty obejcts etc.) but there’s no reason for it to not work. We ran some tests and it all seems to be in order, including aquick URL inserter - it seemd to work just fine :slight_smile:

Cheers,
Martin

Imported Google Group message.
Sender:Gavin Williams.
Date:Monday, 13 July 2015 14:49:06 UTC+1.

Next challenge then :slight_smile:

I’m trying to get a developer portal setup, but dont seem to be having any luck…

I’ve defined an org of ‘Test’, and created a user in that org.
Then went and defined the API in the Catalogue, and added a default policy.
I’ve also defined a Page called ‘Welcome’, let everything else blank, and ticked the ‘make homepage’ option.

When I nav to: http://tyk.local/Test/portal, I get ‘Home page not found’.

Any pointers?

Cheers
Gavin

  • show quoted text -

Imported Google Group message.
Sender:Martin Buhr.
Date:Monday, 13 July 2015 15:23:17 UTC+1.

Ah, so portals can be a little tricky.

First off, you’ll need to make sure there’s a settings document in place for your Portal - just visit the “Settings” page of the portal and it will auto-create it.

You’re on the right track, if you are accessing the portal directly (not recommended), it will be under: http://tyk.local/{org-id}/portal/ (that trailing slash is important, and you will need to use the organisation ID instead of the slug). You will need to glean the org-id by looking at your database or by pulling it via the REST API, it’s also part of every object that gets used in the dashboard, so you could jsut take a look at the network view and pull it out of one of the JSON documents.

It’s not really meant to be used that way though, when you visit the portal home page, it will work, but the other navigation links will not, that’s because they are hard-coded to root (/) because the portal is meant to be exposed as a subdomain with a URL rewrite via NginX. This sounds crazy, but it’s all so we can have a clean way to segment out Organisation portals and a lot of flexibility. We even have a service that manages all this URL handling for you called the host manager (https://tyk.io/v1.7/host-manager/).

To get you started quickly though, you might want to edit the file portal/templates/navigation.html and add “/{{.OrgId}}” in front of all the links so that it gets auto-included (you’ll need to restart the tyk dashboard) once you’ve made the change.

e.g. From this:

    {{ range $index, $menuItem := .Menus.Main}}
  • {{$menuItem.Title}}
  • {{ end }}
  • API Catalogue
  • {{ if not .PortalConfig.DisableSignup }} {{ if not .UserData }}
  • Register
  • {{ end }} {{ end }} {{ if not .PortalConfig.DisableLogin }} {{ if not .UserData }}
  • Log in
  • {{ end }} {{ end }}

To this:

    {{ range $index, $menuItem := .Menus.Main}}
  • {{$menuItem.Title}}
  • {{ end }}
  • API Catalogue
  • {{ if not .PortalConfig.DisableSignup }} {{ if not .UserData }}
  • Register
  • {{ end }} {{ end }} {{ if not .PortalConfig.DisableLogin }} {{ if not .UserData }}
  • Log in
  • {{ end }} {{ end }}

Let me know how you get on :slight_smile:

Cheers,
Martin

Imported Google Group message.
Sender:Gavin Williams.
Date:Monday, 13 July 2015 15:49:17 UTC+1.

Martin

Cheers again for prompt reply…

So I went and set-up Nginx to proxy on Tyk portals, and when I changed from the Org-Name to Org-ID, I was able to get a login page :slight_smile: :smiley:

Was then able to login as my test user, view the API catalogue, and request a key for my test catalogue…

However when I try and use that key, I’m not getting a ‘HTTP 429 Too Many Requests’ response… Which sounds a bit strange considering I’ve only just registered the key… :S
The policy is set to allow 1000 requests in 60 seconds, with a ‘Max Quota’ of -1. It’s also got access rights to the API I’m attempting to use…

Debug logs are here: Tyk Middleware testing... · GitHub

Any ideas?

Cheers
Gavin

  • show quoted text -

Imported Google Group message.
Sender:Gavin Williams.
Date:Tuesday, 14 July 2015 08:59:24 UTC+1.

Ah, that looks better…

Can see Tyk finding and loading policies now… :slight_smile:

Right, lets see if can get back to testing the middleware :slight_smile:

Cheers
Gavin

On 13 Jul 2015, at 20:32, Martin Buhr [email protected] wrote:

Aha! I think I found it… It was buried in the debug logs:

NFO[0000] Loading policies
INFO[0000] No policy record name defined, skipping…
In your tyk.conf the policies section is empty, you’ll need to change:

policy_source:“mongo” and policy_record_name: “tyk_policies”.

This means Tyk will load your policies at startup and then apply them.

That explains why there was a rate limit error.

I should have seen it earlier, but I may have been hungry… Apologies :-/

Let me know if that fixes things :slight_smile:

Cheers,
Martin


From: Gavin Williams [email protected]
Sent: Monday, July 13, 2015 19:37
Subject: Re: Use version in target URL
To: Martin Buhr [email protected]

No worries… Just updated the gist with the latest API output.

Cheers
Gav

On 13 Jul 2015, at 19:17, Martin Buhr < [email protected]> wrote:

Hi Gavin,
Ok, have tested with a similar configuration and I can’t seem to replicate (with both hashed and un-hashed configurations). It might be in the API Definition. I think the one in the gist is probably out of date, would you mind sharing that too?
Cheers,
Martin
From: Gavin Williams [mailto:[email protected]]
Sent: 13 July 2015 18:36
To: Martin Buhr
Subject: Re: Use version in target URL
OK, creating a key manually without a policy seems to work… Once I set a long-enough expiry time…
Have attached my 2 conf files.
Let me know if you need anything else to assist.
Cheers
Gavin

Imported Google Group message.
Sender:Gavin Williams.
Date:Tuesday, 14 July 2015 09:27:30 UTC+1.

OK, I’m back to testing the middleware function now that I’ve got a working Key :slight_smile:

However I still seem to be hitting an issue…

I now seem to be getting an error when running through an API request. However the request completes without issue…
The error I’m seeing is:
ERRO[0008] Failed to decode middleware request data on return from VM: invalid character ‘u’ looking for beginning of value

And a full log is here: Tyk Middleware testing... · GitHub

Any ideas?

Cheers
Gavin

  • show quoted text -