Tyk and consul service descovery integration

Support Installation and Deployment
, ,
1

Hi guys,

Ive been testing out tyk with consul service descovery. But unfortunatly having an issue.

So we have service descovery set up. I can see in the logs that the ip and port is being resolved by tyk. The problem is tyk tries to parse the ip and port using url.parse the format is 1.0.0.2:22222. The parse function thows an exception saying it cant have a colon in it. I have read online that this can be due to the string being parsed not having the protocol on it for example http://1.0.0.2:22222.

Here lies my probem i have no way to make consul return a address with protocol. All the examples data i have seen returned from consul never include a protocol.

I think i could make this work without using non standard ports but this would involve giving each container a ip of its own which i dont really want to do.

Please advise

Thank you in advance.

2

Hey James

Looking at our own tests the format should be fine tyk/service_discovery_test.go at master · TykTechnologies/tyk · GitHub

Can you share how you have it configured in your API definition? In the tests for example the fields are setup like this: tyk/service_discovery_test.go at master · TykTechnologies/tyk · GitHub

Cheers
Josh

1 Like
3

Sure no problem at all :blush:

config:
“service_discovery”: {
“use_discovery_service”: true,
“query_endpoint”: “http://10.0.0.4:8500/v1/catalog/service/httpbin”,
“use_nested_query”: false,
“parent_data_path”: “”,
“data_path”: “Address”,
“port_data_path”: “ServicePort”,
“target_path”: “”,
“use_target_list”: true,
“cache_timeout”: 20,
“endpoint_returns_list”: true
}

Error received from tyk gateway docker logs:

time="Mar 03 18:00:16" level=error msg="[PROXY] [LOAD BALANCING] Couldn't parse target URL:parse "10.0.0.2:31973": first path segment in URL cannot contain colon"

As you can see from the error above I can see that tyk has been given the correct IP address and correct port of the service that I currently have in a nomad consul service mesh. Its just this exception is thrown above.

looking in the git hub I’m pretty sure we are hitting here:
tyk/reverse_proxy.go at master · TykTechnologies/tyk · GitHub),-%7D%20else%20%7B

Having a quick google around the second half of the exception message: first path segment in URL cannot contain colon

This seems to be a message you get when not passing http:// to the function url.Parse.

any help would be much appreciated

4

Thanks James - can you also share the response from your service catalog endpoint so I can do some quick testing too if possible.

5 
[
    {
        "ID": "5d5c41c2-b65a-3855-4119-102e0e72a765",
        "Node": "ip-10-0-0-2",
        "Address": "10.0.0.2",
        "Datacenter": "eu-west-1",
        "TaggedAddresses": null,
        "NodeMeta": null,
        "ServiceKind": "",
        "ServiceID": "_nomad-task-3c6891c4-f8db-ea07-0ee1-ce7cac6df9f3-frontend-httpbin-http",
        "ServiceName": "httpbin",
        "ServiceTags": [
            "http"
        ],
        "ServiceAddress": "10.0.0.2",
        "ServiceTaggedAddresses": {
            "lan_ipv4": {
                "Address": "10.0.0.2",
                "Port": 31973
            },
            "wan_ipv4": {
                "Address": "10.0.0.2",
                "Port": 31973
            }
        },
        "ServiceWeights": {
            "Passing": 1,
            "Warning": 1
        },
        "ServiceMeta": {
            "external-source": "nomad"
        },
        "ServicePort": 31973,
        "ServiceSocketPath": "",
        "ServiceEnableTagOverride": false,
        "ServiceProxy": {
            "Mode": "",
            "MeshGateway": {},
            "Expose": {}
        },
        "ServiceConnect": {},
        "CreateIndex": 5582,
        "ModifyIndex": 5582
    },
    {
        "ID": "5d5c41c2-b65a-3855-4119-102e0e72a765",
        "Node": "ip-10-0-0-2",
        "Address": "10.0.0.2",
        "Datacenter": "eu-west-1",
        "TaggedAddresses": null,
        "NodeMeta": null,
        "ServiceKind": "",
        "ServiceID": "_nomad-task-62558968-f06f-06ac-fdf2-341b57b6fbcd-frontend-httpbin-http",
        "ServiceName": "httpbin",
        "ServiceTags": [
            "http"
        ],
        "ServiceAddress": "10.0.0.2",
        "ServiceTaggedAddresses": {
            "lan_ipv4": {
                "Address": "10.0.0.2",
                "Port": 30874
            },
            "wan_ipv4": {
                "Address": "10.0.0.2",
                "Port": 30874
            }
        },
        "ServiceWeights": {
            "Passing": 1,
            "Warning": 1
        },
        "ServiceMeta": {
            "external-source": "nomad"
        },
        "ServicePort": 30874,
        "ServiceSocketPath": "",
        "ServiceEnableTagOverride": false,
        "ServiceProxy": {
            "Mode": "",
            "MeshGateway": {},
            "Expose": {}
        },
        "ServiceConnect": {},
        "CreateIndex": 5583,
        "ModifyIndex": 5583
    }
]
6

One more thing please - can you post me the full API definition

7

Sorry the swagger is to large to post here, but i took the raw output from this site API Specifications - httpbin.org

8

I was looking for the Tyk JSON definition for the API (what you would normally see in the raw definition part of API designer). If that’s too big too i’ll just crack on and check without it.

It looks like for some reason the scheme isnt being appended properly to your service discovery entries but it’s not obviously why.

9

The full definition is very large and over 3000 lines and i belive the forum is blocking me sending it.

10

Probably enough is just anything from root level so feel free to prune a lot of it

11

please find api doc here : https://sharetext.me/vbyppvdfum

12

Cheers James - I’ve got my own little demo going now and seeing the same errors as you. I’ll got chat to the engineering team as it doesn’t make much sense as the format in our discovery documents off just ip:port is the same we test against in our unit tests.

Will get back to you when I know more - hopefully just some niggly configuration issue rather than a bug.

(sidenote: I do have a way of modifying the input so it could work right away by proxying the query endpoint via Tyk itself and using a plugin but that’s a hack so will try and see how this is meant to work OOTB)

1 Like
13

Ok, looks like it is borked for all GW versions above 3.2. So for the moment if this is something in the hot path for your evaluation, the options are either step back gw version for now or I can provide you with a workaround using body transformation to cheat it basically. Let me know and apologies for you running into this.

14

Thank you that was my impression also glad to get it confirmed :slight_smile:

15

We have lowered the gateway version to 3.1.1 and are still experiencing issues, do you think we need to down grade the whole suite to fix this?

16

No it is a gateway only bug - I’d rather we kept you on a more recent GW and if you use this body transform template as in front of your query endpoint it will fix avoid the parsing bug.

[
{{range $index, $element := .}} {{if $index}},
    {
    "Address": http://{{ .Address}},
    "ServicePort": {{.ServicePort}}
    }{{else}}
    {
    "Address": http://{{ .Address}},
    "ServicePort": {{.ServicePort}}
    }
{{end}}{{end}}
]

Let me know if that makes sense or I can give you some more info on using the body transformation middleware.

Cheers

17

Okay so we have setup a second api, called “servicecatalog” we added the body transform using the body you sent us. We tested it in the UI and it seemed to work well.

We have then updated our httpbin api endpoint to use the new service catalogue endpoint but it does not change the error, we can see in the debug logs it still received the extended service catalogue with all other values. It seems the body transform is not being applied any idea why ?

18

This was user error, we had the template in the request rather then the response, we also noticed our data had an extra array wrapping it once the request had gone through tyk. So the template needed to be updated to:

[
{{range $index, $element := .array}} {{if $index}},
    {
    "Address": "http://{{ .Address}}",
    "ServicePort": {{.ServicePort}}
    }{{else}}
    {
    "Address": "http://{{ .Address}}",
    "ServicePort": {{.ServicePort}}
    }
{{end}}{{end}}
]

Now it is working thank you!

2 Likes
19

Awesome! Thanks for tweaking it too :muscle:

20

Hi Josh/James,
I am also trying to integrate Tyk with Consul and facing the same issue.
Could you please tell me where exactly the above template needs to be added ?

JFYI: I am using Tyk community and calling POST http://localhost:8080/tyk/apis to add the configuration.

Thanks.