Hybrid errors: slug mis-matching randomly?

We have two separate APIs defined

  • /user/

  • /user-license/

Randomly requests to /user-license/[email]/operation are being taken by Tyk but passed to the /user/ API.

So we end up w/ a failure message in the logs that looks like:

"request error: Requested endpoint is forbidden" api_id=9exxxx org_id=59xxxxx path="/-license/[email protected]/unassigned" server_name="https://some.backend:8443/user/" user_id= user_ip=""

Again this appears to be random its not all requests, but some requests. We have several docker containers running, and the issue seems to get pinned to one of N containers, i.e. it only occurs on one or two of N containers. Very odd and critical for us at this point.

noted w/ more detail in: http://support.tyk.io/support/tickets/1513

Cannot reproduce this behavior locally w/ docker quickstart. Only occurring in our hybrid setup

Appears that restarting one of the containers makes the issue go away, but eventually it re-appears.

Note: our configs are stable, no config updates are going on.

The hybrid cluster uses a shared redis.

Note for anyone else who stumbles on this: root issue and resolution described @ API loader is fragile when many endpoints are set up on a single domain · Issue #1271 · TykTechnologies/tyk · GitHub