Url rewrite match pattern not working as expected

Support
,
1

Hello!

I have an api definition with url_rewrites as follows:
“url_rewrites”: [
{
“path”: “/v4/cards/{id}”,
“method”: “GET”,
“match_pattern”: “^/v4/cards/(.)?",
“rewrite_to”: “http://card-lifecycle:3000/v4/migration/cards$1”
},
{
“path”: “/v4/cards”,
“method”: “POST”,
“match_pattern”: "^/v4/cards(.)?”,
“rewrite_to”: “http://card-lifecycle:3000/v4/migration/cards$1”
},
{
“path”: “/v4/cards”,
“method”: “GET”,
“match_pattern”: “^/v4/cards(.*)?”,
“rewrite_to”: “http://api:3000/v4/cards$1”
}
]
The proxy part is as follows
“proxy”: {
“listen_path”: “/v4/cards”,
“strip_listen_path”: false,
“target_url”: “http://card-lifecycle:3000”
},

My tests with incoming request GET /v4/cards is not working - from the logs (I activated debug logs), I don’t see that rewriter is fired.

I have Tyk version 5.3.4

2

I just wanted to share the solution and things I learned

{
  "name": "V4 Cards",
  "api_id": "67cbc305-87c8-4e44-b835-db237b7f6203",
  "org_id": "default",
  "disable_rate_limit": true,
  "use_keyless": false,
  "enable_coprocess_auth": true,
  "enable_context_vars": true,
  "definition": {
    "location": "header",
    "key": "version"
  },
...
    "allowed_headers": [
      "*"
    ],
    "exposed_headers": [],
    "allow_credentials": true,
    "max_age": 24,
    "options_passthrough": false,
    "debug": false
  },
  "version_data": {
    "not_versioned": true,
    "versions": {
      "Default": {
        "name": "Default",
        "global_headers": {
          "X-Firstparty-Request-Id": "$tyk_context.headers_X_Request_Id"
        },
        "use_extended_paths": true,
        "extended_paths": {
          "url_rewrites": [
            {
              "path": "^/renew",
              "method": "POST",
              "match_pattern": "^/v4/cards/renew(.*)?",
              "rewrite_to": "http://card-lifecycle:3000/v4/migration/cards/renew$1"
            },
            {
              "path": "^/([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})/limits",
              "method": "PATCH",
              "match_pattern": "^/v4/cards/(.*)/limits(.*)?",
              "rewrite_to": "http://card-lifecycle:3000/v4/migration/cards/$1/limits$2"
            },
            {
              "path": "^/([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})/limits",
              "method": "PUT",
              "match_pattern": "^/v4/cards/(.*)/limits(.*)?",
              "rewrite_to": "http://card-lifecycle:3000/v4/migration/cards/$1/limits$2"
            },
            {
              "path": "^/([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})/numbers$",
              "method": "GET",
              "match_pattern": "^/v4/cards/(.*)/numbers(.*)?",
              "rewrite_to": "http://card-lifecycle:3000/v4/migration/cards/$1/numbers$2"
            },
            {
              "path": "^/([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})$",
              "method": "DELETE",
              "match_pattern": "^/v4/cards/(.*)?",
              "rewrite_to": "http://card-lifecycle:3000/v4/migration/cards/$1"
            },
            {
              "path": "^/([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})$",
              "method": "GET",
              "match_pattern": "^/v4/cards/(.*)?",
              "rewrite_to": "http://card-lifecycle:3000/v4/migration/cards/$1"
            },
            {
              "method": "POST",
              "match_pattern": "^/v4/cards(.*)?",
              "rewrite_to": "http://card-lifecycle:3000/v4/migration/cards$1"
            },
            {
              "method": "GET",
              "match_pattern": "^/v4/cards(.*)?",
              "rewrite_to": "http://api:3000/v4/cards$1"
            }
          ]
        }
      }
    }
  },
...
    "driver": "grpc"
  },
  "proxy": {
    "listen_path": "/v4/cards",
    "strip_listen_path": false,
    "target_url": "http://card-lifecycle:3000"
  },
  "detailed_tracing": true
}

Things I learned:

  • the path in the url_rewrites entry, must not include the listen_path, which is stripped when it’s matched with the inbound path (regardless of how the property strip_listen_path is set)
  • the match_pattern instead is using the full listen_path (i.e. non-stripped) for the match
  • the path in the url_rewrites entries can contain a regex
  • (this is written in the doc, but should be highlighted better) if you use placeholders in the path (e.g., /books/{id}), the placeholder is replaced with (.*) which might screw up your rewrites
  • if your listen_path is /hello and you want to match GET /hello and POST /hello with different rewrites, don’t use path in the url_rewrites entry
  • the most specific path in the url_rewrites must come first in the array

Is it possible to create a merge request to improve docs?

3

Do you think that the above findings are correct and we could open a MR to help the community?

4

Hi!

Thanks for posting your findings which are correct.

This is the case when the listen path doesn’t end in a /. If it does end in a / then using a path of / works too.

This is correct. The dashboard product does this automatically for you, but when using the gateway alone you must do it manually. I’m sorry this is not made clear in the documentation.

That would be super helpful! Thanks!

Cheers,
Pete