Unable to integrate Tyk Gateway and Keycloak

Hi there,

I am trying to setup a POC using Docker and Tyk Gateway 4.1, involving one protected API, integrating with Tyk API Gateway, and a 3rd party OIDC service Keycloak. Basically what I am trying to achieve here is quite a common setup, after going a login flow with Keycloak, the client tries to access a protected api by sending a JWT token via HTTP headers and the Tyk API Gateway will verify the legitimacy of this JWT token using the JWKS retrieved from the Keycloak server, and subsequently accepts or rejects the request.

However, I am having some trouble getting the verification part to work, and have tried the following approaches.

Somehow the system thinks I am trying to embed links and new users can’t have more than 2 so you will notice that the http and https protocols are missing colons… And also, I can’t upload images because of some issue in the system as well, so please pardon me.

Approach 1: Verify directly with Keycloak server using Docker service name
Note: JWT Token contains “iss” claims which is the proxied URL (xxx.xxx.xxx.xxx:8080/keycloak/realms/…)

Relevant configuration:

"openid_options": {
    "providers": [
      {
        "issuer": "http//keycloak/realms/master",
        "client_ids": {
          "YWNjb3VudA==": "default"
        }
      }
  ]
}
  1. Client attempts to access protected API and sends JWT in headers to Tyk Gateway (which is exposed on the host PC, xxx.xxx.xxx.xxx:8080/protected/).
  2. Tyk Gateway attempts to verify token using http//keycloak/realms/…

This fails because of the following:
tyk-gateway_1 | time=“Nov 11 05:50:39” level=warning msg=“JWT Invalid” api_id=protected-api api_name=“Protected API” error=“Validation error. Validation error. No provider was registered with issuer: https//xxx.xxx.xxx.xxx/keycloak/realms/master” mw=OpenIDMW org_id=default origin=172.25.0.1 path=“/protected/”
tyk-gateway_1 | time=“Nov 11 05:50:39” level=warning msg=“Attempted access with invalid key.” api_id=protected-api api_name=“Protected API” key=“****JWT]” mw=OpenIDMW org_id=default origin=172.25.0.1 path=“/protected/”

I guess it’s because there’s a mismatch in the “iss” claim and the provider object in openid_options:

Possible solution: Have an option that can ignore issuer matching?

Approach 2: Verify with Keycloak server using going through Tyk Gateway API (going through itself)

Relevant configuration:

"openid_options": {
    "providers": [
      {
        "issuer": "https//xxx.xxx.xxx.xxx/keycloak/realms/master",
        "client_ids": {
          "YWNjb3VudA==": "default"
        }
      }
  ]
}
  1. Client attempts to access protected API and sends JWT in headers to Tyk Gateway (which is exposed on the host PC, xxx.xxx.xxx.xxx:8080/protected/).
  2. Since the “iss” claim and provider object issuer matches (https//xxx.xxx.xxx.xxx/keycloak/realms/master), the previous approach’s issue is no longer there.

When using SSL/TLS, in a production or staging environment, I think there will be no issue as we will be using legitimate domain names with signed certificates. However, in development, because we’re unable to use localhost in a Docker environment (localhost in host PC and localhost in Tyk Gateway is different), we are forced to used IP addresses, and Tyk Gateway will throw the following error:

tyk-gateway_1 | time=“Nov 11 06:16:05” level=warning msg=“JWT Invalid” api_id=protected-api api_name=“Protected API” error=“Validation error. Validation error. Failure while contacting the configuration endpoint https//xxx.xxx.xxx.xxx/keycloak/realms/master/.well-known/openid-configuration.” mw=OpenIDMW org_id=default origin=172.25.0.1 path=“/protected/”
tyk-gateway_1 | time=“Nov 11 06:16:05” level=warning msg=“Attempted access with invalid key.” api_id=protected-api api_name=“Protected API” key=“****JWT]” mw=OpenIDMW org_id=default origin=172.25.0.1 path=“/protected/”
tyk-gateway_1 | 2022/11/11 06:16:05 http TLS handshake error from 172.25.0.1:56400: remote error: tls: bad certificate

I believe this is due to the mismatch of the CN in the self-signed certificate:
`
curl: (60) SSL: certificate subject name ‘example.com’ does not match target host name ‘xxx.xxx.xxx.xxx’
More details here: https//curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
`
I have tried the following to no avail:

  1. Add the certificate to the /etc/ssl/certs.
  2. Under the /protected/ api, add proxy.transport.ssl_insecure_skip_verify = true
  3. Under the /protected/ api, add proxy.transport.ssl_force_common_name_check= false
  4. Under the tyk gateway configuration, add proxy_ssl_insecure_skip_verify = true
  5. Under the tyk gateway configuration, add jwt_ssl_insecure_skip_verify = true
  6. Under the tyk gateway configuration, add ssl_force_common_name_check = false

We probably need some way to either: 1. Ignore the issuer value mismatch, 2. Ignore any SSL/TLS related errors if Tyk Gateway needs to make a request that passes through itself.

Any suggestions?

Thank you.

Prakoso

There’s an unrelated error even when using http, I’ve searched online and found that apparently you must inject the policy id into the JWT claim? I’ve already done that and also use the jwt_policy_field_name to point to “default” (policy id) in the api definition, but still doesn’t work.

tyk-gateway_1 | time=“Nov 11 08:23:08” level=error msg=“No matching policy found!” api_id=protected-api api_name=“Protected API” mw=OpenIDMW org_id=60f08cb4b4c0be0001a87762 origin=172.25.0.1 path=“/protected/requests”

Hi @Osokarp ,

Welcome to the community :partying_face:

Great effort :muscle:

Is your gateway using SSL? Think you might need this http_server_options.ssl_insecure_skip_verify.
You can share your tyk.conf for a review too.

Hey, thanks! Appreciate the great work you guys are doing!

Anyways here is the config I am using:

{
  "log_level": "info" ,
  "listen_port": 8080,
  "secret": "352d20ee67be67f6340b4c0605b044b7",
  "template_path": "/opt/tyk-gateway/templates",
  "tyk_js_path": "/opt/tyk-gateway/js/tyk.js",
  "middleware_path": "/opt/tyk-gateway/middleware",
  "use_db_app_configs": false,
  "app_path": "/opt/tyk-gateway/apps/",
  "storage": {
    "type": "redis",
    "host": "tyk-redis",
    "port": 6379,
    "username": "",
    "password": "",
    "database": 0,
    "optimisation_max_idle": 2000,
    "optimisation_max_active": 4000
  },
  "enable_analytics": false,
  "analytics_config": {
    "type": "",
    "ignored_ips": []
  },
  "health_check": {
    "enable_health_checks": false,
    "health_check_value_timeouts": 60
  },
  "optimisations_use_async_session_write": false,
  "enable_non_transactional_rate_limiter": true,
  "enable_sentinel_rate_limiter": false,
  "enable_redis_rolling_limiter": false,
  "allow_master_keys": false,
  "policies": {
    "policy_source": "file",
    "policy_record_name": "/opt/tyk-gateway/policies/policies.json"
  },
  "hash_keys": true,
  "close_connections": false,
  "http_server_options": {
    "enable_websockets": true,
    "ssl_insecure_skip_verify": true,
    "use_ssl": true,
    "certificates": [
      {
        "domain_name": "example.com",
        "cert_file": "/opt/tyk-gateway/certs/example.crt",
        "key_file": "/opt/tyk-gateway/certs/example.pem"
      }
    ]
  },
  "allow_insecure_configs": true,
  "coprocess_options": {
    "enable_coprocess": true,
    "coprocess_grpc_server": ""
  },
  "enable_bundle_downloader": true,
  "bundle_base_url": "",
  "global_session_lifetime": 100,
  "force_global_session_lifetime": false,
  "max_idle_connections_per_host": 500,
  "enable_jsvm": true,
  "proxy_ssl_insecure_skip_verify": true,
  "jwt_ssl_insecure_skip_verify": true,
  "ssl_force_common_name_check": false
}

Notice how I have possibly configured all SSL related configuration, I am not sure what I am missing here though, even the “ssl_force_common_name_check” setting is set to false but to no avail.

Thanks!

Hi @Osokarp,
Are you able to share the full API Definition?

This error you’re getting:

tyk-gateway_1 | time=“Nov 11 08:23:08” level=error msg=“No matching policy found!” api_id=protected-api api_name=“Protected API” mw=OpenIDMW org_id=60f08cb4b4c0be0001a87762 origin=172.25.0.1 path=“/protected/requests”

…would be related to scope claim mapping and the client_id configured in the providers section.

I am so far unable to reproduce the TLS handshake error in the first issue though. Can you give some more info about your setup:

I am unsure whether the TLS handshake error is occurring between your client (curl) and the Tyk Gateway, or between the Tyk Gateway and Keycloak Server. The logs seem to contain symptoms of both.

Are you able to reach other API endpoints through the gateway? e.g some other API in tyk proxying to httpbin.org

Are you running the Keycloak server on the same machine? With Docker? With SSL?

One of those ssl_insecure configs should have solved this. Are you restarting Tyk Gateway / Reloading API Definition when you make these changes?

Hi, yes I can, please refer to the following:

I am unsure whether the TLS handshake error is occurring between your client (curl) and the Tyk Gateway, or between the Tyk Gateway and Keycloak Server. The logs seem to contain symptoms of both.

I strongly believe that the issue is due to the first scenario: your client (curl/Tyk Gateway) and the Tyk Gateway. For any upstream traffic forwarded to Keycloak, it’s forwarded using http protocol. Since TLS termination happens at the Tyk gateway, there is no way that it could’ve been caused by any TLS handshake between Tyk gateway and Keycloak.

Since Tyk Gateway needs to retrieve the JWKS from Keycloak, it would need to call itself (we cannot use Docker service name to refer to issuer), so it’s both the client and server in this scenario, fo.

Are you able to reach other API endpoints through the gateway? e.g some other API in tyk proxying to httpbin.org

No, the API definition I created is supposed to forward the request to http://mockbin.org, it’s just another request echoing service just like httpbin.org, and its protected by this Keycloak integration through the JWT verification process handled by Tyk.

For the TLS setup, with any unprotected service Tyk (HTTPS) → any upstream service (http), it will work.

However, I think the reason is because of this particular setup whereby Tyk is calling itself whilst trying to contact the Keycloak server. In the API definition, note how the issuer is the route path that should be handled by Tyk and not the Docker service name of the Keycloak service.

Therefore, Tyk will attempt to call https://<IPADDR>:8080/oidc/realms/master/* (it will hit itself (FAILS HERE), before supposedly routing to the Keycloak server using Docker service name), at this point, since Tyk detect that the target URL is using a self-signed certificate with mismatch in CN, it will fail. And any SSL / TLS configurations I can find on the documentation does not resolve the issue.

Protected API definition:

{
  "name": "Protected API",
  "api_id": "protected-api",
  "org_id": "60f08cb4b4c0be0001a87762",
  "use_openid": true,
  "openid_options": {
    "providers": [
      {
        "issuer": "https://<IPADDR>:8080/oidc/realms/master",
        "client_ids": {
          "Y2xpZW50": "default"
        }
      }
    ],
      "segregate_by_client": true
  },
  "definition": {
    "location": "header",
    "key": "version"
  },
  "auth": {
    "auth_header_name": "Authorization"
  },
  "version_data": {
    "not_versioned": true,
    "versions": {
      "Default": {
        "name": "Default"
      }
    }
  },
  "proxy": {
    "listen_path": "/protected/",
    "target_url": "http://mockbin.org",
    "strip_listen_path": true,
    "transport": {
      "ssl_insecure_skip_verify": true,
      "ssl_force_common_name_check": false
    }
  },
  "enable_context_vars": true
}

Keycloak API definition

{
  "name": "oidc",
  "expiration": "",
  "slug": "",
  "listen_port": 0,
  "protocol": "",
  "enable_proxy_protocol": false,
  "api_id": "oidc",
  "org_id": "1",
  "use_keyless": true,
  "use_oauth2": false,
  "use_openid": false,
  "oauth_meta": {
    "allowed_access_types": null,
    "allowed_authorize_types": null,
    "auth_login_redirect": ""
  },
  "auth": {
    "name": "",
    "use_param": false,
    "param_name": "",
    "use_cookie": false,
    "cookie_name": "",
    "disable_header": false,
    "auth_header_name": "Authorization",
    "use_certificate": false,
    "validate_signature": false,
    "signature": {
      "algorithm": "",
      "header": "",
      "use_param": false,
      "param_name": "",
      "secret": "",
      "allowed_clock_skew": 0,
      "error_code": 0,
      "error_message": ""
    }
  },
  "auth_configs": null,
  "use_basic_auth": false,
  "basic_auth": {
    "disable_caching": false,
    "cache_ttl": 0,
    "extract_from_body": false,
    "body_user_regexp": "",
    "body_password_regexp": ""
  },
  "use_mutual_tls_auth": false,
  "client_certificates": null,
  "upstream_certificates": null,
  "upstream_certificates_disabled": false,
  "pinned_public_keys": null,
  "certificate_pinning_disabled": false,
  "enable_jwt": false,
  "use_standard_auth": false,
  "use_go_plugin_auth": false,
  "enable_coprocess_auth": false,
  "jwt_signing_method": "",
  "jwt_source": "",
  "jwt_identity_base_field": "",
  "jwt_client_base_field": "",
  "jwt_policy_field_name": "",
  "jwt_default_policies": null,
  "jwt_issued_at_validation_skew": 0,
  "jwt_expires_at_validation_skew": 0,
  "jwt_not_before_validation_skew": 0,
  "jwt_skip_kid": false,
  "scopes": {
    "jwt": {
      "scope_claim_name": "",
      "scope_to_policy": null
    },
    "oidc": {
      "scope_claim_name": "",
      "scope_to_policy": null
    }
  },
  "jwt_scope_to_policy_mapping": null,
  "jwt_scope_claim_name": "",
  "notifications": {
    "shared_secret": "",
    "oauth_on_keychange_url": ""
  },
  "enable_signature_checking": false,
  "hmac_allowed_clock_skew": 0,
  "hmac_allowed_algorithms": null,
  "request_signing": {
    "is_enabled": false,
    "secret": "",
    "key_id": "",
    "algorithm": "",
    "header_list": null,
    "certificate_id": "",
    "signature_header": ""
  },
  "base_identity_provided_by": "",
  "definition": {
    "enabled": false,
    "name": "",
    "default": "",
    "location": "header",
    "key": "x-api-version",
    "strip_path": false,
    "strip_versioning_data": false,
    "versions": null
  },
  "version_data": {
    "not_versioned": true,
    "default_version": "",
    "versions": {
      "Default": {
        "name": "Default",
        "expires": "",
        "paths": {
          "ignored": null,
          "white_list": null,
          "black_list": null
        },
        "use_extended_paths": true,
        "extended_paths": {},
        "global_headers": null,
        "global_headers_remove": null,
        "global_response_headers": null,
        "global_response_headers_remove": null,
        "ignore_endpoint_case": false,
        "global_size_limit": 0,
        "override_target": ""
      }
    }
  },
  "uptime_tests": {
    "check_list": null,
    "config": {
      "expire_utime_after": 0,
      "service_discovery": {
        "use_discovery_service": false,
        "query_endpoint": "",
        "use_nested_query": false,
        "parent_data_path": "",
        "data_path": "",
        "port_data_path": "",
        "target_path": "",
        "use_target_list": false,
        "cache_timeout": 0,
        "endpoint_returns_list": false
      },
      "recheck_wait": 0
    }
  },
  "proxy": {
    "preserve_host_header": true,
    "listen_path": "/oidc",
    "target_url": "http://keycloak:8080",
    "disable_strip_slash": false,
    "strip_listen_path": false,
    "enable_load_balancing": false,
    "target_list": null,
    "check_host_against_uptime_tests": false,
    "service_discovery": {
      "use_discovery_service": false,
      "query_endpoint": "",
      "use_nested_query": false,
      "parent_data_path": "",
      "data_path": "",
      "port_data_path": "",
      "target_path": "",
      "use_target_list": false,
      "cache_timeout": 0,
      "endpoint_returns_list": false
    },
    "transport": {
      "ssl_insecure_skip_verify": false,
      "ssl_ciphers": null,
      "ssl_min_version": 0,
      "ssl_max_version": 0,
      "ssl_force_common_name_check": false,
      "proxy_url": ""
    }
  },
  "disable_rate_limit": false,
  "disable_quota": false,
  "custom_middleware": {
    "pre": null,
    "post": null,
    "post_key_auth": null,
    "auth_check": {
      "name": "",
      "path": "",
      "require_session": false,
      "raw_body_only": false
    },
    "response": null,
    "driver": "",
    "id_extractor": {
      "extract_from": "",
      "extract_with": "",
      "extractor_config": null
    }
  },
  "custom_middleware_bundle": "",
  "cache_options": {
    "cache_timeout": 0,
    "enable_cache": false,
    "cache_all_safe_requests": false,
    "cache_response_codes": null,
    "enable_upstream_cache_control": false,
    "cache_control_ttl_header": "",
    "cache_by_headers": null
  },
  "session_lifetime": 0,
  "active": true,
  "internal": false,
  "auth_provider": {
    "name": "",
    "storage_engine": "",
    "meta": null
  },
  "session_provider": {
    "name": "",
    "storage_engine": "",
    "meta": null
  },
  "event_handlers": {
    "events": null
  },
  "enable_batch_request_support": false,
  "enable_ip_whitelisting": false,
  "allowed_ips": null,
  "enable_ip_blacklisting": false,
  "blacklisted_ips": null,
  "dont_set_quota_on_create": false,
  "expire_analytics_after": 0,
  "response_processors": null,
  "CORS": {
    "enable": false,
    "allowed_origins": null,
    "allowed_methods": null,
    "allowed_headers": null,
    "exposed_headers": null,
    "allow_credentials": false,
    "max_age": 0,
    "options_passthrough": false,
    "debug": false
  },
  "domain": "",
  "domain_disabled": false,
  "certificates": null,
  "do_not_track": false,
  "enable_context_vars": false,
  "config_data": null,
  "tag_headers": null,
  "global_rate_limit": {
    "rate": 0,
    "per": 0
  },
  "strip_auth_data": false,
  "enable_detailed_recording": false,
  "graphql": {
    "enabled": false,
    "execution_mode": "",
    "version": "",
    "schema": "",
    "type_field_configurations": null,
    "playground": {
      "enabled": false,
      "path": ""
    },
    "engine": {
      "field_configs": null,
      "data_sources": null
    },
    "proxy": {
      "auth_headers": null
    },
    "subgraph": {
      "sdl": ""
    },
    "supergraph": {
      "subgraphs": null,
      "merged_sdl": "",
      "global_headers": null,
      "disable_query_batching": false
    }
  },
  "analytics_plugin": {
    "enable": false,
    "plugin_path": "",
    "func_name": ""
  },
  "tags_disabled": false,
  "tags": null,
  "is_oas": false
}

Let me know if you need anything else that can help, thank you for your support.

You shouldn’t need an API Definition for Keycloak. Are you not able to access your Keycloak server via http://<IP-Addr>:8080 in your browser?
Have a watch here? API Authentication with OIDC, KeyCloak & Tyk API Gateway - YouTube

Yes you’re right, we could technically not put Keycloak behind Tyk, but preferrably we would like to do so as we want each and every backend service we have to be behind the API Gateway. As we are still in the POC stage in using Tyk, we are only using the open-source version, so we have no access to the dashboard. Also, we are trying possibilities that best suit our use case, and overall it’s just more cleaner in terms of the architecture design.

I think you meant https://<IP-Addr>:8080 right?

If so, using the browser is fine because it offers an option to bypass invalid certificates (even with mismatch in CN!), basically each and every communication works regardless with or without TLS, except when we use offload the responsibility of JWT validation to Tyk. In this case, every API that is protected under this scheme will fail because the when Tyk tries to contact the issuer URL, it actually points to itself which would eventually lead to Keycloak correctly IF we could bypass the invalid CN issue present in the self-signed certificate.

Current flow: Tyk Business Logic → https://<IPADDR>:8080/oidc (Keycloak route, fails here, hasn’t forwarded the request to Keycloak yet) → http://keycloak:8080 (Docker service, not directly accessible to client, but impossible to reach here as it Tyk fails the TLS handshake with itself).

If you see the OP, my first approach doesn’t have this problem, but there’s no way for me to configure Tyk to ignore the mismatch of issuer in the JWT and API definition.

Original flow: Tyk Business Logic → http://keycloak:8080 (Docker service, not directly accessible to client, but will work because Tyk doesn’t need to perform TLS handshake with itself which presents an invalid certificate with mismatch in CN).

Just to re-iterate this will not have an issue in production, only development. We need to configure TLS in Tyk for development because our front-end application uses some browser APIs that require the content to be served via HTTPS, thus, we need it work in locally our tester’s PC.

The whole situation is a tad complicated, let me know if you need further clarifications.

i.e The Tyk gateway logic of validating JWT / fetching JWKS from the issuer fails IF:

  1. The request goes through itself before being routed to the OIDC service
  2. Tyk’s TLS certificate’s CN doesn’t match the queried URL (e.g certificate CN = localhost, URL queried = <IPADDR>)

Thanks this is clear. I’ve done some investigation, and here’s what I’ve found.

jwt_insecure_skip_verify is relevant to a validation flow different from OIDC which this API implements. Particularly, it is useful for JWT-secured APIs, that implement Dynamic public key rotation using public JWKs URL.
See its implementation in the codebase here and here.

proxy.transport.ssl_insecure_skip_verify in the API definition is useful against the target upstream, which, is mockbin.org in this case.

For the openid flow, there isn’t implementation to support skipping TLS validation, neither is there configuration for Tyk to ignore issuer mismatch… If you like, you can take a look at the source here. [1] [2]

With these constraints I think a way forward, though not without some effort, will be to use a Pre Request Plugin to modify the request. It would modify the issuer in the jwt payload, before the authentication step, to match the issuer configured in API definition (as in Approach 1). This solves the issuer mismatch failure and eliminates the need to skip TLS verification (in Approach 2) as Tyk would be calling the Keycloak server directly.

Manually, I’m able modify the jwt payload with help from jwt.io and https://www.base64decode.org.
Here’s a custom plugin that does some jwt manipulation.

Hi,

Thank you so much for the recommended solution, I will try it out.

Hi Ubong,

I have tried using the pre-request plugin method you’ve suggested, I have managed to modify the JWT token using a Javascript middleware, however rightfully so, I think Tyk rejects the modified JWT token because the token has become invalid due to the fact that the contents being changed and thus, does not match the original signature in the token.

I have tried enable_signature_checking: false, and auth.validate_signature: false in the API definition, but to no avail… Any advice?

Thanks again.

Hmmm sorry about this.

Can you share the gateway logs associated with this rejection?
And maybe the plugin if you wouldn’t mind.

Hi @Ubong,

Sure here are the gateway logs:

tyk-gateway_1     | time="Nov 29 08:25:23" level=warning msg="JWT Invalid" api_id=protected-api api_name="Protected API" error="Validation error. Jwt token validation failed." mw=OpenIDMW org_id=60f08cb4b4c0be0001a87762 origin=172.22.0.1 path="/protected/requests"
tyk-gateway_1     | time="Nov 29 08:25:23" level=warning msg="Attempted access with invalid 
key." api_id=protected-api api_name="Protected API" key="****JWT]" mw=OpenIDMW org_id=60f08cb4b4c0be0001a87762 origin=172.22.0.1 path="/protected/requests"

And this is the pre-request plugin I wrote:

var modifyJwtIssuer = new TykJS.TykMiddleware.NewMiddleware({});

modifyJwtIssuer.NewProcessRequest(function(request, session, config) {
  var authorizationHeaderArray = request.Headers.Authorization;
  if (Array.isArray(authorizationHeaderArray)) {
    var authorizationHeader = authorizationHeaderArray[0];
    if (typeof authorizationHeader === "string") {
      var jwtToken = authorizationHeader.substring(7);
      var splitJwtToken = jwtToken.split(".");
      var claims = JSON.parse(b64dec(splitJwtToken[1]));
      claims.iss = "http://keycloak:8080/oidc/realms/master";
      var base64Claims = b64enc(JSON.stringify(claims));
      splitJwtToken[1] = base64Claims;
      request.SetHeaders['Authorization'] = "Bearer " + splitJwtToken.join(".");
    }
  }

	return modifyJwtIssuer.ReturnData(request, {});
});

I verified that I was at least changing the Authorization part properly, because without the last line in the if statement, there wouldn’t be this error

Hi @Osokarp,

I think we’ve got it now.

Did some testing and I see that otto’s JSON.stringify(), which Tyk uses, reorders the object elements in alphabetical order, and Keycloak doesn’t like this so it rejects the token with the JWT invalid message seen in the logs.
View discussion/issue on JSON.stringify() here.
I’ve modified to manually but reliably stringify the object in the same order as issued from Keycloak

Also, the re-encoded claim string could contain “=” characters at the end, which could invalidate the token. (I’ve found specifically when the issuer protocol is http vs https).
I’ve added comments to help see and correct that depending on the use case.

var modifyJwtIssuer = new TykJS.TykMiddleware.NewMiddleware({});

modifyJwtIssuer.NewProcessRequest(function(request, session, config) {
  var authorizationHeaderArray = request.Headers.Authorization;
  if (Array.isArray(authorizationHeaderArray)) {
    var authorizationHeader = authorizationHeaderArray[0];
    if (typeof authorizationHeader === "string") {
      var jwtToken = authorizationHeader.substring(7);
      var splitJwtToken = jwtToken.split(".");
      // View original claims
      // log("Original Claims: " + b64dec(splitJwtToken[1]));
      var claims = JSON.parse(b64dec(splitJwtToken[1]));
      claims.iss = "http://keycloak:8080/oidc/realms/master";

      claimsOrder = ["exp", "iat", "auth_time", "jti", "iss", "aud", "sub", "typ", "azp", "session_state", "at_hash", "acr", "sid", "email_verified", "name", "preferred_username", "given_name", "family_name", "email"];
      claimsCount = claimsOrder.length;
      var modifiedClaims = "{";
      for (var i = 0; i < claimsCount; i++) {        
        modifiedClaims += "\"" + claimsOrder[i] + "\"" + ":";

        // log(claimsOrder[i] + " claim is " + typeof claims[claimsOrder[i]] + " variable");
        modifiedClaims += typeof claims[claimsOrder[i]] === "string" ? "\"" + claims[claimsOrder[i]] + "\"" : claims[claimsOrder[i]];

        if (i != claimsCount - 1) {
          modifiedClaims += ",";
        }
      }
      modifiedClaims += "}";
      
      // View modified claims
      // log("Modified Claims: " + modifiedClaims);

      // when re-encoded, some '=' characters might be introduced at end of the string
      // view with 
      // log("This is Modified Claims base64 encoded: " + b64enc(modifiedClaims));

      // strip off last characters using slice.
      // log("This is base64 encoded minus last 2: " + (b64enc(modifiedClaims)).slice(0, -2));
      
      var base64Claims = b64enc(modifiedClaims)
      splitJwtToken[1] = base64Claims;
      request.SetHeaders['Authorization'] = "Bearer " + splitJwtToken.join(".");
    }
  }

	return modifyJwtIssuer.ReturnData(request, {});
});

My testing so far has been successful. Let me know how you get on

Hi @Ubong,

Thank you for the modified script, I have tweaked it slightly to match my use case as I have some additional claims that doesn’t exist in the provided script. I also made sure that the claims were in the exactly same order-wise it originally was in. So the only thing that changed here is the iss claim.

However I still receive this error log:

tyk-gateway-docker-tyk-gateway-1     | time="Dec 01 04:10:44" level=warning msg="JWT Invalid" api_id=protected-api api_name="Protected API" error="Validation error. Jwt token validation failed." mw=OpenIDMW org_id=60f08cb4b4c0be0001a87762 origin=192.168.112.1 path="/protected/requests"
tyk-gateway-docker-tyk-gateway-1     | time="Dec 01 04:10:44" level=warning msg="Attempted access with invalid key." api_id=protected-api api_name="Protected API" key="****JWT]" mw=OpenIDMW org_id=60f08cb4b4c0be0001a87762 origin=192.168.112.1 path="/protected/requests"

Correct me if I’m wrong, but I think we could only have a partially correct hypothesis of what is going on here, while the property ordering could indeed be a problem, there’s another contributing factor which is what I have mentioned before, about how Tyk validates data integrity.

Consider the following:
Original JWT: <HEADER>.<BODY A>.<SIGNATURE>

After modifying the iss claim, the JWT will become:
Modified JWT <HEADER>.<BODY B>.<SIGNATURE>

Since we have modified the iss claim, the eventually base64 encoded BODY will have a different value compared to the original JWT. To verify data integrity, which is an integral part of any JWT validation process, I believe Tyk should be using some cryptographic protocol listed in the header with both the BODY and HEADER to generate a signature that should equals the SIGNATURE part of the JWT.

However, this is not possible as we have already modified the BODY. So I don’t think this is the way we should be proceeding, UNLESS there’s a way to ignore the validity checks of the JWT signature (which of course should not be done in production), which was why I was trying out this property at two different parts of the API definition:
enable_signature_checking: false and auth.validate_signature: false, but alas, it didn’t work, I could not find the documentation of what this property does, but it was the only thing that rang a bell to me. Perhaps I misunderstood the purpose of this property.

Appreciate your feedback, and looking forward to hear from you.

EDIT: I tried auth_configs.oidc.validate_signature: false as well

Hi @Ubong,

Any updates?

Thank you.

Hi @Osokarp,

Sorry for the long silence. I wonder if you’ve made headway?

I understand your thought process and I think it makes sense.

In my tests, I can access API after modifying the “iss” claim. Here’s a summary of my flow:

FLOW 1
→ Get Keycloak-issued JWT
→ Manually modify (using jwt.io and base64decode) the iss claim to an issuer does not exist in my Tyk API definition
→ Call the API with the token

Tyk spits out error in Approach 1 at the beginning of the thread
error=“Validation error. Validation error. No provider was registered with issuer: https//xxx.xxx.xxx.xxx/keycloak/realms/master”

This is expected

FLOW 2
→ Get Keycloak-issued JWT
→ Manually modify the iss claim to an issuer does not exist in my Tyk API definition
→ Add the javascript middleware (the one that re-orders the claims) that corrects the “iss” claim
→ Call the API with the token

“No provider” error is gone, but this is seen:
level=warning msg="JWT Invalid" error="Validation error. Jwt token validation failed." mw=OpenIDMW

FLOW 3
→ Get Keycloak-issued JWT
→ Manually modify the iss claim to an issuer does not exist in my Tyk API definition
→ Add javascript middleware (the one that retains the order of the claims) that corrects the “iss” claim
→ Call the API with the token

Tyk grants access to the API

I’m not confident that the JWT validation is happening at Tyk since Tyk didn’t issue it in the first place. I believe the error comes from this point in the code, and unlike other errors, there’s no additional message about the reason for the error (e.g “No issuer or audiences found!”, or “No matching policy found!” )

I think the validation is happening at Keycloak, and I think some configuration at Keycloak might be the way to fix it. Perhaps something to do with the additional claims you mentioned? Maybe the scopes are not assigned to the client? Here’s some discussion I came across.

You can share your JWT, perhaps as a direct message, maybe there’s something I could find.

Btw, I get the same “JWT token validation failed” error if my token is expired :thinking:. The logs could have been more specific.

Sorry for the long silence again. Looking forward to hearing from you.

1 Like

Hi @Ubong,

Thank you for your reply, really appreciate your support so far.

It’s been a while since I visited this issue, I will try again and let you know my findings. As for the conclusion that you’ve arrived at, whereby you think that the validation is happening at Keycloak, I don’t think that’s true. Responsibility of validating the JWT token lies in the API server, or in API Gateway (since we’re talking about Tyk), Keycloak being the OIDC server only provides a list of public keys (via the JWKS endpoint in any OAuth2.0 compliant server) for the API server / gateway used to validate the token.

Technically it is impossible to for the signature to match after changing any value in the payload / header of the JWT token, but I’m not sure why it will pass in Tyk’s case. Regardless, I will try Flow 2 and Flow 3 as described and see if it solves my issue.

Thank you!

1 Like