Keycloak integration example docs

Hi, i have setup keycloak and created the realm and client for my simple app.
The service behind tyk-gateway has two endpoints, and i configure one of them to use open id, so the configuration is like:

{
    "name": "Stock Price API Protected",
    "api_id": "stocks-protected",
    "org_id": "default",
    "definition": {
        "location": "header",
        "key": "version"
    },
    "use_openid": true,
    "openid_options": {
        "providers": [
            {
                "issuer": "http://keycloak:8080/auth/realms/tyk-realm",
                "client_ids": {
                    "dHlrLWdhdGV3YXk=": "default"
                }
            }
        ]
    },
    "version_data": {
        "not_versioned": true,
        "versions": {
            "Default": {
                "name": "Default"
            }
        }
    },
    "custom_middleware": {},
    "proxy": {
        "listen_path": "/securestocks",
        "target_url": "http://app-web:9091/securestocks",
        "strip_listen_path": true
    }
}

I feel like this may not be enough, but can’t figure out what else i need, so maybe you can point it to me.

So I get the access token for that client from calling this keycloak endpoint, http://localhost:8080/realms/tyk-realm/protocol/openid-connect/token, which then i use it as a bearer token to access that protected service, but i got back error “Key not authorised”

I am still new on this, so apologize if this is a silly question, but I have no idea what i missed here. Btw, I saw this script JSON Web Tokens (JWT), do i need this similar script for my case?
I hope there are more examples on keycloak integration, because i feel like the current docs are not sufficient, and GitHub - TykTechnologies/tyk-demo: Tyk Docker Demo with full Pro Tyk Installation and more! is not very clear to me. Thanks!

anyone here who can give me pointers?

Hello -

You should try to use regular JWT auth.

In your API Definition, set the signing method, identity source: sub and the public key as the JWKS url.

You can find the jwks url from your OpenID well known endpoint - it’s usually https://KEYCLOAK_HOST/realms/REALM/protocol/openid-connect/certs

You then need to create a security policy, which grants access to that API.

And the default policy on the API definition to be that policy you just created.

Your API Definition snippet will look somehting like the following:

  server:
    authentication:
      enabled: true
      stripAuthorizationData: true
      securitySchemes:
        jwtAuth:
          header:
            enabled: true
            name: Authorization
          identityBaseField: sub
          policyFieldName: pol
          enabled: true
          defaultPolicies: []
          signingMethod: ecdsa
          source: >-
            aHR0cDovL2xvY2FsaG9zdDo4MDgxL3JlYWxtcy9mYXBpLWRlbW8vcHJvdG9jb2wvb3BlbmlkLWNvbm5lY3QvY2VydHM= <--------- Base 64 encoded JWKS URL
    listenPath:
      strip: true
      value: /foo

You can then auth via keycloak, using client credentials or authorization code flow - and when you obtain your access token, you should be able to use that to access your APIs via the gateway.

You can find full documentation here JSON Web Tokens (JWT)

Thanks @AhmetSoormally for the pointers. I followed your suggestion and also looking at that docs (apparently i used tyk classic definition, and now switching to using OAS), but the service/endpoint is not seen from tyk gateway, can you see what’s wrong from my API definition below?

{
    "components": {
        "securitySchemes": {
            "jwtAuth": {
                "type": "http",
                "scheme": "bearer",
                "bearerFormat": "JWT",
                "description": "JWT authentication for secure access to the stocks API"
            }
        }
    },
    "paths": {},
    "servers": [
        {
            "url": "http://localhost:8081/securestocks",
            "description": "Local development server for the secure stocks API"
        }
    ],
    "x-tyk-api-gateway": {
        "info": {
            "id": "stocks-secure",
            "orgId": "default",
            "name": "Secure Stocks API",
            "state": {
                "active": true,
                "internal": false
            }
        },
        "server": {
            "authentication": {
                "enabled": true,
                "stripAuthorizationData": true,
                "securitySchemes": {
                    "jwtAuth": {
                        "header": {
                            "enabled": true,
                            "name": "Authorization"
                        },
                        "identityBaseField": "sub",
                        "policyFieldName": "pol",
                        "enabled": true,
                        "defaultPolicies": [],
                        "signingMethod": "ecdsa",
                        "source": "aHR0cDovL2xvY2FsaG9zdDo4MDgwL3JlYWxtcy90eWstcmVhbG0vcHJvdG9jb2wvb3BlbmlkLWNvbm5lY3QvY2VydHM="
                    }
                }
            },
            "listenPath": {
                "strip": true,
                "value": "/securestocks/"
            }
        },
        "upstream": {
            "url": "http://app-web:9091/securestocks"
        }
    }
}

@AhmetSoormally sorry to bother you, i wonder if you can give me another pointer, or perhaps the issue is pretty obvious. I apologize for my ignorance on this stuff. Thanks and have a great weekend!

Hi @z11373

Thank you for your patience.

You may need to use host.docker.internal (or a similar alternative) for the jwks_url, as localhost will not work from within the Gateway container if Keycloak is running externally.

It appears that a default policy has not been configured—this is required for proper access control. Please ensure a default policy is set.

The signing method is likely rsa, but any misconfiguration should become apparent once the JWT validation runs.

If the issue persists after making these changes, set the Gateway log level to debug, make a request with a token, and share the resulting logs so we can assist further.

thanks @Ubong for the reply and also pointing me some of those things.

for configuring the policy, i am not using tyk dashboard, can I do it via configuration file?

actually, i have this section in my tyk.conf:

  "policies": {
    "policy_source": "file",
    "policy_path": "/opt/tyk-gateway/policies"
  }

and in my docker compose file, i have the volumes mapping:

    volumes:
      - ${PWD}/tyk.conf:/opt/tyk-gateway/tyk.conf
      - ${PWD}/apps:/opt/tyk-gateway/apps
      - ${PWD}/middleware:/opt/tyk-gateway/middleware
      - ${PWD}/certs:/opt/tyk-gateway/certs
      - ${PWD}/policies:/opt/tyk-gateway/policies

will this work? if yes, then the file policy.json that i have may have incorrect configuration.

Hi @z11373,

Yes, this is supposed to work. Please see a sample in the docs.
Config ref.

You may share your policy.json

Please review similar posts

Thanks @Ubong!
I just looked at the gateway logs, and i am not sure if the secured API is loaded (which may be the reason why I can’t access it via tyk endpoint), because though it said loading stock-price-secure.json but i don’t see the log has “API Loaded” for that endpoint. Thanks for the doc pointers, I will look at them, and look into more on my API spec.

time="Jun 06 13:15:36" level=info msg="Tyk API Gateway v4.0.0" prefix=main
time="Jun 06 13:15:36" level=warning msg="Insecure configuration allowed" config.allow_insecure_configs=true prefix=checkup
time="Jun 06 13:15:36" level=debug msg="Notifier will not work in hybrid mode" prefix=main
time="Jun 06 13:15:36" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Jun 06 13:15:36" level=info msg="Starting Poller" prefix=host-check-mgr
time="Jun 06 13:15:36" level=debug msg="---> Initialising checker" prefix=host-check-mgr
time="Jun 06 13:15:36" level=debug msg="[HOST CHECKER] Config:TriggerLimit: 3"
time="Jun 06 13:15:36" level=debug msg="[HOST CHECKER] Config:Timeout: ~10"
time="Jun 06 13:15:36" level=debug msg="[HOST CHECKER] Config:WorkerPool: 4"
time="Jun 06 13:15:36" level=debug msg="[HOST CHECKER] Init complete"
time="Jun 06 13:15:36" level=debug msg="---> Starting checker" prefix=host-check-mgr
time="Jun 06 13:15:36" level=debug msg="[HOST CHECKER] Starting..."
time="Jun 06 13:15:36" level=debug msg="[HOST CHECKER] Check loop started..."
time="Jun 06 13:15:36" level=debug msg="[HOST CHECKER] Host reporter started..."
time="Jun 06 13:15:36" level=debug msg="---> Checker started." prefix=host-check-mgr
time="Jun 06 13:15:36" level=info msg="PIDFile location set to: /var/run/tyk/tyk-gateway.pid" prefix=main
time="Jun 06 13:15:36" level=warning msg="The control_api_port should be changed for production" prefix=main
time="Jun 06 13:15:36" level=debug msg="Initialising default org store" prefix=main
time="Jun 06 13:15:36" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
time="Jun 06 13:15:36" level=debug msg="Connecting to redis cluster"
time="Jun 06 13:15:36" level=debug msg="Creating new Redis connection pool"
time="Jun 06 13:15:36" level=info msg="--> [REDIS] Creating single-node client"
time="Jun 06 13:15:36" level=debug msg="Loaded API Endpoints" prefix=main
time="Jun 06 13:15:36" level=info msg="--> Standard listener (http)" port=":8081" prefix=main
time="Jun 06 13:15:36" level=warning msg="Starting HTTP server on:[::]:8081" prefix=main
time="Jun 06 13:15:36" level=info msg="Initialising distributed rate limiter" prefix=main
time="Jun 06 13:15:36" level=debug msg="DRL: Setting node ID: solo-4a087ed2-9a51-4370-89f6-e2c7964a5d62|867ba692b936"
time="Jun 06 13:15:36" level=info msg="Tyk Gateway started (v4.0.0)" prefix=main
time="Jun 06 13:15:36" level=info msg="--> Listening on address: (open interface)" prefix=main
time="Jun 06 13:15:36" level=info msg="--> Listening on port: 8081" prefix=main
time="Jun 06 13:15:36" level=info msg="--> PID: 1" prefix=main
time="Jun 06 13:15:36" level=info msg="Starting gateway rate limiter notifications..."
time="Jun 06 13:15:36" level=debug msg="Connecting to redis cluster"
time="Jun 06 13:15:36" level=debug msg="Creating new Redis connection pool"
time="Jun 06 13:15:36" level=info msg="--> [REDIS] Creating single-node client"
time="Jun 06 13:15:36" level=debug msg="Default JSVM timeout used: 5s" prefix=jsvm
time="Jun 06 13:15:36" level=info msg="Loading policies" prefix=main
time="Jun 06 13:15:36" level=debug msg="No policy record name defined, skipping..." prefix=main
time="Jun 06 13:15:36" level=info msg="Loading API Specification from /opt/tyk-gateway/apps/keyless-plugin-api.json"
time="Jun 06 13:15:36" level=debug msg="Loading custom PRE-PROCESSOR middleware: testJSVMData" prefix=main
time="Jun 06 13:15:36" level=debug msg="Connecting to redis cluster"
time="Jun 06 13:15:36" level=debug msg="Creating new Redis connection pool"
time="Jun 06 13:15:36" level=info msg="--> [REDIS] Creating single-node client"
time="Jun 06 13:15:36" level=debug msg="Default JSVM timeout used: 5s" prefix=jsvm
time="Jun 06 13:15:36" level=warning msg="Legacy path detected! Upgrade to extended."
time="Jun 06 13:15:36" level=info msg="Loading API Specification from /opt/tyk-gateway/apps/stock-price-secure.json"
time="Jun 06 13:15:36" level=info msg="Loading API Specification from /opt/tyk-gateway/apps/stock-price.json"
time="Jun 06 13:15:36" level=warning msg="Legacy path detected! Upgrade to extended."
time="Jun 06 13:15:36" level=info msg="Detected 3 APIs" prefix=main
time="Jun 06 13:15:36" level=info msg="Loading API configurations." prefix=main
time="Jun 06 13:15:36" level=info msg="Tracking hostname" api_name="Tyk Test Keyless API" domain="(no host)" prefix=main
time="Jun 06 13:15:36" level=info msg="Tracking hostname" api_name="Stock Price API" domain="(no host)" prefix=main
time="Jun 06 13:15:36" level=info msg="Tracking hostname" api_name= domain="(no host)" prefix=main
time="Jun 06 13:15:36" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
time="Jun 06 13:15:36" level=debug msg="Loaded API Endpoints" prefix=main
time="Jun 06 13:15:36" level=info msg="API bind on custom port:0" prefix=main
time="Jun 06 13:15:36" level=debug msg="Initializing API" api_id=keyless api_name="Tyk Test Keyless API" org_id=default
time="Jun 06 13:15:36" level=debug msg="Loading custom PRE-PROCESSOR middleware: testJSVMData" prefix=main
time="Jun 06 13:15:36" level=info msg="Loading JS File: ./middleware/injectHeader.js" prefix=jsvm
time="Jun 06 13:15:36" level=info msg="Checking security policy: Open" api_id=keyless api_name="Tyk Test Keyless API" org_id=default
time="Jun 06 13:15:36" level=debug msg=Init api_id=keyless api_name="Tyk Test Keyless API" mw=DynamicMiddleware org_id=default
time="Jun 06 13:15:36" level=debug msg=Init api_id=keyless api_name="Tyk Test Keyless API" mw=VersionCheck org_id=default
time="Jun 06 13:15:36" level=debug msg=Init api_id=keyless api_name="Tyk Test Keyless API" mw=RateCheckMW org_id=default
time="Jun 06 13:15:36" level=debug msg="Setting Listen Path: /keyless-test/" api_id=keyless api_name="Tyk Test Keyless API" org_id=default
time="Jun 06 13:15:36" level=info msg="API Loaded" api_id=keyless api_name="Tyk Test Keyless API" org_id=default prefix=gateway server_name=-- user_id=-- user_ip=--
time="Jun 06 13:15:36" level=info msg="API bind on custom port:0" prefix=main
time="Jun 06 13:15:36" level=debug msg="Initializing API" api_id=stocks api_name="Stock Price API" org_id=default
time="Jun 06 13:15:36" level=info msg="Checking security policy: Open" api_id=stocks api_name="Stock Price API" org_id=default
time="Jun 06 13:15:36" level=debug msg=Init api_id=stocks api_name="Stock Price API" mw=VersionCheck org_id=default
time="Jun 06 13:15:36" level=debug msg=Init api_id=stocks api_name="Stock Price API" mw=RateCheckMW org_id=default
time="Jun 06 13:15:36" level=debug msg="Setting Listen Path: /stocks" api_id=stocks api_name="Stock Price API" org_id=default
time="Jun 06 13:15:36" level=info msg="API Loaded" api_id=stocks api_name="Stock Price API" org_id=default prefix=gateway server_name=-- user_id=-- user_ip=--
time="Jun 06 13:15:36" level=info msg="API bind on custom port:0" prefix=main
time="Jun 06 13:15:36" level=error msg="Listen path is empty" api_id= api_name= org_id=
time="Jun 06 13:15:36" level=warning msg="Spec not valid, skipped!" api_id= api_name= org_id=
time="Jun 06 13:15:36" level=debug msg="Checker host list" prefix=main
time="Jun 06 13:15:36" level=info msg="Loading uptime tests..." prefix=host-check-mgr
time="Jun 06 13:15:36" level=debug msg="--- Setting tracking list up" prefix=host-check-mgr
time="Jun 06 13:15:36" level=debug msg="Reset initiated" prefix=host-check-mgr
time="Jun 06 13:15:36" level=debug msg="[HOST CHECKER] Checker reset queued!"
time="Jun 06 13:15:36" level=debug msg="Checker host Done" prefix=main
time="Jun 06 13:15:36" level=info msg="Initialised API Definitions" prefix=main
time="Jun 06 13:15:36" level=info msg="API reload complete" prefix=main
time="Jun 06 13:15:46" level=debug msg="Error trying to get value:redis: nil"
time="Jun 06 13:15:46" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Jun 06 13:15:46" level=warning msg="Reconnecting storage: Redis is either down or was not configured" prefix=pub-sub
time="Jun 06 13:15:47" level=debug msg="[HOST CHECKER] Host list reset"
time="Jun 06 13:15:56" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr

Hi @z11373

Thank you for sharing the logs.

Gateway v4.0.0 does not have full support for OAS definitions, which may be why it’s not loading as expected.
Please consider using an LTS version (v5.8 currently), or continuing with a Classic definition as you initially started.

Ah, i thought by using tag latest, it will get latest version, but it’s actually v4.0.0

so i change my docker compose file to use tag v5.7, which should support OAS, but somehow it still doesn’t work. For using OAS, i started without auth at all, since i know that the classic spec works, but somehow it still doesn’t work :frowning:

Here is my simple OAS spec:

{
    "components": {},
    "info": {
        "title": "Stocks API",
        "description": "API for stock prices",
        "version": "1.0.0"
    },
    "openapi": "3.0.3",
    "paths": {},
    "servers": [
        {
            "url": "http://localhost:8081"
        }
    ],
    "x-tyk-api-gateway": {
        "info": {
            "id": "stocks",
            "orgId": "default",
            "name": "stocks",
            "state": {
                "active": true
            }
        },
        "server": {
            "listenPath": {
                "strip": false,
                "value": "/stocks/"
            }
        },
        "upstream": {
            "url": "http://app-web:9091"
        }
    }
}

and looking at the gateway logs, i see it said the spec is invalid and the listen path is empty… hmm, do you know what I missed here? Thanks again!

time="Jun 11 01:46:37" level=debug msg="Set log level to \"debug\"" prefix=main
time="Jun 11 01:46:37" level=debug msg="Using /opt/tyk-gateway/tyk.conf for configuration" prefix=main
time="Jun 11 01:46:37" level=info msg="Tyk API Gateway 5.7.3" prefix=main
time="Jun 11 01:46:37" level=warning msg="Insecure configuration allowed" config.allow_insecure_configs=true prefix=checkup
time="Jun 11 01:46:37" level=error msg="storage: Redis is either down or was not configured"
time="Jun 11 01:46:37" level=error msg="Could not set version in versionStore" error="storage: Redis is either down or was not configured" prefix=main
time="Jun 11 01:46:37" level=error msg="storage: Redis is either down or was not configured"
time="Jun 11 01:46:37" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Jun 11 01:46:37" level=error msg="storage: Redis is either down or was not configured"
time="Jun 11 01:46:37" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Jun 11 01:46:37" level=info msg="Starting Poller" prefix=host-check-mgr
time="Jun 11 01:46:37" level=debug msg="---> Initialising checker" prefix=host-check-mgr
time="Jun 11 01:46:37" level=debug msg="[HOST CHECKER] Config:TriggerLimit: 3"
time="Jun 11 01:46:37" level=debug msg="[HOST CHECKER] Config:Timeout: ~10"
time="Jun 11 01:46:37" level=debug msg="[HOST CHECKER] Config:WorkerPool: 4"
time="Jun 11 01:46:37" level=debug msg="[HOST CHECKER] Init complete"
time="Jun 11 01:46:37" level=debug msg="---> Starting checker" prefix=host-check-mgr
time="Jun 11 01:46:37" level=debug msg="[HOST CHECKER] Starting..."
time="Jun 11 01:46:37" level=debug msg="[HOST CHECKER] Check loop started..."
time="Jun 11 01:46:37" level=debug msg="[HOST CHECKER] Host reporter started..."
time="Jun 11 01:46:37" level=debug msg="---> Checker started." prefix=host-check-mgr
time="Jun 11 01:46:37" level=debug msg="Notifier will not work in hybrid mode" prefix=main
time="Jun 11 01:46:37" level=info msg="PIDFile location set to: /var/run/tyk/tyk-gateway.pid" prefix=main
time="Jun 11 01:46:37" level=warning msg="The control_api_port should be changed for production" prefix=main
time="Jun 11 01:46:37" level=debug msg="Initialising default org store" prefix=main
time="Jun 11 01:46:37" level=error msg="storage: Redis is either down or was not configured"
time="Jun 11 01:46:37" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
time="Jun 11 01:46:37" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
time="Jun 11 01:46:37" level=info msg="job run successful" name=purge-oauth-tokens prefix=scheduler
time="Jun 11 01:46:37" level=debug msg="Creating new default Storage connection"
time="Jun 11 01:46:37" level=debug msg="Creating new cache Storage connection"
time="Jun 11 01:46:37" level=debug msg="Creating new analytics Storage connection"
time="Jun 11 01:46:37" level=debug msg="Loaded API Endpoints" prefix=main
time="Jun 11 01:46:37" level=info msg="--> Standard listener (http)" port=":8081" prefix=main
time="Jun 11 01:46:37" level=warning msg="Starting HTTP server on:[::]:8081" prefix=main
time="Jun 11 01:46:37" level=info msg="[RATELIMIT] DRL with Redis Rate Limiter enabled (using pipeline)"
time="Jun 11 01:46:37" level=info msg="Initialising distributed rate limiter" prefix=main
time="Jun 11 01:46:37" level=debug msg="DRL: Setting node ID: solo-437d1201-00ee-4ea0-a6c2-ca6b8f56a59b|4b2e9dddb55c"
time="Jun 11 01:46:37" level=info msg="Tyk Gateway started (5.7.3)" prefix=main
time="Jun 11 01:46:37" level=info msg="--> Listening on address: (open interface)" prefix=main
time="Jun 11 01:46:37" level=info msg="--> Listening on port: 8081" prefix=main
time="Jun 11 01:46:37" level=info msg="--> PID: 1" prefix=main
time="Jun 11 01:46:37" level=info msg="Starting gateway rate limiter notifications..."
time="Jun 11 01:46:37" level=error msg="storage: Redis is either down or was not configured"
time="Jun 11 01:46:37" level=debug msg="Default JSVM timeout used: 5s" prefix=jsvm
time="Jun 11 01:46:37" level=info msg="Loading policies" prefix=main
time="Jun 11 01:46:37" level=info msg="Loading policy from dir /opt/tyk-gateway/policies/policy.json"
time="Jun 11 01:46:37" level=info msg="Policies found (1 total):" prefix=main
time="Jun 11 01:46:37" level=debug msg=" - " prefix=main
time="Jun 11 01:46:37" level=info msg="Loading API Specification from /opt/tyk-gateway/apps/stock-price.json"
time="Jun 11 01:46:37" level=info msg="Detected 1 APIs" prefix=main
time="Jun 11 01:46:37" level=info msg="Loading API configurations." prefix=main
time="Jun 11 01:46:37" level=info msg="Tracking hostname" api_name= domain="(no host)" prefix=main
time="Jun 11 01:46:37" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
time="Jun 11 01:46:37" level=debug msg="Loaded API Endpoints" prefix=main
time="Jun 11 01:46:37" level=info msg="API bind on custom port:0" prefix=main
time="Jun 11 01:46:37" level=error msg="Listen path is empty" api_id= api_name= org_id=
time="Jun 11 01:46:37" level=warning msg="Spec not valid, skipped!" api_id= api_name= org_id=
time="Jun 11 01:46:37" level=debug msg="Checker host list" prefix=main
time="Jun 11 01:46:37" level=info msg="Loading uptime tests..." prefix=host-check-mgr
time="Jun 11 01:46:37" level=debug msg="--- Setting tracking list up" prefix=host-check-mgr
time="Jun 11 01:46:37" level=debug msg="Reset initiated" prefix=host-check-mgr
time="Jun 11 01:46:37" level=debug msg="[HOST CHECKER] Checker reset queued!"
time="Jun 11 01:46:37" level=debug msg="Checker host Done" prefix=main
time="Jun 11 01:46:37" level=info msg="Initialised API Definitions" prefix=main
time="Jun 11 01:46:37" level=warning msg="All APIs are protected with mTLS, except for the control API. We recommend configuring the control API port or control hostname to ensure consistent security measures" prefix=main
time="Jun 11 01:46:37" level=info msg="API reload complete" prefix=main
time="Jun 11 01:46:44" level=debug msg="[HOST CHECKER] Host list reset"
time="Jun 11 01:46:47" level=debug msg="Error trying to get value:key not found"
time="Jun 11 01:46:47" level=debug msg="No Primary instance found, assuming control" prefix=host-check-mgr
time="Jun 11 01:46:57" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr
time="Jun 11 01:47:07" level=debug msg="Primary instance set, I am master" prefix=host-check-mgr

Hi @z11373,

Thank you for your patience.

It appears that loading OAS APIs directly from a directory is not yet supported—this functionality is still in the backlog (ref: TT-6635).
Please use the /tyk/apis/oas endpoint to create the API.

You can also refer to the posts below for additional context.