Apply policy to pre-authenticated user

Hi,

I wonder if there is an existing solution to my problem. And if not, if you would be interested in a pull request for a new middleware supporting this use case:

As far as I know, there are currently only two middlewares that support on-the-fly creation of sessions based on pre-defined policies: The JWT-middleware and the openid-middleware. Both of these allow the usage of an existing identity provider, so that they automatically create a new session based on the “sub”-claim if the provided JWT is valid.

In my case, the JWT is already verified by an Apache server before the request reaches Tyk-gateway, so there is no need for Tyk to verify the JWT again. The apache server inserts the claims of the JWT into new http headers, such as the “sub”-claim (subject). Now I want the Tyk-gateway to create a session on-the-fly (if not already existing) by just using a subject extracted from a custom header inserted by Apache. There may be another header carrying the policy-id to apply.

The middleware “mw_jwt.go” uses the function “generateSessionFromPolicy(…)” to create a session on the fly for centralized JWTs. Maybe I need to create a similar middleware that reads the data from http headers instead from a JWT. What do you think?

If yu just want to respect an arbitrary token (forget it’s a JWT), you could use the Gateway API to PUT the WHOLE token (the bit after "Bearer: " as a token with a session object and set the API Definition to use “Auth Token”, that will work, but you would need to integrate.

Alternatively, create a custom Auth middleware in JS or Python that does the above.

Thanks for your quick response!

Your first approach won’t work, as the JWT will be different every few minutes (because of the “expires”-field). This is not acceptable, as I want to apply rate-limiting to every user.

I don’t yet know how to write custom middleware in JS, but I will have a look. Thanks!

Well, maybe it would be easier to just let Tyk validate the JWT again. This isn’t necessary, but the overhead would probably be the same as using a custom javascript middleware :slight_smile:

For reference: My use-case seems to be very similar to the one of this discussion:

The only difference is, that I want to accept any Authorization-header value as valid, instead of requesting an external auth server.

I think all of this is unnecessarily complicated and the documentation is lacking in a lot of places (like auth-plugins; what exactly the id_extractor is used for, etc.). I really like the tyk-gateway, but a lot of functionality can only be understood by looking at the code. Only by looking at the code I found out about auth-plugins (otto_auth_mw_example.js) and their limitations (for example, when adding “apply_policy_id” to the session created by the auth-plugin, the policy doesn’t seem to be applied).

Also, it isn’t clear to me when I should use a pre-plugin combined with an existing middleware like mw_auth_key.go or when I should use a non-documented auth-plugin (like the example in otto_auth_mw_example.js). The discussion liked above seems to end by preferring the pre-plugin, but it is not really clear to me why.

Even when looking at the code, it is poorly commented and some code seems to be duplicated: For example, compare the function “generateSessionFromPolicy()” of mw_jwt.go with the function ApplyPolicyIfExists() of handler_success.go.

I know writing documentation isn’t fun, but i think this is the most lacking aspect of Tyk.

Hi,

Thanks for the docs feedback - we’re very aware of the hole in the documentation for plugins and are actively working on adding more example.

The problem you have here is that you want to apply rate limits across JWTs for the same user. That is exactly what the JWT and OpenID middleware does (it applies policy and rate limit data to the subject - I.e. The underlying user).

If you are worried about the overhead of the JWT validation, I really wouldn’t, especially considering the overhead that would be added by custom middleware.

The custom auth middleware for rich plugins and Otto work the same way, as do policies, so that just may be a misunderstanding.

I would suggest just going with the provided functionality.

Well, yes and no. Please forget about the jwt. What i really want is to apply rate limits to a user identified by a set header value (not the JWT! But a header value that was extracted from the JWT by the Apache server, like the sub-claim for example). The header value will always be the same for a user.

You may be right about just using the JWT middleware for simplicity. But it is still good to know about how to use a custom pre-plugin together with the auth-key middleware to satisfy the use-case stated in the previous paragraph, just in case if my requirements may change.

Here’s a step by step guide with Python writing an auth plugin:

https://tyk.io/docs/customise-tyk/plugins/rich-plugins/python/custom-auth-python-tutorial/

You could then combine this with the ID extractor, just add this to the custom moddleware section of the manifest and replace the header name:

"id_extractor": {
    "extract_from": "header",
    "extract_with": "value",
    "extractor_config": {
      "header_name": "Authorization"
    }
  },

This will then attempt to cache the header value so that we don’t need to run the middleware over and over.

Thanks, I got is working now, finally! But for other people reading this: These are the issues I stumbled upon, mixed with some questions:

  • I wondered for a long time why my JS-pre-middleware wasn’t loaded by Tyk. I expected the attribute custom_middleware.pre[].path inside my /apps/api.json to be relative to the path configured in middleware_path in my tyk.conf. This is not the case. The configured middleware_path is only used for plugin-bundles (and maybe for other middleware drivers than JSVM). Real complicated stuff! Better use absolute paths in your api configuration!

  • Even better alternative to the previous paragraph: Only configure the middleware_path inside your tyk.conf and don’t touch your api-conf at all! Then place your JS-plugins into [middleware_path]/[api-id]/pre/ (for pre-plugins, in my case /etc/tyk/middleware/7ca7dc85-eee5-4dda-ba4e-f9b7a71f2336/pre/generateKeyMiddleware.js). These plugins will automatically picked up by Tyk without defining them inside your api configuration. When doing it this way, the function inside your js-file needs to be named exactly like the js-file, sans the “.js”-suffix. I don’t know if this mechanism is documented anywhere; I found out about this by looking at the code. This took a while.

  • Don’t forget to set "enable_jsvm": true in your tyk.conf!

  • This took me a loooong time to debug: When accessing request.Headers[]from your plugin, keep in mind that the header names are transformed to camel case. In my case, OIDC_CLAIM_sub has been transformed to Oidc_claim_sub.

  • If your JS-files contain syntax errors or throw exceptions, Tyk won’t help you. Prepare to see the message Failed to decode middleware request data on return from VM: invalid character 'u' looking for beginning of value.

  • Question: Why do I need to define the access_rights for the session when calling TykSetKeyData() when the session-object also contains apply_policy_id? I think it’s redundant to have the access_rights defined inside the policy while being forced to also define them in the session-object created by the pre-plugin.

  • Bonus question: Why do I have to state the api_id both as the name of the object inside the access_rights-object, and then again for the api_id attribute of this very object? Can there be a use-case where the name of the object differs from the api_id specified inside the object?

For completion, here are my configs and plugin source:

/opt/tyk-gateway/tyk.conf:

{
  "listen_address": "",
  "listen_port": 8090,
  "secret": "IT'S A SECRET TO EVERYBODY",
  "template_path": "/opt/tyk-gateway/templates",
  "tyk_js_path": "/opt/tyk-gateway/js/tyk.js",
  "use_db_app_configs": false,
  "app_path": "/etc/tyk/apps",
  "middleware_path": "/etc/tyk/middleware",
  "storage": {
    "type": "redis",
    "host": "localhost",
    "port": 6379,
    "username": "",
    "password": "",
    "database": 0,
    "optimisation_max_idle": 500
  },
  "enable_jsvm": true,
  "enable_analytics": true,
  "health_check": {
    "enable_health_checks": true,
    "health_check_value_timeouts": 60
  },
  "optimisations_use_async_session_write": true,
  "allow_master_keys": false,
  "policies": {
        "policy_source": "file",
        "policy_record_name": "/etc/tyk/policies.json"
    },
  "hash_keys": false,
  "suppress_redis_signal_reload": false
}

/etc/tyl/apps/api.json:

{
      "name": "my-api",
      "api_id": "7ca7dc85-eee5-4dda-ba4e-f9b7a71f2336",
      "enable_context_vars": false,
      "auth": {
        "auth_header_name": "Oidc_claim_sub"
      },
      "version_data": {
          "not_versioned": true,
          "versions": {
              "Default": {
                 "use_extended_paths": true,
                 "name": "Default"
              }
          }
      },
      "proxy": {
          "listen_path": "/",
          "target_url": "http://localhost:8080/"
      },
      "active": true
}

/etc/tyk/middleware/7ca7dc85-eee5-4dda-ba4e-f9b7a71f2336/pre/generateKeyMiddleware.js:

var generateKeyMiddleware = new TykJS.TykMiddleware.NewMiddleware({});

generateKeyMiddleware.NewProcessRequest(function(request, session) {
    var token = request.Headers["Oidc_claim_sub"];
    log("Token: " + token);
    if (!token) {
        request.ReturnOverrides.ResponseCode = 401;
        request.ReturnOverrides.ResponseError = "missing token";
        return generateKeyMiddleware.ReturnData(request, {});
    }

    var session = JSON.parse(TykGetKeyData(token));
    if (session.status == "error") {
        log("Session not found, creating new one");
        session = {
            "apply_policy_id": "vp_api",
            "access_rights": {
                "7ca7dc85-eee5-4dda-ba4e-f9b7a71f2336": {
                    "api_id": "7ca7dc85-eee5-4dda-ba4e-f9b7a71f2336",
                    "versions": [
                        "Default"
                    ]
                }
            }
        };
        TykSetKeyData(token, JSON.stringify(session));
    }

    return generateKeyMiddleware.ReturnData(request, {});
});

// Ensure init with a post-declaration log message
log("Initializing generateKeyMiddleware");

I would love to omit the access_rights-object from the js-plugin…

Thank you for all your help this far!

2 Likes

This has (thankfully) been fixed in v2.4 :slight_smile:

If an access_rights list is empty, then it can be considered a master key, these keys have access to all APIs. Since it would be possible to create a policy, and a token that references it, and then delete the policy, without the token ever being used (the policy is applied on use, not cascading like you would expect in a DB, it’s much faster and more efficient when dealing with millions of keys). Then you could inadvertently create a master key, so we insist on the access rights being set even with a policy ID.

Ah, yes this is a quirk of how tyk creates routing trees, having the data twice this way increases some processing speed under the hood (e.g. no iteration needed with a map, but then we still need a reference once it has been mapped out).

1 Like