Tyk policy not enforced correctly for API that uses grpc plugin

jeffrngu · November 8, 2018, 5:00am

Hi,

I’m using Tyk CE 2.3.5. The APIs are configured to use JWT authentication with Tyk policy ID embedded as a claim. A grpc plugin is configured as a “post_key_auth” hook as shown below. MyPosthook does some request transformation before sending the modified request to upstream service.

The policy is configured with white lists (allowed_urls) for upstream APIs with the list of allowed HTTP verbs. Some APIs are configured to forward request to upstream service without grpc plugin while others are configured to invoke grpc plugin after authentication is successful.

For Tyk API without a grpc plugin, the allowed_url is enforced correctly. For example, if the white-list only supports GET verb for endpoint A, and the same endpoint is hit with POST verb, Tyk returns HTTP 403 with error “resource has been disallowed” or something along that line and the request is NOT sent to upstream URL.

However, for Tyk API with a grpc plugin configured, the allowed_url is not enforced correctly. For a similar case as above, Tyk returns the same HTTP 403 and error as stated in the case without grpc. However, the grpc plugin is STILL invoked. I’m expecting Tyk to just return the HTTP 403 without invoking the grpc plugin.

Does this look like a bug or am I missing something in my manifest.json or API config to enforce the policy correctly with the grpc?

manifest.json:
{
“file_list”: ,
“custom_middleware”: {
“pre”:,
“post_key_auth”:[{
“name”: “MyPosthook”
}],
“post”: ,
“driver”: “grpc”
},
“checksum”: “”,
“signature”: “”
}

Josh · November 8, 2018, 6:01am

Hi
Can you update to the most recent gateway first as 2.3 is now quite out of date. If you have similar issues I’ll be more able to help on the newer build.

Thanks
Josh

jeffrngu · December 11, 2018, 7:24pm

@Josh sorry it took me a while to get back to you. Updating to the latest Tyk GW is in our plan but that’ll take a while. In the meantime, I’ve tried configuring the middleware using the “post” processor instead of the “post_key_auth” one. However, with that, I’m hitting another issue: the middleware is not dispatched at all (again with the “post_key_auth” option, the middleware is always dispatched even if authorization check fails). I’ve also set “require_session” to true but that didn’t help.

Does anyone have any idea? What is the difference between the two processor options “post” and “post_key_auth”?

I’m thinking of an alternative where I’ll continue using post_key_auth processor. However, in the Dispatch() method, I’ll need to check the result of authorization check returned from GranularAccessMiddleware. How do I do this?

matiasb · December 13, 2018, 3:35am

post_key_auth are called immediately after the authentication occurs, post is called after several others, we have a graphic to illustrate the flow here.

jeffrngu · December 13, 2018, 7:11am

Thanks @matiasb for the link. A couple of followup questions:

In the Middleware Execution Order diagram, if any of the middleware fails, does Tyk stop processing the rest of the middleware in the chain? For example, if Granular access check fails and returns Http 403, Tyk will skip the rest of the MW starting from “Transform Request” correct?
When I configured my MW with “post_key_auth”, it is called before the Granular access check, which is expected. When I use “post”, my MW is not triggered but the Granular access still happens. So somewhere in the chain after the Granular Access check MW, some MW fails. If I just want my MW with the “Post” processor to be called after “Granular Access Check” passed and don’t care about the rest of the MWs, what kind of API config do I need to achieve this?
As I mentioned in my previous reply, the other workaround I’m looking at is to continue using “post_key_auth” and add kind of redundant logic for granular access check. The only issue I have with this is the HTTP verb is not injected into the session object. Only the Url is available because the protobuf message doesn’t include the HTTP verb. Any idea how to get around this without modifying Tyk code?

Thanks

jeffrngu · December 15, 2018, 2:34am

@matiasb, it looks like HTTP method was added to protobuf in release-2.4 branch under commit id 91db44e3701cc17769433b6e287156b22aeaa25c (#1386). However, when I tried deploying both el/7/tyk-gateway-2.4.0-1.x86_64.rpm - tyk/tyk-gateway · packagecloud and el/7/tyk-gateway-2.4.1-1.x86_64.rpm - tyk/tyk-gateway · packagecloud, neither includes the method in the protobuf message when the MW is invoked.

Can you point me to an RPM when we started having this fix?

I’ve confirmed in my MW code, the protobuf binding class has the “Method” attribute.

type MiniRequestObject struct {
	Headers         map[string]string `protobuf:"bytes,1,rep,name=headers" json:"headers,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
	SetHeaders      map[string]string `protobuf:"bytes,2,rep,name=set_headers,json=setHeaders" json:"set_headers,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
	DeleteHeaders   []string          `protobuf:"bytes,3,rep,name=delete_headers,json=deleteHeaders" json:"delete_headers,omitempty"`
	Body            string            `protobuf:"bytes,4,opt,name=body" json:"body,omitempty"`
	Url             string            `protobuf:"bytes,5,opt,name=url" json:"url,omitempty"`
	Params          map[string]string `protobuf:"bytes,6,rep,name=params" json:"params,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
	AddParams       map[string]string `protobuf:"bytes,7,rep,name=add_params,json=addParams" json:"add_params,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
	ExtendedParams  map[string]string `protobuf:"bytes,8,rep,name=extended_params,json=extendedParams" json:"extended_params,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
	DeleteParams    []string          `protobuf:"bytes,9,rep,name=delete_params,json=deleteParams" json:"delete_params,omitempty"`
	ReturnOverrides *ReturnOverrides  `protobuf:"bytes,10,opt,name=return_overrides,json=returnOverrides" json:"return_overrides,omitempty"`
	Method          string            `protobuf:"bytes,11,opt,name=method" json:"method,omitempty"`
}

jeffrngu · December 17, 2018, 5:00am

Another question: for post-processor MW, is it possible to indirectly write response back to the client? It doesn’t have access to response writer obj though.

matiasb · December 18, 2018, 5:38am

Hi, I suggest trying Tyk 2.7, the additional fields you mentioned should be available in this release, you will find the packages on PackageCloud.

I’m investigating the granular access call you describe, in the meantime post_key_auth could be used.

When generating Protobuf bindings, make sure you use the latest definitions here.

jeffrngu · December 19, 2018, 3:18am

Thanks @matiasb. I found the fix in 2.4.3 so will stick with that for now to minimize deployment changes. Plan to eventually get to a more recent version. I also found out when I use post-processor MW, the chain breaks at virtual endpoint MW, when the virtual endpoint returns a response which is the body of the original request. That is the API response I’m getting when I tried the post-processor option.

I think there should be a processor option for MW execution immediately after the granular access check, similar to the “post_key_auth”; something like “post_granular_access” option. In the absence of this, I have to add redundant granular access check to my post_key_auth" MW.