Python Plugin for IP Rate Limiting

Support Operation and Testing
, ,
1

Hello,

We’re working to integrate tyk to our company, and want to leverage the powerful python plugins.
We have managed to get the JavaScript middle wear for Ip Rate limiting working, but we want to write the same functionality in python to be able to pair with other plugins easier.

So far, we’ve been unable to get tyk to recognize the key or session data. We are receiving the following errors in the log:

[Apr 3 20:09:57] DEBUG Querying local cache [Apr 3 20:09:57] DEBUG Querying keystore [Apr 3 20:09:57] DEBUG STORE: Getting WAS: 10.55.12.107 [Apr 3 20:09:57] DEBUG Input key was: apikey-e0a1c264 [Apr 3 20:09:57] DEBUG STORE: Getting: apikey-e0a1c264 [Apr 3 20:09:57] DEBUG Input key was: apikey-e0a1c264 [Apr 3 20:09:57] DEBUG Error trying to get value:redigo: nil returned [Apr 3 20:09:57] DEBUG auth-mgr: Could not get session detail, key not found err=Key not found inbound-key=****.107 [Apr 3 20:09:57] DEBUG Querying authstore [Apr 3 20:09:57] DEBUG STORE: Getting WAS: 10.55.12.107 [Apr 3 20:09:57] DEBUG Input key was: apikey-e0a1c264 [Apr 3 20:09:57] DEBUG STORE: Getting: apikey-e0a1c264 [Apr 3 20:09:57] DEBUG Input key was: apikey-e0a1c264 [Apr 3 20:09:57] DEBUG Error trying to get value:redigo: nil returned [Apr 3 20:09:57] WARN auth-mgr: Key not found in storage engine err=Key not found inbound-key=****.107 [Apr 3 20:09:57] INFO Attempted access with non-existent key. key=10.55.12.107 origin=127.0.0.1 path=/policy/ [Apr 3 20:09:57] ERROR gateway: request error: Key not authorised api_id=cd6235cbdb404a4646f947ac20837e16 org_id=58cad6dfd81dd609b3d249cc path=/ server_name=https://www.amazon.com user_id= user_ip=127.0.0.1

We have been extending the python initial plugin example, and are currently at the following state:

from tyk.decorators import *
from gateway import TykGateway as tyk

from tyk.session import AccessSpec, AccessDefinition, BasicAuthData, JWTData, Monitor, TykSession

@Hook
def MyPreMiddleware(request, session, spec):
print(request)

print("Session --->")
session.apply_policy_id = "58cae541d81dd609b3d249d5"
session.org_id = "58cad6dfd81dd609b3d249cc"

print("Spec ------->")
print(spec)

#print(TykSession)

request.add_header('myheader', 'myvalue')
ip = request.get_header('X-Real-Ip')
print(ip)
print("Hello")
request.add_header('x-limit-authorization', ip)
session.metadata = ip
print(session)
print(tyk)
#tyk.store_data(ip,"hello",10000)
#tyk.store_data(ip,keyDetails,10090)
#metadata[ip] = keyDetails
#print(metadata)

return request, session`

We’ve tried to edit the session object within the Pre Hook, as well as saving with the store-data function.

It would be greatly appreciated if we could get some help, or some more advanced examples of python plugins to read through.

Thanks,

-Arturo

2

I’d suggest that instead of using the pre middleware, you write an auth plugin instead (see the authentication section in this doc):

3

Thanks for the help. We moved on to using the Auth Middlewear for the python rate limiter. We currently have this snippet:
session.rate = 10.0 session.per = 5.0 session.apply_policy_id = "58ed16c2d81dd63719640c19" metadata['token'] = request.get_header('X-Real-Ip')

Our problem is that the apply_policy_id section doesn’t load in the policy that we want. Otherwise, the rate and per do rate limit correctly based on the metadata['token']

Are we missing anything in terms of applying policies through the session object?

4

Can you confirm what the value in your tyk.conf is for disable_cached_session_state? The in-memory cache, if enabled, might stop the policy.

If it’s disabled (set to true), then there may be a bug we need to look at (cc @matiasb)

5

Right, it might be useful to see your tyk.conf settings.

6

Hey, sorry for the late response. Here’s our tyk.conf settings:

{
“listen_port”: 8080,
“node_secret”: “352d20ee67be67f6340b4c0605b044b7”,
“secret”: “352d20ee67be67f6340b4c0605b044b7”,
“template_path”: “/opt/tyk-gateway/templates”,
“tyk_js_path”: “/opt/tyk-gateway/js/tyk.js”,
“use_db_app_configs”: true,
“db_app_conf_options”: {
“connection_string”: “”,
“node_is_segmented”: false,
“tags”: []
},
“disable_dashboard_zeroconf”: false,
“app_path”: “/opt/tyk-gateway/apps”,
“middleware_path”: “/opt/tyk-gateway/middleware”,
“storage”: {
“type”: “redis”,
“host”: “localhost”,
“port”: 6379,
“username”: “”,
“password”: “”,
“database”: 0,
“optimisation_max_idle”: 2000,
“optimisation_max_active”: 4000
},
“enable_analytics”: true,
“analytics_config”: {
“type”: “mongo”,
“pool_size”: 100,
“csv_dir”: “/tmp”,
“mongo_url”: “”,
“mongo_db_name”: “”,
“mongo_collection”: “”,
“purge_delay”: 100,
“ignored_ips”: [],
“enable_detailed_recording”: false,
“enable_geo_ip”: false,
“geo_ip_db_path”: “”,
“normalise_urls”: {
“enabled”: true,
“normalise_uuids”: true,
“normalise_numbers”: true,
“custom_patterns”: []
}
},
“health_check”: {
“enable_health_checks”: false,
“health_check_value_timeouts”: 60
},
“optimisations_use_async_session_write”: true,
“allow_master_keys”: true,
“policies”: {
“policy_source”: “service”,
“policy_connection_string”: “”,
“policy_record_name”: “tyk_policies”,
“allow_explicit_policy_id”: true
},
“hash_keys”: true,
“suppress_redis_signal_reload”: false,
“use_redis_log”: true,
“close_connections”: true,
“enable_non_transactional_rate_limiter”: true,
“enable_sentinel_rate_limiter”: false,
“experimental_process_org_off_thread”: true,
“local_session_cache”: {
“disable_cached_session_state”: true
},
“http_server_options”: {
“enable_websockets”: true
},
“uptime_tests”: {
“disable”: false,
“config”: {
“enable_uptime_analytics”: false,
“failure_trigger_sample_size”: 3,
“time_wait”: 1,
“checker_pool_size”: 50
}
},
“hostname”: “”,
“enable_custom_domains”: true,
“enable_jsvm”: true,
“oauth_redirect_uri_separator”: “;”,

“coprocess_options”: {
“enable_coprocess”: true
},
“enable_bundle_downloader”: false,
“bundle_base_url”: “”,
“pid_file_location”: “./tyk-gateway.pid”,

“allow_insecure_configs”: true,
“public_key_path”: “”,
“close_idle_connections”: false,
“allow_remote_config”: false,

“global_session_lifetime”: 100,
“force_global_session_lifetime”: false,
“max_idle_connections_per_host”: 100
}