Python middleware error when trying example from documentation

Support Administration and Operation
, ,
1

Getting the error when implementing simple Python plugin based on the manual
Tyk info:
deployment option: OSS
tyk version: docker.tyk.io/tyk-gateway/tyk-gateway:v3.1.2

Expected result:
Steps from manual enable simple middleware.

Actual result:
{Middleware error} response when trying to access the application

Steps to reproduce

  1. spin up a new container from docker.tyk.io/tyk-gateway/tyk-gateway:v3.1.2 image
  2. install python prerequisites based on the manual:
apt install python3 python3-dev python3-pip build-essential
pip3 install protobuf grpcio
  1. Create Python file in /mnt/tyk-gateway/middleware with content from the manual.
  2. Update API JSON file based on the manual
  3. Reload Tyk
2

in Debug logs i can see
`time=“Jun 28 08:58:13” level=debug msg=“Registering coprocess middleware, hook name: ResponseHookhook type: Post, driver: python” api_id=1 api_name=MTAPI org_id=LW prefix=coprocess

time=“Jun 28 08:58:13” level=error msg=“Driver ‘python’ isn’t loaded” prefix=coprocess`

3

Looks the same as Driver 'python' isn't loaded

4

Hi Yegor,

Would you mind uploading your API defn please?

Cheers,
Pete

5

Hi Yegor,

Please have a look at Tutorial Add a python plugin bundle to your Gateway and try using a plugin bundle. I’m having success using that but not when placing the python code directly in the middleware directory.

I’ll continue to investigate but this will provide a work-around until I have a better answer.

Cheers,
Pete

6

Before trying bundle, let me share other findings, as they make me doubt that bundle is the way to go.
First, the error says that Python is not loaded.
And I found in the documentation - that to enable python middleware, i have to start another service, which is tyk-gateway-python instead of tyk-gateway
But when I tried this from the container, obviously, it failed because port 8080 is already busy.

So my question is:
what service or what prerequisites shall I run/install to prepare the runtime to execute python?
Because the bundle is just a way of getting the code to execute, while the problem above - runtime setup which is not documented well

7

“I’m having success using that”

Could you please specify, what container image do you use? what entrypoint? what linux service?
Did you install any additional libs before running bundle?
Thanks

8

Hi Yegor,

I’m afraid that is out of date. Python support has been in the default binary since the release of 2.9. There is no extra service that needs to be started.

Cheers,
Pete

9

and also
what is the difference between these serices?
I have checked the init.d files - they are identical

image
image2003×834 49.4 KB

10

Hi,

That script was included for compatibility. For systems which were configured before python was built in, an identical script is included for backwards compatibility

Cheers,
Pete

11

I’ve tested against both 3.1.2 and 3.2.1 docker images.

No extra libs.
This was the manifest I used

{
  "file_list": [
    "4441.py"
  ],
  "custom_middleware": {
    "driver": "python",
    "response": [
    {
      "name": "ResponseHook"
    }
    ]
  }
}

And as I’ve mentioned above there’s no need to run another linux service since Tyk 2.9 was released.

The API was a simple one infront of httpbin but the return headers have

< Injectedkey: injectedvalue

Cheers,
Pete

12

@Pete thank you for the help.
I tried to switch to bundle but to no avail
Based on the manual - create and uploaded sample and updated API config. Tyk downloads it to somewhere but then fails to find it on the file system

Could you, please, help me to nail the issue?

Error:
time="Jun 29 08:44:48" level=info msg="----> Fetching Bundle: bundle" prefix=main time="Jun 29 08:44:48" level=debug msg="----> Saving Bundle: bundle" prefix=main time="Jun 29 08:44:48" level=info msg="----> Loading bundle: bundle" prefix=main time="Jun 29 08:44:48" level=error msg="Couldn't load bundle: open /mnt/tyk-gateway/middleware/bundles/1_94377c156735b39dfa4ac607234cb87c/manifest.json: no such file or directory" api_id=1 org_id=LW path=- prefix=main server_name="http://mlt-mt-api.mlt-modernmt.svc.cluster.local/" user_id=- user_ip=-

Tyk gateway config:

"coprocess_options": {
    "enable_coprocess": true,
    "python_path_prefix": "/opt/tyk-gateway"
},
"enable_bundle_downloader": true,
"bundle_base_url": "https://storage.cloud.google.com/mlt-sandbox/tyk-plugins/",

API gateway config:

  "custom_middleware": {
    "pre": null,
    "post": null,
    "post_key_auth": null,
    "auth_check": {
      "name": "",
      "path": "",
      "require_session": false,
      "raw_body_only": false
    },
    "response": null,
    "driver": "python",
    "id_extractor": {
      "extract_from": "",
      "extract_with": "",
      "extractor_config": null
    }
  },
  "custom_middleware_bundle": "bundle",

bundle content, based on shared above - can be downloaded via
https://storage.cloud.google.com/mlt-sandbox/tyk-plugins/bundle.zip

13

Hi Yegor,

Could you post the manifest file you’re using to create the bundle please? I’d like to see what’s in it before its bundled.

Cheers,
Pete

14

Sure. Here you are

{
    "file_list": [
        "middleware.py"
    ],
    "custom_middleware": {
      "driver": "python",
      "response": [{
        "name": "ResponseHook"
      }]
    }
}
15

Hi Yegor,

I’m not able to recreate the problem, even when using the bundle you attached
A couple of things though.

You don’t need "driver": "python", specified in the API defn when you are using a bundle. That info comes from the manifest.

Perhaps most importantly, when you change or rebuild the bundle you have to either change its name and update the API defn or remove the directory containing it from under /opt/tyk-gateway/middleware/bundles and restart the gateway.

Here are the logs from loading and running your bundle.You can see the error returned from python about 100 being an int.

time="Jun 29 12:31:41" level=info msg="----> Loading bundle: tyk-plugins_bundle" prefix=main
time="Jun 29 12:31:41" level=info msg="----> Verifying bundle: tyk-plugins_bundle" prefix=main
time="Jun 29 12:31:41" level=info msg="----> Bundle verification failed: tyk-plugins_bundle" prefix=main
time="Jun 29 12:31:41" level=info msg="----> Bundle is valid, adding to spec: tyk-plugins_bundle" prefix=main
time="Jun 29 12:31:41" level=info msg="Loading module: '03887748b566415d6e7ab28335400e2e_1e824b2eed8a5ee24cd7cdf54ba30c5e'" prefix=python
time="Jun 29 12:31:41" level=info msg="Checking security policy: Open" api_id=03887748b566415d6e7ab28335400e2e api_name=pytest2 org_id=60db07e75ec6770023473eaf
time="Jun 29 12:31:41" level=info msg="Initialised API Definitions" prefix=main
time="Jun 29 12:31:41" level=info msg="API reload complete" prefix=main
time="Jun 29 12:32:00" level=info msg="ResponseHook request_body contained " prefix=python
time="Jun 29 12:32:00" level=error msg="PyObjectCallObject failed" prefix=python
time="Jun 29 12:32:00" level=error msg="Hook 'ResponseHook' returned an error: 100 has type int, but expected one of: bytes, unicode" prefix=python

And here’s the section from the API

"custom_middleware": {
  "pre": [],
  "post": [],
  "post_key_auth": [],
  "auth_check": {
    "name": "",
    "path": "",
    "require_session": false,
    "raw_body_only": false
  },
  "response": [],
  "driver": "",
  "id_extractor": {
    "extract_from": "",
    "extract_with": "",
    "extractor_config": {}
  }
},
"custom_middleware_bundle": "tyk-plugins_bundle",

Cheers,
Pete

16

I was able to reproduce your positive flow only when I updated custom_middleware_bundle by adding file extension
"custom_middleware_bundle": "bundle.zip",
This change lead me to the same 100 has type int error

Thanks