Opentracing/zipkin

Hi,

Is there a plan to add support for opentracing/zipkin? If not, anytime sooner, is it possible to provide a official JS middleware/plugin as this would be required for anyone dealing with many micro services in their product.

Thanks,
PR

Hi PR,

There’s no plan to support Zipkin out of the box, although the latest release of Tyk Gateway (v2.3) is now supporting Rich Plugins.
If you’re already using Tyk, check the upgrade guide.
Furthermore, I will forward your suggestion to the product team to consider it for a future release.

Thanks,
Kos @ Tyk Support Team

Thanks for the reply. Does rich plugins have same level of capability as javascript middleware such that we can write pre and post functionalities as well as generate http requests from rich plugins? It is not very clear from documentation on when one would use javascript middleware vs rich plugins.

Thanks,
PR

The JavaScript middleware system KS just an ECMAScript interpreter, so the environment is not that rich in terms of libraries and driver access (for example, there’s no DB drivers etc, importing node modules is pretty complex.

Rich plugins give you a full environment, so all of the power of Python3 or all of the power of Java (if using a gRPC implementation).

So for example there’s a Python plugin for datadog and another for loggly using native Python libs to extend Tyk.

https://tyk.io/extend-tyk/tyk-awesome-plugins/

Thanks. I am trying to test a python plugin based on instructions at https://tyk.io/tyk-documentation/customise-tyk/plugins/rich-plugins/python/tutorial-add-demo-plugin-api/. Couple of questions

  1. The “custom_middleware_bundle”: “test-bundle” ==> should this be filename with extension like bundle.zip?
  2. Where should this file be placed? /opt/tyk-gateway/middleware/python?

Anything else?

That tutorial isn’t particularly helpful in getting you started - you will need to use the bundle CLI tool to make your bundle signed and zipped up.

Then place the bundle onto an asset server like a web server, and set that as the bundle base URL

Then all you need to do is set the bundle ID (the file name portion before he zip extension - the bundle will assign an ID for you) in the api definition.

Check it the readme of the Python demo:

And the bundler readme:

The bundler is shipped with the gateway it should be in the gateway in a sub directory (/opt/tyk-gateway/utils/tyk-cli)

Thanks Martin. I am still not able to get it working. I did the following

  1. Created a bundle.zip using tyk-cli with -y option
  2. Put the bunde.zip in /vagrant folder in my vagrant box
  3. Started a simple golang rest server that serves file using below go code
  4. Added following entries in tyk.conf and restarted tyk-gateway
    “enable_bundle_downloader”: true,
    “bundle_base_url”: “http://127.0.0.1:10080/”,
  5. Added following entry in api definition
    “custom_middleware_bundle”: “bundle”,

I dont see any prints in tyk-gateway.log related to loading of bundle or any errors and plugin isnt getting invoked either. What did I miss?

package main

import (
“log”
“net/http”
)

func main() {
// Simple static webserver:
log.Fatal(http.ListenAndServe(":10080", http.FileServer(http.Dir("/vagrant"))))
}

Hi PR, did you write a manifest file for your bundle?

Can you share it here?

Best.

Hi Matias,

It is this one https://github.com/TykTechnologies/tyk-plugin-correlation

{
“file_list”: [
“middleware.py”,
“correlation.py”
],
“custom_middleware”: {
“pre”: [
{
“name”: “PreMiddleware”
}
],
“post”: [
{
“name”: “PostMiddleware”
}
],
“driver”: “python”
},
“checksum”: “”,
“signature”: “”
}

Is your bundle file served as “bundle.zip” or “bundle”?

File name is bundle.zip

Ah, great!
It seems your API definition uses bundle, you will need to use bundle.zip.

Also, if the right version of Tyk is being loaded, the following message will show up in the log:

Initializing interpreter, Py_Initialize()

Hmm, I used bundle.zip as well with same result. I tried this demo plugin as well (https://github.com/TykTechnologies/tyk-plugin-demo-python/blob/master/manifest.json). What else could I be missing? I dont see the initializing line in tyk-gateway.log.

How do you start the gateway?

sudo service tyk-gateway start

Try this:
sudo service tyk-gateway stop

Then:
sudo service tyk-gateway-python start

Issue still persists. I have pasted full output of tyk-gateway-python.log below

[email protected]:/vagrant$ sudo cat /var/log/upstart/tyk-gateway-python.log
[Dec 20 14:34:46] INFO Connection dropped, connecting…
[Dec 20 14:34:46] INFO host-check-mgr: Starting Poller
[Dec 20 14:34:46] INFO main: Setting up analytics normaliser
[Dec 20 14:34:46] INFO main: PIDFile location set to: ./tyk-gateway.pid
[Dec 20 14:34:46] INFO main: Initialising Tyk REST API Endpoints
[Dec 20 14:34:46] INFO main: --> Standard listener (http)
[Dec 20 14:34:46] INFO main: Setting up Server
[Dec 20 14:34:46] INFO main: Waiting for zeroconf signal…
[Dec 20 14:34:48] WARN pub-sub: Insecure configuration detected (allowing)!
[Dec 20 14:34:48] INFO pub-sub: Hostname set with dashboard zeroconf signal
[Dec 20 14:34:48] INFO main: Registering node.
[Dec 20 14:34:48] INFO dashboard: Node registered id=67d7fb9b-ff56-4672-734b-8d6579a99c74
[Dec 20 14:34:48] INFO main: Initialising distributed rate limiter
[Dec 20 14:34:48] INFO Starting gateway rate imiter notifications…
[Dec 20 14:34:48] INFO main: Detected 3 APIs
[Dec 20 14:34:48] INFO main: Loading API configurations.
[Dec 20 14:34:48] INFO main: Tracking hostname api_name=Test API2 domain=(no host)
[Dec 20 14:34:48] INFO main: Tracking hostname api_name=Test API3 domain=(no host)
[Dec 20 14:34:48] INFO main: Tracking hostname api_name=Test API domain=(no host)
[Dec 20 14:34:48] INFO main: Loading API api_name=Test API2
[Dec 20 14:34:48] INFO main: Checking security policy: Token api_name=Test API2
[Dec 20 14:34:48] INFO gateway: Loaded: Test API2 api_id=d7516f793fcc4c8e5e4f61bdbd800a06 org_id=585880030640fd19584db29e server_name=-- user_id=-- user_ip=–
[Dec 20 14:34:48] INFO main: Loading API api_name=Test API3
[Dec 20 14:34:48] INFO main: Checking security policy: Token api_name=Test API3
[Dec 20 14:34:48] INFO gateway: Loaded: Test API3 api_id=d2b339ac83544e246443bec559a37aa2 org_id=585880030640fd19584db29e server_name=-- user_id=-- user_ip=–
[Dec 20 14:34:48] INFO main: Loading API api_name=Test API
[Dec 20 14:34:48] INFO main: Checking security policy: Token api_name=Test API
[Dec 20 14:34:48] INFO gateway: Loaded: Test API api_id=538b1f22a3f145715135261dab32e030 org_id=585880030640fd19584db29e server_name=-- user_id=-- user_ip=–
[Dec 20 14:34:49] INFO main: Processed and listening on: /test-api2/{rest:.}
[Dec 20 14:34:49] INFO main: Processed and listening on: /test-api3/{rest:.
}
[Dec 20 14:34:49] INFO main: Processed and listening on: /test-api/{rest:.*}
[Dec 20 14:34:49] INFO host-check-mgr: Loading uptime tests…
[Dec 20 14:34:49] INFO main: Initialised API Definitions
[Dec 20 14:34:49] INFO main: Loading policies
[Dec 20 14:34:49] INFO main: Using Policies from Dashboard Service
[Dec 20 14:34:49] INFO policy: Mutex lock acquired… calling
[Dec 20 14:34:49] INFO policy: Calling dashboard service for policy list
[Dec 20 14:34:49] INFO policy: Processing policy list
[Dec 20 14:34:49] INFO main: Gateway started (v2.3.0.33)
[Dec 20 14:34:49] INFO main: --> Listening on address: (open interface)
[Dec 20 14:34:49] INFO main: --> Listening on port: 8080
[Dec 20 14:34:49] INFO main: --> PID: 16683
[Dec 20 14:34:51] INFO pub-sub: Using /opt/tyk-gateway/tyk.conf for configuration
[Dec 20 14:34:51] INFO pub-sub: Configuration request responded.

Have you modified your tyk.conf to enable rich plugins?

The most important setting is:

    "coprocess_options": {
      "enable_coprocess": true,
    }

Cool that knob made it progress a bit more. Now I see below error.

[Dec 20 14:49:05] ERROR main: Couldn’t create bundle directory: mkdir middleware/bundles/d7516f793fcc4c8e5e4f61bdbd800a06-bundle.zip: no such file or directory api_id=d7516f793fcc4c8e5e4f61bdbd800a06 org_id=585880030640fd19584db29e path=- server_name=http://my-tyk-instance.com:9090/ user_id=- user_ip=-
[Dec 20 14:49:05] INFO main: Checking security policy: Token api_name=Test API2
[Dec 20 14:49:05] INFO gateway: Loaded: Test API2 api_id=d7516f793fcc4c8e5e4f61bdbd800a06 org_id=585880030640fd19584db29e server_name=-- user_id=-- user_ip=–

Initialize the bundle directory using:

mkdir -p /opt/tyk-gateway/middleware/bundles

And then restart Tyk.

We’re currently working on updating the docs for this feature.

Best.