Opentracing/zipkin

PR1 · December 17, 2016, 1:54am

Hi,

Is there a plan to add support for opentracing/zipkin? If not, anytime sooner, is it possible to provide a official JS middleware/plugin as this would be required for anyone dealing with many micro services in their product.

Thanks,
PR

Kos · December 19, 2016, 9:40am

Hi PR,

There’s no plan to support Zipkin out of the box, although the latest release of Tyk Gateway (v2.3) is now supporting Rich Plugins.
If you’re already using Tyk, check the upgrade guide.
Furthermore, I will forward your suggestion to the product team to consider it for a future release.

Thanks,
Kos @ Tyk Support Team

PR1 · December 19, 2016, 7:18pm

Thanks for the reply. Does rich plugins have same level of capability as javascript middleware such that we can write pre and post functionalities as well as generate http requests from rich plugins? It is not very clear from documentation on when one would use javascript middleware vs rich plugins.

Thanks,
PR

Martin · December 19, 2016, 7:48pm

The JavaScript middleware system KS just an ECMAScript interpreter, so the environment is not that rich in terms of libraries and driver access (for example, there’s no DB drivers etc, importing node modules is pretty complex.

Rich plugins give you a full environment, so all of the power of Python3 or all of the power of Java (if using a gRPC implementation).

So for example there’s a Python plugin for datadog and another for loggly using native Python libs to extend Tyk.

https://tyk.io/extend-tyk/tyk-awesome-plugins/

PR1 · December 20, 2016, 2:18am

Thanks. I am trying to test a python plugin based on instructions at https://tyk.io/tyk-documentation/customise-tyk/plugins/rich-plugins/python/tutorial-add-demo-plugin-api/. Couple of questions

The “custom_middleware_bundle”: “test-bundle” ==> should this be filename with extension like bundle.zip?
Where should this file be placed? /opt/tyk-gateway/middleware/python?

Anything else?

Martin · December 20, 2016, 7:11am

That tutorial isn’t particularly helpful in getting you started - you will need to use the bundle CLI tool to make your bundle signed and zipped up.

Then place the bundle onto an asset server like a web server, and set that as the bundle base URL

Then all you need to do is set the bundle ID (the file name portion before he zip extension - the bundle will assign an ID for you) in the api definition.

Check it the readme of the Python demo:

github.com

TykTechnologies/tyk-plugin-demo-python/blob/master/README.md

# Custom Authentication With a Python Plugin

This repository provides a sample [Python](https://www.python.org/) plugin for [Tyk](https://tyk.io).

This plugin implements a custom authentication middleware, check [our tutorial](https://tyk.io/docs/customise-tyk/plugins/rich-plugins/python/custom-auth-python-tutorial/) for more details.

And the bundler readme:

github.com

TykTechnologies/tyk-cli/blob/develop/README.md

# tyk-cli

Tyk CLI utility.

[![Build Status](https://travis-ci.org/TykTechnologies/tyk-cli.svg?branch=master)](https://travis-ci.org/TykTechnologies/tyk-cli)
[![Coverage Status](https://coveralls.io/repos/github/TykTechnologies/tyk-cli/badge.svg?branch=master)](https://coveralls.io/github/TykTechnologies/tyk-cli?branch=master)

**Note:** Starting on Tyk Gateway v2.8, the bundle command is included as part of the gateway binary so it's no longer required to install `tyk-cli` as a separate tool, see [here](https://tyk.io/docs/plugins/rich-plugins/plugin-bundles/#getting-the-bundler-tool) for more details.

## Install

	go get -u github.com/TykTechnologies/tyk-cli

## Available modules

### Bundle

This module provides useful commands for working with custom middleware bundles. The most basic command is `build`:

Assuming you're on a directory that contains your required bundle files and a **bundle manifest**, you could run:

This file has been truncated. show original

The bundler is shipped with the gateway it should be in the gateway in a sub directory (/opt/tyk-gateway/utils/tyk-cli)

PR1 · December 20, 2016, 7:27pm

Thanks Martin. I am still not able to get it working. I did the following

Created a bundle.zip using tyk-cli with -y option
Put the bunde.zip in /vagrant folder in my vagrant box
Started a simple golang rest server that serves file using below go code
Added following entries in tyk.conf and restarted tyk-gateway
“enable_bundle_downloader”: true,
“bundle_base_url”: “http://127.0.0.1:10080/”,
Added following entry in api definition
“custom_middleware_bundle”: “bundle”,

I dont see any prints in tyk-gateway.log related to loading of bundle or any errors and plugin isnt getting invoked either. What did I miss?

package main

import (
“log”
“net/http”
)

func main() {
// Simple static webserver:
log.Fatal(http.ListenAndServe(“:10080”, http.FileServer(http.Dir(“/vagrant”))))
}

matiasb · December 21, 2016, 11:50am

Hi PR, did you write a manifest file for your bundle?

Can you share it here?

Best.

PR1 · December 21, 2016, 5:06pm

Hi Matias,

It is this one GitHub - TykTechnologies/tyk-plugin-correlation: Add a correlation ID to your requests, and trace them as they pass through your infrastructure. Requirements

{
“file_list”: [
“middleware.py”,
“correlation.py”
],
“custom_middleware”: {
“pre”: [
{
“name”: “PreMiddleware”
}
],
“post”: [
{
“name”: “PostMiddleware”
}
],
“driver”: “python”
},
“checksum”: “”,
“signature”: “”
}

matiasb · December 21, 2016, 5:08pm

Is your bundle file served as “bundle.zip” or “bundle”?

PR1 · December 21, 2016, 5:26pm

File name is bundle.zip

matiasb · December 21, 2016, 5:29pm

Ah, great!
It seems your API definition uses bundle, you will need to use bundle.zip.

Also, if the right version of Tyk is being loaded, the following message will show up in the log:

Initializing interpreter, Py_Initialize()

PR1 · December 21, 2016, 5:33pm

Hmm, I used bundle.zip as well with same result. I tried this demo plugin as well (tyk-plugin-demo-python/manifest.json at master · TykTechnologies/tyk-plugin-demo-python · GitHub). What else could I be missing? I dont see the initializing line in tyk-gateway.log.

matiasb · December 21, 2016, 6:02pm

How do you start the gateway?

PR1 · December 21, 2016, 6:10pm

sudo service tyk-gateway start

matiasb · December 21, 2016, 6:14pm

Try this:
sudo service tyk-gateway stop

Then:
sudo service tyk-gateway-python start

PR1 · December 21, 2016, 6:26pm

Issue still persists. I have pasted full output of tyk-gateway-python.log below

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ sudo cat /var/log/upstart/tyk-gateway-python.log
[Dec 20 14:34:46] INFO Connection dropped, connecting…
[Dec 20 14:34:46] INFO host-check-mgr: Starting Poller
[Dec 20 14:34:46] INFO main: Setting up analytics normaliser
[Dec 20 14:34:46] INFO main: PIDFile location set to: ./tyk-gateway.pid
[Dec 20 14:34:46] INFO main: Initialising Tyk REST API Endpoints
[Dec 20 14:34:46] INFO main: → Standard listener (http)
[Dec 20 14:34:46] INFO main: Setting up Server
[Dec 20 14:34:46] INFO main: Waiting for zeroconf signal…
[Dec 20 14:34:48] WARN pub-sub: Insecure configuration detected (allowing)!
[Dec 20 14:34:48] INFO pub-sub: Hostname set with dashboard zeroconf signal
[Dec 20 14:34:48] INFO main: Registering node.
[Dec 20 14:34:48] INFO dashboard: Node registered id=67d7fb9b-ff56-4672-734b-8d6579a99c74
[Dec 20 14:34:48] INFO main: Initialising distributed rate limiter
[Dec 20 14:34:48] INFO Starting gateway rate imiter notifications…
[Dec 20 14:34:48] INFO main: Detected 3 APIs
[Dec 20 14:34:48] INFO main: Loading API configurations.
[Dec 20 14:34:48] INFO main: Tracking hostname api_name=Test API2 domain=(no host)
[Dec 20 14:34:48] INFO main: Tracking hostname api_name=Test API3 domain=(no host)
[Dec 20 14:34:48] INFO main: Tracking hostname api_name=Test API domain=(no host)
[Dec 20 14:34:48] INFO main: Loading API api_name=Test API2
[Dec 20 14:34:48] INFO main: Checking security policy: Token api_name=Test API2
[Dec 20 14:34:48] INFO gateway: Loaded: Test API2 api_id=d7516f793fcc4c8e5e4f61bdbd800a06 org_id=585880030640fd19584db29e server_name=-- user_id=-- user_ip=–
[Dec 20 14:34:48] INFO main: Loading API api_name=Test API3
[Dec 20 14:34:48] INFO main: Checking security policy: Token api_name=Test API3
[Dec 20 14:34:48] INFO gateway: Loaded: Test API3 api_id=d2b339ac83544e246443bec559a37aa2 org_id=585880030640fd19584db29e server_name=-- user_id=-- user_ip=–
[Dec 20 14:34:48] INFO main: Loading API api_name=Test API
[Dec 20 14:34:48] INFO main: Checking security policy: Token api_name=Test API
[Dec 20 14:34:48] INFO gateway: Loaded: Test API api_id=538b1f22a3f145715135261dab32e030 org_id=585880030640fd19584db29e server_name=-- user_id=-- user_ip=–
[Dec 20 14:34:49] INFO main: Processed and listening on: /test-api2/{rest:.}
[Dec 20 14:34:49] INFO main: Processed and listening on: /test-api3/{rest:.}
[Dec 20 14:34:49] INFO main: Processed and listening on: /test-api/{rest:.*}
[Dec 20 14:34:49] INFO host-check-mgr: Loading uptime tests…
[Dec 20 14:34:49] INFO main: Initialised API Definitions
[Dec 20 14:34:49] INFO main: Loading policies
[Dec 20 14:34:49] INFO main: Using Policies from Dashboard Service
[Dec 20 14:34:49] INFO policy: Mutex lock acquired… calling
[Dec 20 14:34:49] INFO policy: Calling dashboard service for policy list
[Dec 20 14:34:49] INFO policy: Processing policy list
[Dec 20 14:34:49] INFO main: Gateway started (v2.3.0.33)
[Dec 20 14:34:49] INFO main: → Listening on address: (open interface)
[Dec 20 14:34:49] INFO main: → Listening on port: 8080
[Dec 20 14:34:49] INFO main: → PID: 16683
[Dec 20 14:34:51] INFO pub-sub: Using /opt/tyk-gateway/tyk.conf for configuration
[Dec 20 14:34:51] INFO pub-sub: Configuration request responded.

matiasb · December 21, 2016, 6:30pm

Have you modified your tyk.conf to enable rich plugins?

The most important setting is:

    "coprocess_options": {
      "enable_coprocess": true,
    }

PR1 · December 21, 2016, 6:35pm

Cool that knob made it progress a bit more. Now I see below error.

[Dec 20 14:49:05] ERROR main: Couldn’t create bundle directory: mkdir middleware/bundles/d7516f793fcc4c8e5e4f61bdbd800a06-bundle.zip: no such file or directory api_id=d7516f793fcc4c8e5e4f61bdbd800a06 org_id=585880030640fd19584db29e path=- server_name=http://my-tyk-instance.com:9090/ user_id=- user_ip=-
[Dec 20 14:49:05] INFO main: Checking security policy: Token api_name=Test API2
[Dec 20 14:49:05] INFO gateway: Loaded: Test API2 api_id=d7516f793fcc4c8e5e4f61bdbd800a06 org_id=585880030640fd19584db29e server_name=-- user_id=-- user_ip=–

matiasb · December 21, 2016, 6:36pm

Initialize the bundle directory using:

mkdir -p /opt/tyk-gateway/middleware/bundles

And then restart Tyk.

We’re currently working on updating the docs for this feature.

Best.