Opentracing/zipkin

Product Feature Requests & Feedback
21

Cool that worked.

Question - Does the python plugin execute as a separate process? If so is there an ipc between gateway and plugin during an api call?

1 Like
22

Awesome.

The Python interpreter runs as part of the Tyk process, in the main thread. It’s basically an embedded Python interpreter.

During an API call, the GIL is acquired and released as soon as the Python plugin returns.

23

Another question. The current way of enabling plugin seems to be at granularity of single api level. Is there a way to enable plugins at a coarser granularity like group of apis or all apis with exclude list etc?

24

That’s not possible at this time, we may consider it for future releases.

If you’re looking to share Python code, you could introduce a library as part of your Python installation, and load/call it from different bundles.

25

The thread here helped me enormously - I was stuck getting my python middleware bundle to load. I was missing the mkdir and tyk-python steps. Now when tyk starts I see it getting the bundle.zip from the web server. However I am now stuck at the next step…

tyk_1 | time=“Jan 17 16:21:44” level=info msg="----> Loading bundle: bundle.zip"
tyk_1 | time=“Jan 17 16:21:44” level=info msg="----> Verifying bundle: bundle.zip"
tyk_1 | time=“Jan 17 16:21:44” level=info msg="----> Bundle is valid, adding to spec: bundle.zip"
tyk_1 | Fatal Python error: drop_gil: GIL is not locked
tyk_1 | SIGABRT: abort
tyk_1 | PC=0x7fce23ee3c37 m=3
tyk_1 | signal arrived during cgo execution

I am using the code exactly as in the github sample

from tyk.decorators import *
from gateway import TykGateway as tyk

@Hook
def MyPreMiddleware(request, session, spec):
    request.add_header('myheader', 'myvalue')
    return request, session
26

You’re using the Ubuntu version, right?

27

Here’s my Dockerfile.

FROM ubuntu:14.04
RUN apt-get update
RUN apt-get install -y wget curl ca-certificates apt-transport-https curl
RUN curl https://packagecloud.io/gpg.key | apt-key add -
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10

RUN apt-get update && apt-get install -y ca-certificates
RUN apt-get install -y redis-server
RUN apt-get install -y nginx
RUN apt-get install -y wget
RUN apt-get install -y build-essential
RUN apt-get install -y libluajit-5.1-2
RUN apt-get install -y luarocks
RUN luarocks install lua-cjson

RUN wget https://github.com/google/protobuf/releases/download/v3.1.0/protobuf-python-3.1.0.tar.gz
RUN tar -xvzf protobuf-python-3.1.0.tar.gz
RUN cd protobuf-3.1.0/ &&  ./configure -prefix=/usr && make && make install

RUN apt-get install -y python3-setuptools
RUN apt-get install -y python3-dev
RUN cd protobuf-3.1.0/python && python3 setup.py build --cpp_implementation && python3 setup.py install --cpp_implementation
RUN apt-get install -y libpython3.4
RUN apt-get install -y python3-pip
RUN pip3 install grpcio

RUN echo "deb https://packagecloud.io/tyk/tyk-gateway/ubuntu/ trusty main" | sudo tee /etc/apt/sources.list.d/tyk_tyk-gateway.list

RUN echo "deb-src https://packagecloud.io/tyk/tyk-gateway/ubuntu/ trusty main" | sudo tee -a /etc/apt/sources.list.d/tyk_tyk-gateway.list

RUN apt-get update
RUN apt-get install -y tyk-gateway=2.3.1.2

RUN wget https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64 \
    && mv jq-linux64 /usr/local/bin/jq \
    && chmod +x /usr/local/bin/jq

COPY ./tyk.conf /opt/tyk-gateway/tyk.conf
COPY ./run_filter.jq /opt/tyk-gateway/run_filter.jq
COPY ./run.sh /opt/tyk-gateway/run.sh

WORKDIR /opt/tyk-gateway
RUN mkdir -p /opt/tyk-gateway/middleware/bundles
RUN chmod +x run.sh

CMD ["./run.sh"]
EXPOSE 8080

if I exec in to the container and run python3 I note that I cant import gateway at all

root@cd8f23b428ea:/opt/tyk-gateway/coprocess/python/tyk# python3
Python 3.4.3 (default, Nov 17 2016, 01:08:31)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from gateway import TykGateway as tyk
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'gateway'

gateway.pyx/c/h are in /opt/tyk-gateway/coprocess/python/tyk
should I be building tyk instead?

go build -tags 'coprocess python'.

Thanks for your help

Chris

1 Like
28

Do you get the GIL error everytime?
The import error is ok, when Tyk starts, the embedded Python interpreter sets all the paths.

29

It happened everytime before I posted. Now it doesn’t happen at all. I am not at all sure I know what I did different. I am going to drop it unless I see it again. Thanks so much for your responsiveness

30

I ran into a similar problem while running from the docker images. I added this - /opt/tyk-gateway/middleware/bundles to the tyk_gateway section in the docker_compose file. That seemed to resolve the problem. Just thought of letting you know , so that you can update your docker_compose files in the next iteration

2 Likes