Problem update TYK Pump 1.2.0 elasticsearch/kibana 6.8.6

I have updated TYK 3.1.2 and TYK Pump 1.2.0. TYK Pump has stopped working for us, with elasticsearch / kibana 6.8.6. We have verified after various tests that it is the TYK Pump version. Don’t create the index tyk_analytics in elasticsearch. I upload proof of concept to reproduce the problem. The commented versions inside docker-compose are the old versions, that work fine.

< docker-compose>
version: “3”

    services:

      elasticsearch:
        image: elasticse`Preformatted text`arch:6.8.6
        networks:
         - es
        ports:
        - 9200:9200     
        environment:
          - "ES_JAVA_OPTS:-Xms512m-Xmx512m"       
          - discovery.type=single-node
      kibana:
        image: docker.elastic.co/kibana/kibana:6.8.6
        networks:
         - es
        ports:
        - 5601:5601
        environment:
          - ELASTICSEARCH_HOSTS=http://elasticsearch:9200      
        depends_on:
          - elasticsearch
      redis:
        image: redis:6.2.1-alpine
        networks:
        - tyk
      pump:
        image: tykio/tyk-pump-docker-pub:latest
        #image: tykio/tyk-pump-docker-pub:v0.8.3
        depends_on:
          - redis
          - elasticsearch
        networks:
          - tyk
          - es
        volumes:
          - ./data/pump/pump.json:/opt/tyk-pump/pump.conf
      tyk:
        image: tykio/tyk-gateway:v3.1.2
        depends_on:
          - redis
        networks:
          - tyk
        ports:
         - 8080:8080
        tty: true
        volumes:
        - ./data/tyk/apps:/opt/tyk-gateway/apps
        - ./data/tyk/policies/policies.json:/opt/tyk-gateway/policies/policies.json
        environment:
          - TYK_GW_ENABLEANALYTICS=true
          - TYK_GW_HEALTHCHECK_ENABLEHEALTHCHECKS=false
          - TYK_GW_SECRET=foo
        command: tyk

    networks:
      tyk:
        driver: bridge
      es:
        driver: bridge

<pump.conf>
<
{
“analytics_storage_type”: “redis”,
“analytics_storage_config”: {
“type”: “redis”,
“host”: “redis”,
“port”: 6379,
“optimisation_max_idle”: 100,
“optimisation_max_active”: 200,
“enable_cluster”: false
},
“purge_delay”: 5,
“health_check_endpoint_name”: “hello”,
“health_check_endpoint_port”: 8083,
“pumps”: {
“elasticsearch”: {
“type”: “elasticsearch”,
“meta”: {
“index_name”: “tyk_analytics”,
“elasticsearch_url”: “http://elasticsearch:9200”,
“enable_sniffing”: true,
“document_type”: “tyk_analytics”,
“rolling_index”: false,
“extended_stats”: false,
“version”: “6”
}
}
},
“dont_purge_uptime_data”: true
}

Thanks, Greetings

Hi @juanjo_granada,

Thank you for letting us know about this error.

When running with Tyk Pump 1.2.0 what error did you see on your terminal? I just want to confirm that it was in fact Tyk Pump because on my end I received an exit code on elasticsearch. To add to this I received the same error on my end with the image: tykio/tyk-pump-docker-pub:v0.8.3 as well.


Valmir

Thanks @Valmir, attached log pump version 1.2.0:

`time="Apr 12 07:08:57" level=info msg="## Tyk Analytics Pump, v1.2.0 ##"`

`time="Apr 12 07:08:57" level=info msg="--> [REDIS] Creating single-node client"`

`time="Apr 12 07:08:57" level=info msg="Serving health check endpoint at http://localhost:8083/hello ..."`

`time="Apr 12 07:08:58" level=info msg="Elasticsearch URL: http://elasticsearch:9200"`

`time="Apr 12 07:08:58" level=info msg="Elasticsearch Index: tyk_analytics"`

`time="Apr 12 07:09:03" level=error msg="Elasticsearch connection failed: health check timeout: Head http://elasticsearch:9200: dial tcp 172.19.0.2:9200: connect: connection refused: no Elasticsearch node available"`

`time="Apr 12 07:09:13" level=error msg="Elasticsearch connection failed: health check timeout: Head http://elasticsearch:9200: dial tcp 172.19.0.2:9200: connect: connection refused: no Elasticsearch node available"`

`time="Apr 12 07:09:23" level=error msg="Elasticsearch connection failed: health check timeout: Head http://elasticsearch:9200: dial tcp 172.19.0.2:9200: connect: connection refused: no Elasticsearch node available"`

`time="Apr 12 07:09:29" level=info msg="Init Pump: Elasticsearch Pump"`

`time="Apr 12 07:09:29" level=info msg="Starting purge loop @5, chunk size 0"`

`time="Apr 12 07:09:34" level=warning msg="Connection dropped, connecting.."`

`time="Apr 12 09:03:14" level=info msg="Writing 2 records"`

`time="Apr 12 09:03:24" level=info msg="Writing 3 records"`

Writing appears but, the index is not created in elasticsearch.
If we use the version of pump 0.8.3 the writing appears and creates the index.
If you want the complete example, tell me an email address and I will send it to you.

@juanjo_granada Yes please if you wouldn’t mind. My email is [email protected].

Also are you able to to use the 0.8.3 version until there’s a solution for the latest version?


Valmir

Yes, we can use version 0.8.3
Thx

1 Like

Hi @valmir, have you received the email?
greetings thx

Hi @juanjo_granada

Yes I have - thank you for sending it. Apologies on my end as I haven’t had time to look at it yet but I will take a look before the day ends!


Valmir

I can confirm too that pump v0.8.4 works but v1.2.0 doesn’t.
I see the same logs as @juanjo_granada does.

Hi,

Just an update that I was able to recreate this with the versions provided on here and an internal ticket has been created for this issue.

Thank you for reporting this!


Valmir

1 Like