Upgrade tyk2.0 from older version, Also want to setup in docker

Is mongodb is a dependency for Tyk gateway 2.0 & for Tyk dashboard? Docker documents says mongo db is required. Want to upgrade to V2.0 on docker.

Mongo DB is a dependency for Tyk Dashboard

Hello Martin and tushardwivedi sorry to meddle in but my setup also involves docker and tyk. I’m trying to see dashboard, like I used to, but appears a license requesting screen. Is this normal? The Tyk dashboard it’s not free in this new version?What can I do to upgrade and maintain my previous setup? I already have seen the upgrade guide but only thought that the tweaking of tyk.conf and tyk.analytics.conf was necessary.

Ttyk Dashboard is free for 1 gateway, any more it is not. You can get a free license from the site

There’s upgrade notes here, and tweaking the configs is the main thing you need to do, and add a license :slight_smile:

Hello Martin,

I followed the conf files in the docker setup config. And also requested a licence key. But the dashboard still ask me for the licence key. The login page only becomes available when i put the licence key by hand. What I’m missing?

Here the my config file sample:

I assume all those curly braces ({{range etc.) are some kind of custom configuration management? Because that’s definitely not what it’s supposed to look like :-/

If you’ve added the license to the tyk_analytics.conf file, and are mounting the conf file into the container, then you shouldn’t be getting a license request screen unless the config isn’t being read.

Can you share the output of the tyk_dashboard container when it starts up? It’ll tell you a lot.

Hello martin the brackets are for my consul integration. But I have tested by defining static values and is the same.

Dashboard Output log:
time="May 24 09:46:57" level=info msg="Using /opt/tyk-dashboard/tyk_analytics.conf for configuration" time="May 24 09:46:58" level=info msg="Creating new Redis connection pool" time="May 24 09:46:58" level=info msg="Creating new Redis connection pool" time="May 24 09:47:00" level=info msg="Using /opt/tyk-dashboard/tyk_analytics.conf for configuration" time="May 24 09:47:00" level=info msg="Creating new Redis connection pool" time="May 24 09:47:00" level=info msg="Creating new Redis connection pool" time="May 24 09:47:00" level=info msg="Creating new Redis connection pool" time="May 24 09:47:00" level=info msg="Creating new Redis connection pool" time="May 24 09:47:00" level=info msg="Adding available nodes..." time="May 24 09:47:00" level=info msg="Tyk Analytics Dashboard 1.1.0.0" time="May 24 09:47:00" level=info msg="Copyright Martin Buhr 2016" time="May 24 09:47:00" level=info msg="https://www.tyk.io" time="May 24 09:47:00" level=info msg="Listening on port: 3000" time="May 24 09:47:00" level=info msg="Loading routes..." time="May 24 09:47:00" level=info msg="Registering nodes..." time="May 24 09:47:00" level=info msg="Adding available nodes..." time="May 24 09:47:00" level=info msg="--> Standard listener (http)" time="May 24 09:47:00" level=info msg="Creating new Redis connection pool" time="May 24 09:47:00" level=info msg="Socket server started" time="May 24 09:47:01" level=error msg="No nodes available"
Gateway Log:
time="May 24 09:46:52" level=info msg="Connection dropped, connecting.." time="May 24 09:46:53" level=info msg="Setting up Server" time="May 24 09:46:53" level=info msg="--> Standard listener (http)" time="May 24 09:46:53" level=info msg="Loading API Specification from /opt/tyk-gateway/apps/app_sample.json" time="May 24 09:46:53" level=info msg="Detected 1 APIs" time="May 24 09:46:53" level=info msg="--> Loading API: Tyk Test API" time="May 24 09:46:53" level=info msg="----> Tracking: (no host)" time="May 24 09:46:53" level=info msg="----> Checking security policy: Token" time="May 24 09:46:53" level=info msg="Loading uptime tests..." time="May 24 09:46:53" level=info msg="Starting Poller" time="May 24 09:46:53" level=info msg="Gateway started (v2.1.0.2)" time="May 24 09:46:53" level=info msg="--> Listening on port: 8080" 2016/05/24 09:46:54 terminated time="May 24 09:46:55" level=info msg="Connection dropped, connecting.." time="May 24 09:46:55" level=warning msg="Cache purging is no longer part of Tyk Gateway, please use Tyk-Pump." time="May 24 09:46:55" level=info msg="Setting up Server" time="May 24 09:46:55" level=info msg="--> Standard listener (http)" time="May 24 09:46:55" level=info msg="Registering node." time="May 24 09:47:01" level=error msg="Failed to register node, retrying in 5s"

Side Note:

I’m not using Tyk pump. For now i only need dashboard and gateway to work.

So first off, your gateway is misconfigured. You are running part of the gateway in standalone (file-based) mode, you need to configure it to include the dashboard connection string (as per our upgrade docs):

"use_db_app_configs": true,
    "db_app_conf_options": {
        "connection_string": "http://dashboard_host:port",
        "node_is_segmented": false,
        "tags": []
 },

And:

"policies": {
        "policy_source": "service",
        "policy_connection_string": "http://dashboard_host:port"
},

You will also need to set the shared secret:

tyk.conf
"node_secret": "node-secret-goes-here",

tyk_analytics.conf
"shared_node_secret": "node-secret-goes-here",

This log entry here:

Tells you that the node can’t speak to the dashboard, or when it is speaking to the dashboard, is failing because the shared secrets don’t matchup.

This implies that your tyk_analytics.conf does not have a valid license, is the tyk_analytics.conf file properly mounted into the dashboard container in the right location? How are you starting the dashboard container?

Hello Martin I’m starting my container by docker compose. Now I need the consul integration to work and for this I launch my containers with supervisor and the correspondent task for dashboard is initiated. I would like to send my generated tyk and tyk_analytics conf files. How can i do that?

That might be a better question for a consul forum :wink: but I assume you will somehow need to hijack the volume mount functionality in docker to override the tyk_analyitcs.conf and tyk.conf file that is in the container.

With docker this is usually a -v from:to flag in the command line that starts the container, it’s right there in the docs for docker. As for compose, there’s a config entry for that.

I’m not sure how to bridge the gap between consul and supervisord / file system so that you can execute the mount though.

Sorry Martin for my English :frowning:

What wanted to do is to send my config files to you so that you see that i have in place the config changes that you told me to do. And it’s not working still. I could posted them here but they are a little big.

No worries Carlos,

Just to confirm the symptoms again: The dashboard is always asking for a license, and from the looks of it, you gateway is misconfigured.

Now I have a feeling that the config files you are mounting are not actually being read by the docker container, because otherwise the license would be read and respected.

Can you confirm, by going into the container and obtaining a shell, whether or not the files have been mounted to the correct location where the gateway is installed?

Martin,
Yes I can confirm the symptoms

In the image you can see the aspect of my containers

Now I don’t have a volume with tyk config files. I copy the consul configuration template to a folder and consul template generates the appropriate config file. When a change occurs the tyk executable is restarted. But like I said before I have provide a single static file for config and dashboard always asks for licence…And gateway can’t register the single node.

How is the Tyk executable started? Are you passing the --conf flag?

Yes yes…

The commands are:
Gateway Command
/opt/tyk-gateway/tyk --conf=/opt/tyk-gateway/tyk.conf

Dashboard Command
/opt/tyk-dashboard/tyk-analytics --conf=/opt/tyk-dashboard/tyk_analytics.conf

Looking at your dashboard output, the dashboard seems to be reading the conf file twice:

time="May 24 09:46:57" level=info msg="Using /opt/tyk-dashboard/tyk_analytics.conf for configuration"
time="May 24 09:47:00" level=info msg="Using /opt/tyk-dashboard/tyk_analytics.conf for configuration"

Why would it be doing that? Could it be that the process is being started twice (or killed, then restarted by supervisord?)

The conf file looks ok, from your screenshot, but if Tyk is asking for a license at the login screen, then the instance that is running does not have a license in the tyk_analytics.conf, it’s really not complicated.

If you get rid of all the complexity of consul and supervisord, and just start the dashboard with a single command in the shell of the container (with the process stopped, obvs), then:

  1. Look at the contents of the analytics conf to see if your file is correct and license is present
  2. If the license is present, does it still ask for a manual entry?

I have put the file statically without any automatic intervention. And now the dashboard site does not even open. I will debug and came back to you…

Martin another question about setting up the link between dashboard and gateway. Does it no support full IP’s addresses?

It’s giving me this error on gateway:

time="May 24 15:59:35" level=error msg="Couldn't unmarshal configuration"
time="May 24 15:59:35" level=error msg="invalid character '\"' after object key:value pair"
time="May 24 15:59:35" level=fatal msg="Redis connection details not set, please ensure that the storage type is set to Redis and that the connection parameters are correct."

For example the tyk gateway config file:

{
"listen_port": 8080,
"node_secret": "352d20ee67be67f6340b4c0605b044b7",
"template_path": "/opt/tyk-gateway/templates",
"tyk_js_path": "/opt/tyk-gateway/js/tyk.js",
"middleware_path": "/opt/tyk-gateway/middleware",
"use_db_app_configs": true,
    "db_app_conf_options": {
        "connection_string": "http://172.18.0.21:3000"
        "node_is_segmented": false,
        "tags": []
},
"app_path": "/opt/tyk-gateway/apps/",
"storage": {
    "type": "redis",
    "host": "172.18.0.16",
    "port": 6379,
    "username": "",
    "password": "",
    "database": 0,
    "optimisation_max_idle": 100
},
"enable_analytics": true,
"analytics_config": {
    "type": "mongo",
    "csv_dir": "/tmp",
    "mongo_url": "",
    "mongo_db_name": "",
    "mongo_collection": "",
    "purge_delay": -1,
    "ignored_ips": []
},
"health_check": {
    "enable_health_checks": true,
    "health_check_value_timeouts": 60
},
"optimisations_use_async_session_write": true,
"enable_non_transactional_rate_limiter": true,
"enable_sentinel_rate_limiter": false,
"allow_master_keys": false,
"policies": {
    "policy_source": "service",
    "policy_connection_string": "http://172.18.0.21:3000",
    "policy_record_name": "tyk_policies"
},
"hash_keys": true,
"close_connections": true
}

Dashboard Config

 {
        "listen_port": 3000,
        "tyk_api_config": {
            "Host": "http://172.18.0.18",
            "Port": "8080",
            "Secret": "352d20ee67be67f6340b4c0605b044b7"
        },
        "mongo_url": "mongodb://mongo:27017/tyk_analytics?replicaSet=rs0",
        "license_key": "LICENCE_GOES_HERE",
        "page_size": 10,
        "admin_secret": "12345",
        "shared_node_secret": "352d20ee67be67f6340b4c0605b044b7",
        "redis_port": 6379,
        "redis_host": "172.18.0.16",
        "redis_password": "",
        "enable_cluster": false,
        "force_api_defaults": false,
        "notify_on_change": true,
        "license_key": "",
        "redis_database": 0,
        "redis_hosts": null,
        "hash_keys": true,
        "email_backend": {
            "enable_email_notifications": false,
            "code": "",
            "settings": null,
            "default_from_email": "",
            "default_from_name": ""
        },
        "hide_listen_path": false,
        "sentry_code": "",
        "sentry_js_code": "",
        "use_sentry": false,
        "enable_master_keys": false,
        "enable_duplicate_slugs": true,
        "show_org_id": true,
        "host_config": {
            "enable_host_names": false,
            "disable_org_slug_prefix": true,
            "hostname": "",
            "override_hostname": "",
            "portal_domains": {},
            "portal_root_path": "/portal"
        },
        "http_server_options": {
            "use_ssl": false,
            "certificates": [
                {
                    "domain_name": "",
                    "cert_file": "",
                    "key_file": ""
                }
            ],
            "min_version": 0
        },
        "ui": {
            "login_page": {},
            "nav": {},
            "uptime": {},
            "portal_section": null,
            "designer": {},
            "dont_show_admin_sockets": false,
            "dont_allow_license_management": false,
            "dont_allow_license_management_view": false
        },
        "home_dir": "/opt/tyk-dashboard",
        "identity_broker": {
            "enabled": false,
            "host": {
                "connection_string": "",
                "secret": ""
            }
        },
        "tagging_options": {
            "tag_all_apis_by_org": false
        }
    }

Your JSON is invalid, that line needs a comma after it.

I’d suggest running your configs through a JSON linter :slight_smile:

You also don’t need the replicaset param there, it could cause problems.