Mutual TLS in Tyk open source

sailor · May 24, 2022, 6:11am

I am using Tyk gateway v4.0.0 with docker.
For testing I have set up a mock API behind Tyk gateway.
I was trying out client side mutual TLS(static) on a keyless API.

So far I am only getting the following error:

“error”: “TLS not enabled”

Steps I have followed:

Created the following API definition:

{
    "name": "Mutual TLS auth mock api",
    "api_id": "mutual-tls-auth-mock-api-id",
    "org_id": "basic",
    "auth": {
        "use_certificate": true
    },
    "use_keyless": true,
    "use_mutual_tls_auth": true,
    "client_certificates": null,
    "base_identity_provided_by": "auth_token",
    "certificates": null,
    "version_data": {
        "not_versioned": true,
        "versions": {
            "Default": {
                "name": "Default",
                "use_extended_paths": true,
                "extended_paths": {
                    "ignored": [],
                    "white_list": [],
                    "black_list": []
                }
            }
        }
    },
    "proxy": {
        "listen_path": "/m-tls/",
        "target_url": "http://localhost:7070",
        "strip_listen_path": true
    }
}

Generated a self-signed key pair to use with:

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes

I entered some random text for all the questions. After that I got the key.pem and cert.pem files.

Called the API with curl:

curl -k --cert cert.pem --key key.pem http://localhost:8080/m-tls/mock_details

Result:

{
    "error": "TLS not enabled"
}

with the status code 403.

May I know what am I doing wrong and how to get it right?

Olu · May 24, 2022, 9:26am

Your endpoints are using unsecured connections on HTTP

sailor · May 24, 2022, 9:59am

curl -k --cert cert.pem --key key.pem https://localhost:8080/m-tls/mock_details

Is it what you meant?

On trying it I was getting :

error:1408F10B:SSL routines:ssl3_get_record:wrong version number

curl in verbose mode includes this:

* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

Or did you meant the servers(mock api in my case) behind Tyk have to be https like “https://localhost:7070”

Olu · May 24, 2022, 10:15am

I am unfamiliar with the error and a quick google shows it could be any of the reasons listed here

curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number - Stack Overflow.
How to solve "error:1408F10B:SSL routines:ssl3_get_record:wrong version number"? - Stack Overflow.
Unable to establish SSL connection: wrong version number - Help - Let's Encrypt Community Support

An internal search shows the error happening when an SSL and any other non SSL processes are running on the same port. So you may need to check that.

But I am afraid this isn’t a Tyk error.

sailor · May 24, 2022, 11:05am

From the second link you shared:

It says:

It is very likely that the server does not speak TLS at all.
The client will start with the TLS handshake and the server will reply to this with some non-TLS response. The client expect the server to do its part of the TLS handshake though. Thus it will try to interpret the servers as response as TLS. This will lead to strange error messages depending on the TLS stack used by the client.

I also saw it in when I googled it too.

When performing client mTLS, from what I understood I believe Tyk is the server here. But the strange thing is it doesn’t produce any logs when that endpoint is hit.

Also can you please verify if my implementation is correct or not? I could not find any documentation for implementing it using Tyk OPEN SOURCE gateway other than a mention it supports it. I tried to implement it by comparing with the procedure for Tyk dashboard.

ahmet · May 24, 2022, 11:17am

could you share the output of the following please?

curl -k -v --cert cert.pem --key key.pem --tlsv1.3 https://localhost:8080/m-tls/mock_details

And also any logs from Tyk Gateway (if any)

sailor · May 24, 2022, 11:40am

@ahmet

curl -k -v --cert cert.pem --key key.pem --tlsv1.3 https://localhost:8080/m-tls/mock_details

Trying 127.0.0.1:8080…
TCP_NODELAY set
Connected to localhost (127.0.0.1) port 8080 (#0)
ALPN, offering h2
ALPN, offering http/1.1
successfully set certificate verify locations:
CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
TLSv1.3 (OUT), TLS handshake, Client hello (1):
error:1408F10B:SSL routines:ssl3_get_record:wrong version number
Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

There weren’t any logs produced.

ahmet · May 24, 2022, 11:51am

Could you modify your API Definition to include in the root:

“protocol”: “https”,
“listen_port”: 8443

I think that the Gateway is still listening on HTTP, rather than HTTPS.

{
“name”: “Mutual TLS auth mock api”,
“api_id”: “mutual-tls-auth-mock-api-id”,
“org_id”: “basic”,
“auth”: {
“use_certificate”: true
},
“use_keyless”: true,
“use_mutual_tls_auth”: true,
“client_certificates”: null,
“base_identity_provided_by”: “auth_token”,
“certificates”: null,
“protocol”: “https”,
“listen_port”: 8443,
“version_data”: {
“not_versioned”: true,
“versions”: {
“Default”: {
“name”: “Default”,
“use_extended_paths”: true,
“extended_paths”: {
“ignored”: ,
“white_list”: ,
“black_list”:
}
}
}
},
“proxy”: {
“listen_path”: “/m-tls/”,
“target_url”: “http://localhost:7070”,
“strip_listen_path”: true
}
}

sailor · May 25, 2022, 4:05am

Adding it in the API definition didn’t work.
Upon trying:

curl -k -v --cert cert.pem --key key.pem --tlsv1.3 https://localhost:8080/m-tls/mock_details

I got the same result. “ssl3_get_record:wrong version number”

On trying:

curl -k -v --cert cert.pem --key key.pem --tlsv1.3 https://localhost:8443/m-tls/mock_details

I got Connection refused.

So I added

“protocol”: “https”,
“listen_port”: 8443

in the Tyk gateway configuration also and tried:

curl -k -v --cert cert.pem --key key.pem --tlsv1.3 https://localhost:8443/m-tls/mock_details

Then also it didn’t make any difference and I got the same results as before. “ssl3_get_record:wrong version number”

sailor · May 26, 2022, 6:05am

Adding to it. I think it is not working well.
I found the following error in the gateway logs on adding that two fields to API definition.

error “https:8443 trying to open disabled port”

8553 also did not work either. Is there any additional requirements to be done for that?

Olu · May 26, 2022, 9:12am

The error means the port is not opened from your device or machine. Or you haven’t enabled port whitelisting or disabled port whitelisting in your gateway config file?

Can you share your gateway config and redact any sensitive information (i.e. the secret).

sailor · May 26, 2022, 9:53am

Here is the config:

{
  "protocol": "https",
  "listen_port": 8443,
  "secret": "xxx",
  "template_path": "/opt/tyk-gateway/templates",
  "tyk_js_path": "/opt/tyk-gateway/js/tyk.js",
  "middleware_path": "/opt/tyk-gateway/middleware",
  "use_db_app_configs": false,
  "app_path": "/opt/tyk-gateway/apps/",
  "storage": {
    "type": "redis",
    "enable_cluster": true,
    "addrs": [ "xxx"  ],
    "username": "xxx",
    "password": "xxxx",
    "database": 0,
    "optimisation_max_idle": 20000,
    "optimisation_max_active": 40000
  },
  "enable_analytics": true,
  "analytics_config": { "xxx" },
  "health_check": {
    "enable_health_checks": true,
    "health_check_value_timeouts": 60
  },
  "optimisations_use_async_session_write": false,
  "enable_non_transactional_rate_limiter": true,
  "enable_sentinel_rate_limiter": false,
  "enable_redis_rolling_limiter": false,
  "allow_master_keys": false,
  "policies": {
    "policy_source": "file",
    "policy_record_name": "/opt/tyk-gateway/policies/policies.json"
  },
  "hash_keys": false,
  "enable_hashed_keys_listing": true,
  "close_connections": false,
  "http_server_options": {
    "enable_websockets": true
  },
  "allow_insecure_configs": true,
  "coprocess_options": {
    "enable_coprocess": true,
    "coprocess_grpc_server": ""
  },
  "enable_bundle_downloader": true,
  "bundle_base_url": "",
  "global_session_lifetime": 100,
  "force_global_session_lifetime": false,
  "max_idle_connections_per_host": 500,
  "enable_jsvm": true,
  "log_level": "debug",
  "use_graylog": true,
  "graylog_network_addr": "xxx",
  "tracing": { "xxx" }
}

Olu · May 27, 2022, 3:37pm

Your config file is missing TLS and SSL settings. You would need to set this up and confirm your gateway is working on HTTPS before proceeding with mTLS. You can use a keyless API as a test.

After confirmation, proceed to configuring your API definition using Static mTLS. I see you have already enabled keyless and mutual tls in your API definition. Everything looks fine however you are missing an entry for the client_certificates. You can find out how to add the ID of the client_certificates here.

Considering you have an org_id, the final certificate ID would be a concatination of your org_id + the generated SHA256 fingerprint. For example basicc3a4b867915e56b23f4f945112dd22d15eda22715135ba8f7aa28734e1a8c514. If no org_id is specified in the API definition then the final client_certificate is c3a4b867915e56b23f4f945112dd22d15eda22715135ba8f7aa28734e1a8c514.

Hope this helps