Ip-*-*-*-*.eu-west-1.compute.internal tyk[20069]: 2020/05/04 14:07:56 http: TLS handshake error from *.*.*.*:56836: tls: no certificates configured

Hi,

I am trying to configure SSL certifictae on TYK gateway hosted on AWS behind a load balancer.
i have added certificate to certificate storage and used “ssl_certificates” option to add it in tyk.conf and enabled ssl to true and http2 under http_server_options. However in the status logs it is giving me an tls handshake error and no certificates configured. Please below tyk.conf snippet

“http_server_options”: {
“enable_websockets”: true,
“enable_http2”: true,
“use_ssl”: true,
“use_ssl_le”: true,
“ssl_certificates”: ["5e25979a34f4350001e00152ade139d57080f599af4fc52139e4c***************************]

i am also not able to access the api’s giving 502 “BAD Gateway”

May 07 10:38:22 ip-....eu-west-1.compute.internal tyk[23888]: 2020/05/07 10:38:22 http: TLS handshake error from ...:20116: tls: no certificates configured
May 07 10:38:46 ip-....eu-west-1.compute.internal tyk[23888]: 2020/05/07 10:38:46 http: TLS handshake error from ...:16188: tls: no certificates configured
May 07 10:38:52 ip-....eu-west-1.compute.internal tyk[23888]: 2020/05/07 10:38:52 http: TLS handshake error from ...:20132: tls: no certificates configured
May 07 10:39:16 ip-....eu-west-1.compute.internal tyk[23888]: 2020/05/07 10:39:16 http: TLS handshake error from ...:16200: tls: no certificates configured
May 07 10:39:22 ip-....eu-west-1.compute.internal tyk[23888]: 2020/05/07 10:39:22 http: TLS handshake error from ...:20150: tls: no certificates configured
May 07 10:39:46 ip-....eu-west-1.compute.internal tyk[23888]: 2020/05/07 10:39:46 http: TLS handshake error from ...:16218: tls: no certificates configured
May 07 10:39:52 ip-....eu-west-1.compute.internal tyk[23888]: 2020/05/07 10:39:52 http: TLS handshake error from ...:20164: tls: no certificates configured

openssl s_client -connect localhost:8080 -state
CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL3 alert read:fatal:internal error
SSL_connect:error in SSLv2/v3 read server hello A
139636313372576:error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error:s23_clnt.c:769:

no peer certificate available

No client certificate CA names sent

SSL handshake has read 7 bytes and written 289 bytes

New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1588848292
Timeout : 300 (sec)
Verify return code: 0 (ok)

How have you added the certificate to the store? Was it through the Dashboard ? Here’s what you should see in the Dashboard

This is the ID you need to add to the tyk.conf.

Also, is your AWS LB setup as a NLB or ALB? It needs to be an NLB in order to terminate the SSL connection at the Tyk Gateway

Hi Sedky,

Yes I have added certificate to the dashboard.

and then updated it in the tyk.conf and restated the gateway service.

Have you added both the private + pub key into the file you are uploading?

Yes I have combined end user certificate and private key into a file and uploaded it to the dashboard.

Is there any format in bundling certificate and private key. I have received certificate and it has end user certificate , intermediate certificate and root certificate.

Here’s what a bundled example looks like:

-----BEGIN CERTIFICATE-----
MIIFejCCA2ICCQCLHZYLfT1miDANBgkqhkiG9w0BAQsFADB/MQswCQYDVQQGEwJD
QTELMAkGA1UECAwCT04xDzANBgNVBAcMBkxvbmRvbjEMMAoGA1UECgwDVHlrMQ4w
DAYDVQQLDAVUeWtJbzEZMBcGA1UEAwwQd3d3LnR5ay10ZXN0LmNvbTEZMBcGCSqG
SIb3DQEJARYKc2VkQHR5ay5pbzAeFw0xOTEyMjQxNzU0MDVaFw0yMDEyMjMxNzU0
MDVaMH8xCzAJBgNVBAYTAkNBMQswCQYDVQQIDAJPTjEPMA0GA1UEBwwGTG9uZG9u
MQwwCgYDVQQKDANUeWsxDjAMBgNVBAsMBVR5a0lvMRkwFwYDVQQDDBB3d3cudHlr
LXRlc3QuY29tMRkwFwYJKoZIhvcNAQkBFgpzZWRAdHlrLmlvMIICIjANBgkqhkiG
9w0BAQEFAAOCAg8AMIICCgKCAgEAryNN9qXAf/gAgIDe9SC11rmCF2WLh1nfs08Y
9rle+aIRKYLYU+ilo1Lhs3z8/9hcwlTdfNjdHonK2j9lNTWAhpNdpdGIJa9S+SDg
e9sg6MNolhtUXzKlfbYdpRsuGn9rLX1MMuWLbvbZZoFhz1I0Uanz9/VxPZvp6MJL
O7ljpIEjgImvxiiN4OLPef1OW2yjAAMYEwhw4dM7bdClZbK1Jo3I1/Vv7YvQsFK2
rQh2XbHW0pezyCEhoKOfDNQCV5cJCWy9n7xgrA5vfPJVLMWY34YUgA1Z6r5JzXQ4
Cq7yq3dnITr1WNt3Cd629hoizx6z4BTHGWt6nSrA7RQpgSwsUeT0vtW1kE+1/z2C
JduVxkiYoN8XCJyiFY5etiQFq19yk6+rpJSli/ZSud89QfO3Wm3SGxa5tdOjxA9p
CYVtwXasoOnsJm3iuKYlnstAfd69XoMP6rVN3LqXJ3m1KYpqLC1R7aUbuaTVYCZJ
RbG0xO/Y9mBu800Sij0wYpM1/z91ChC7+T9rJlgSjXBiHmhnyQCVQr5IOp7u3zu1
ztEtidcRzmpLjXCMxC2EQ5hZhthmhHHc/yfW7+ylwtU/VL+UcBXJKeSYJaplzzQ+
VfYJFM1lQSK5b8SgNnkGYdsnJy166sNZ/TagnKChCNJVLWglEzUDauYSijfu6XAC
/hiRArMCAwEAATANBgkqhkiG9w0BAQsFAAOCAgEABik92GJVPCt8Kq9f3JeK8FSZ
Thtlxzn2DfTUmdijes4HiUtiMHLAHdIivB4E3SxUF2lxTO+pLSdX+vXtUBHyEYD6
GE2rMX7li9LQmG7mSaO2yIq/HdQ/EWr7jVnYCEqOtrKpRepmeJR6JETea1Q2p/oC
YiWmJAb9NCGuEMDjdsSpEtTaAExpw8y2ieFoIcOmUjHxOH2OD3OrBJM6oYqb085v
vBtdrNxb6D7uRZy3gIi7JSd1AszDHI/+Lk29ggSfWgBSbxtxJKqZqF1DcdWjJJM1
M/wVzrd4KzkH1pV1UWp7BGE2yvx3Xue2X9BI+MzylGf6aldyaLUtNcTK++VR54Iz
9X6YT7pfmIlZIOPzFSg9ujggVqbsQBMnGcGvcW/SKT0m/4iSPlHonyFRN5xyHVWB
8MnJ4nzcPPWi0QOhKXQpnq44KB7iGWybz0V8IDS/sPoIQxYJtl1WY/KvnKNASiog
cYS1ABge15RhcUFXuzQp2ZfA7tepGmmCPTdMUcw/3bQ+rdskZkJHytCHyrvOW5uh
bFxFIVDRYi/ffMSV8BPxmPn/u0xlIYnQZzTb38ZYaJtpywS2z5CjrUUjX8cp2oN4
nMd+Qef691/a3/S6Ct5WO2xuugGp2YI9B5vfMhsFyXi9XoEhb73QC/pJC+ZhqIZQ
iarNB3Ks+aIKX/DeSXc=
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
MIIJRAIBADANBgkqhkiG9w0BAQEFAASCCS4wggkqAgEAAoICAQCvI032pcB/+ACA
gN71ILXWuYIXZYuHWd+zTxj2uV75ohEpgthT6KWjUuGzfPz/2FzCVN182N0eicra
P2U1NYCGk12l0Yglr1L5IOB72yDow2iWG1RfMqV9th2lGy4af2stfUwy5Ytu9tlm
gWHPUjRRqfP39XE9m+nowks7uWOkgSOAia/GKI3g4s95/U5bbKMAAxgTCHDh0ztt
0KVlsrUmjcjX9W/ti9CwUratCHZdsdbSl7PIISGgo58M1AJXlwkJbL2fvGCsDm98
8lUsxZjfhhSADVnqvknNdDgKrvKrd2chOvVY23cJ3rb2GiLPHrPgFMcZa3qdKsDt
FCmBLCxR5PS+1bWQT7X/PYIl25XGSJig3xcInKIVjl62JAWrX3KTr6uklKWL9lK5
3z1B87dabdIbFrm106PED2kJhW3Bdqyg6ewmbeK4piWey0B93r1egw/qtU3cupcn
ebUpimosLVHtpRu5pNVgJklFsbTE79j2YG7zTRKKPTBikzX/P3UKELv5P2smWBKN
cGIeaGfJAJVCvkg6nu7fO7XO0S2J1xHOakuNcIzELYRDmFmG2GaEcdz/J9bv7KXC
1T9Uv5RwFckp5JglqmXPND5V9gkUzWVBIrlvxKA2eQZh2ycnLXrqw1n9NqCcoKEI
0lUtaCUTNQNq5hKKN+7pcAL+GJECswIDAQABAoICAQCpA0wAo3KCzEVHvZXsPxXc
UwedJoyhIPVFaCayKq/h/inLwo6bYx893wUanprVF+pkS1u3WN0C1HzjY1x7Hdeb
h2lPVUaq/2f2LWkVfaFjnvg2K8efWxPjnjU7mWBDiAVZT9qSH30Qp72jM2a9mRoO
WRGsj7spdmYkDiRmTYFARrOJHtZ3R7mrdZY9plrQ1aTzpQi9jV7KP3vRfRGwc+Zn
f1p8Nko+lrfdAOVzVIBN6QpkDqwbjQWTrXMXAR4FgybgZoUUOsmcIQ4sKPxTiZj+
1LjB3qCQbfRxBh/zB2P+XusyYWBV0O0upDQyiYRgFsQFkmeVO2iHdbPa1qM7GfLl
ts7J3MLvbASCeqSOuvk/7OCyC+L5vUs6SYpYlAj8lTOh5YstpYfKnZ6hcfu31jE1
0no/wK7wa9j8pTg5icTi+zWMZhjyXVaQBYUvC/CDrp3NPka2a9Po3zQ6pXIMrGjj
ULFgDf5wwCQDdTHc9S8XyhQiTMHCNf7tdaamc9MM4h+tkrKqV32rhb1H49QYhJAg
bFEk64pBnipKgmCGjHvoeDR/AC51YXtc+/xOtz3oAn4CU0B/pGCrgG//zCsI/Rgl
gPoYyVxysBhBYbijiNovwTKa5I8DWzxmAXQJcrPxYB9TVwVLYHraF4iFOzZI8IAU
ptQ5sxOT+Et4AFu30C4ROQKCAQEA59F5vLXqesA0lx7pCEpTL9r+/ww/7FBbalJo
yudlyb8G77zj9HmbwoMqzI6/MCtxPZwC7yL4e3MDi4szpLcruEp8dvOIjqaRIv94
1HTyvA/hh7RhkYGK/2w6GXHq3llYk4QBlZEtgzgje9BhjcxLQhS0Z4gc/UuJ7xsj
rnN68bBqfd4sOJIRwyIqwGk+oLA6Qa6yeBMf3TYkS6UM0jKLeWOzB/GuH/wy4cvj
iVZlbz0pFf+53aydo7tHs2yUBO7K1ruRC92zARes5nVPiNCkoZAPGDIWltJxc8Rr
nEchiBArQX8quYi4+Lp1jkv6YKN44oegKKT3cQrN1i4++asQtwKCAQEAwWg5hNPV
jQ5gKT5BfowLRGvrshiUh0FEQEIg70V1qtBFsd7U9W15jp31HUH+0M1/HT5GIA6l
Tkadr/q9zktTGNo9oKcVw+abk8GQPcUnVFyjlb8+w6eUPDavwzHqeRcSx6BNCg3z
V7Fv1z0yyKjSC2ZGhsRc0sxmJ/5DGvo4FoWnWRBFqk/yU/xMrehqIjbKZUbvfaL5
3O1JnDQ6gXAGw7pkHIaHZMzq6mYiXVmRt9NRkPetueoTdZGWXmcOyQ9/9kOXSU61
bkwdxT0EM9MRYSLb+zI6ghMs6BsH8YRWDOgzwuQ3Jy/dp9E3WsUiwyTKlHExW3io
8mw+n2wQHNBp5QKCAQEAr1t3EjguOlLAtXwtXRcUHEQ11mrGxBT5QdE68hnelqEJ
NzqU2W9QUvBz5K8qkt6Z17f3RDwY6RFUoi9Xjob87Hhz9Gs+ZmI7sdS860HYCEif
Gs9pau8qXRLaVDsnJrFBXr1hkWlzmypRNSypTuE2ILOW0CZXf4evvI60DBfQ7Zi8
xZ/VpalhBFIJMump9VW5i8wBtvEIjoaRKCOfMjYH6XhwilbYl9toBgP8h/wx2NbU
CI8cY0JbMOrPTzuYMK8OAyd5jhdTfEUThnVBB8aEcq6NrIjfK5HfPmpb2Ki6Bbnt
bqsqf5ScD5LEsDJrIMUoAXwEvBomD33TvOvYuXYcuwKCAQAxe/k7gvegn4z7AHLB
bhNz7ijf6GoYMRrFViYXEi6JBcDPQbQ7F0eCteeBFOGdivZ7BPPsP9MLtAWkgQMP
RTZ9pVkfQaDP/rDPnTbKlyb47rrsFYJ5jObggR13U84qLT0g1mLfr7ITPdLfv0i/
DnKvd/uTf6pARQveL6/7twmiuY+XuhIZAVt/IY588Ho3Rs2CHCQqgQhs/klXLgXK
GF+itI3ovoLlx4hTcM5aFBC0d9zmung4mSthaV6PyM5/oveairCqkPjav57eWD47
BEXuFH00SCWwIjfbKw9dO/7ITqDvvHRgEl8kEOOY1dePFlq+oDeAr96KKCfdokh9
FtYBAoIBAQCL0tPaYAHeUtilSMyEiEtXFTM6iG8/NIn5M3x5yB99bQ/Sfz6cpXa5
xpwCevEn0u4cW7bUbovfwyiOVpX+TxfsTtkWe7UDEE3MKvJArM5NQZMSxn84bwkV
aaDdAWU1e+z3eQ76zQ8qj5EtNP6UQFN7A9Ct0T+IR7kW1QjzhhJwVYQkiNVjV1aG
BY6THGcZhxRpNCtHsgIShpScEeUVp19cmdTBWh8L2lEj401dp+8fmfXXfqhZ4s/V
PzK9BR7hNtUDJynvGFkkeehVRhSg/HXg2tQF4fFTx6Owfg5lHWCZm5K4g5Wk4pf3
Iv6CbB2EGPocqLEyxqeOzRb3YRnTgqa1
-----END PRIVATE KEY-----

So if I used the command to generate a self signed cert:

$ openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes
$ cat key.pem cert.pem > keyandcert.pem

I would upload “keyandcert.pem” into the Dashboard to use as my Gateway’s server cert

Yeah even i did the same and certificate also shows my domain name.

image

I Tried uploading certificate to server and updated config as well but no luck same tls handshake error : no cert configured

image

How are you mounting the cert on the file system? If you log onto the server, can you see the dev-api.cert and dev-api.key available in the correct location ie next to the tyk binary?

Yes the location is correct and it’s in the tyk binary folder.

Thank-you that’s helpful. I need a few more things so I can further isolate the issue.

Can you query the https://tyk-gw-host/hello endpoint? I want to make sure this is a Tyk conf issue, not an API level issue.

Please post the startup logs of the Gateway as well as the error logs from trying to access the /hello endpoint.

where i should run this query? i tried on server and browser

image

Thank-you Chandra.

Get me these last 4 things please.

  1. tyk.conf with secrets redacted
  2. gateway logs on startup
  3. pass “-v” flag into your curl from server for verbose logging and return that to me, ie “curl -v https://localhost:8080/hello
  4. show me gateway logs at the time of the error please

gateway logs are not getting generated in /var/logs/

image

{
“listen_port”: 8080,
“pid_file_location”: “/var/run/tyk/tyk-gateway.pid”,
“secret”: “”,
“template_path”: “./templates”,
“tyk_js_path”: “./js/tyk.js”,
“middleware_path”: “./middleware”,
“use_db_app_configs”: false,
“app_path”: “./apps/”,
“storage”: {
“type”: “redis”,
“host”: “localhost”,
“port”: 6379,
“username”: “”,
“password”: “”,
“database”: 0,
“optimisation_max_idle”: 500
},
“enable_analytics”: true,
“analytics_config”: {
“type”: “rpc”,
“csv_dir”: “/tmp”,
“mongo_url”: “localhost”,
“mongo_db_name”: “tyk_analytics”,
“mongo_collection”: “tyk_analytics”,
“purge_delay”: 10,
“ignored_ips”: ,
“enable_geo_ip”: true,
“geo_ip_db_path”: “/opt/tyk-gateway/GeoLite2-City.mmdb”,
“normalise_urls”: {
“enabled”: true,
“normalise_uuids”: true,
“normalise_numbers”: true,
“custom_patterns”:
}
},
“health_check”: {
“enable_health_checks”: true,
“health_check_value_timeouts”: 60,
“health_check_endpoint_name”: “/health”
},
“optimisations_use_async_session_write”: true,
“allow_master_keys”: false,
“policies”: {
“policy_source”: “service”,
“policy_connection_string”: “https://tyk-dashboard:3000
},
“db_app_conf_options”: {
“connection_string”: “https://tyk-dashboard:3000
},
“hash_keys”: true,
“suppress_redis_signal_reload”: false,
“use_sentry”: false,
“sentry_code”: “”,
“enforce_org_data_age”: true,
“http_server_options”: {
“enable_websockets”: true,
“enable_http2”: true,
“use_ssl”: true,
“use_ssl_le”: true,
“certificates”: [
{
“domain_name”: “*.com”,
“cert_file”: “./dev-api.cert”,
“key_file”: “./dev-api.key”
}
]
},
“monitor”: {
“enable_trigger_monitors”: true,
“configuration”: {
“method”: “POST”,
“target_path”: “http://cloud.tyk.io/1337/tyk/webhook”,
“template_path”: “templates/monitor_template.json”,
“header_map”: {“x-tyk-monitor-secret”: “sjdkfhjKHKJHkjsdhsufdudfhjHKIHJ1”},
“event_timeout”: 10
},
“global_trigger_limit”: 80.0,
“monitor_user_keys”: false,
“monitor_org_keys”: true
},
“slave_options”: {
“use_rpc”: true,
“rpc_key”: “”,
“api_key”: “”,
“connection_string”: “hybrid.cloud.tyk.io:9091”,
“use_ssl”: true,
“rpc_pool_size”: 20,
“enable_rpc_cache”: true,
“bind_to_slugs”: true
},
“local_session_cache”: {
“disable_cached_session_state”: false,
“cached_session_timeout”: 5,
“cached_session_eviction”: 10
},
“enforce_org_quotas”: false,
“experimental_process_org_off_thread”: true,
“enable_non_transactional_rate_limiter”: true,
“enable_sentinel_rate_limiter”: false,
“auth_override”: {
“force_auth_provider”: true,
“auth_provider”: {
“name”: “”,
“storage_engine”: “rpc”,
“meta”: {}
}
},
“enable_context_vars”: true,
“hostname”: “”,
“enable_api_segregation”: false,
“control_api_hostname”: “”,
“enable_custom_domains”: true,
“enable_jsvm”: true,
“coprocess_options”: {
“enable_coprocess”: false
},
“hide_generator_header”: false,
“event_handlers”: {
“events”: {}
},
“allow_insecure_configs”: true,
“public_key_path”: “”,
“close_idle_connections”: false,
“allow_remote_config”: false,
“enable_bundle_downloader”: true,
“service_discovery”: {
“default_cache_timeout”: 20
},
“close_connections”: true,
“max_idle_connections_per_host”: 100,
“disable_dashboard_zeroconf”: true
}

I see some interesting logs in the gateway, can you please get me the full thing from the

TYK GATEWAY VERSION 2.9.4
to
API RELOAD COMPLETE

The rest of the logs are of tls handshake errors

Chandra, is the AWS LB an ALB or an NLB? IT needs to be a network load balancer if you want to terminate traffic at the Gateway

It’s an ALB, but even if I do curl request on local host is giving error.

Regarding the ALB ssl termination it does not validate the certificate but still enable https connection to its target groups just checking whether certificate is present or not and this where i was getting the error.