The gateway still accepting connections after receiving SIGTERM

Support Administration and Operation
, ,
1

After receiving a SIGTERM signal, the Gateway still would like to do it’s work and accepting connections. Our sidecar container (gRPC Server) shuts down if there are no ongoing requests present. But the Gateway will try to reach that after the signal was received (and as it says it is in a terminating phase.)

Can the Gateway be configured to finish ongoing requests but not to accept new ones?

time,"@pod","@container","combined_log_message"
"2024-08-24T20:46:53.128946498Z","tyk-gateway-7d6c95d84c-sqpfg","vault-agent-init","2024-08-24T20:46:53.128Z [INFO]  agent.template.server: template server received new token"           
"2024-08-24T20:46:53.164129947Z","tyk-gateway-7d6c95d84c-sqpfg","vault-agent-init","2024-08-24T20:46:53.164Z [INFO]  agent: (runner) received finish"                                     
"2024-08-24T20:46:53.164146827Z","tyk-gateway-7d6c95d84c-sqpfg","vault-agent-init","2024-08-24T20:46:53.164Z [INFO]  agent.auth.handler: shutdown triggered, stopping lifetime watcher"   
"2024-08-24T20:47:02.423487371Z","tyk-gateway-7d6c95d84c-sqpfg","vault-agent","2024-08-24T20:47:02.423Z [INFO]  agent.template.server: template server received new token"                
"2024-08-24T20:47:37.654871124Z","tyk-gateway-7d6c95d84c-sqpfg","vault-agent","==> Vault Agent shutdown triggered"                                                                        
"2024-08-24T20:47:37.654893385Z","tyk-gateway-7d6c95d84c-sqpfg","tyk-gw","2024/08/24 20:47:37 terminated"                                                                                 
"2024-08-24T20:47:37.654922035Z","tyk-gateway-7d6c95d84c-sqpfg","tyk-gw","time=""Aug 24 20:47:37"" level=info msg=""Stop signal received."" prefix=main"                                  
"2024-08-24T20:47:37.654991406Z","tyk-gateway-7d6c95d84c-sqpfg","tyk-gw","time=""Aug 24 20:47:37"" level=info msg=Terminating. prefix=main"                                               
"2024-08-24T20:47:37.655608658Z","tyk-gateway-7d6c95d84c-sqpfg","vault-agent","2024-08-24T20:47:37.654Z [INFO]  agent.auth.handler: shutdown triggered, stopping lifetime watcher"        
"2024-08-24T20:47:37.655634598Z","tyk-gateway-7d6c95d84c-sqpfg","vault-agent","2024-08-24T20:47:37.654Z [INFO]  agent: (runner) received finish"                                          
"2024-08-24T20:47:37.655980905Z","tyk-gateway-7d6c95d84c-sqpfg","grpc-server","Received a SIGTERM signal."                                                                                
"2024-08-24T20:47:37.711571698Z","tyk-gateway-7d6c95d84c-sqpfg","grpc-server","Server is down, clean."                                                                                    
"2024-08-24T20:47:37.713438193Z","tyk-gateway-7d6c95d84c-sqpfg","tyk-gw","time=""Aug 24 20:47:37"" level=debug msg=""Couldn't dispatch request object"" error=""rpc error: code = Unavaila
ble desc = connection error: desc = \""transport: Error while dialing: dial tcp 127.0.0.1:5555: connect: connection refused\"""""                                                                     
"2024-08-24T20:47:37.714908Z","tyk-gateway-7d6c95d84c-sqpfg","tyk-gw","time=""Aug 24 20:47:37"" level=debug msg=""Couldn't dispatch request object"" error=""rpc error: code = Unavailable
 desc = connection error: desc = \""transport: Error while dialing: dial tcp 127.0.0.1:5555: connect: connection refused\"""""
2

@sztojka Hello and welcome to the community :partying_face:

No, but there is a very good suggestion from Pete on this topic. Hope it helps.

3

Thanks, I have read that before. It makes sense in a normal environment. :slight_smile:

When a pod is going worn it is not always our deliberate action. The ingress will realize much later, if a pod running the Tyk GTW (and the gRPC sidecar) is missing. So Tyk GTW should close the listening socket(s) after the SIGTERM. The only way to not have conection issues would be to kepp the gRPC Service alive as long as it can be. Eventually it will die out by a SIGKILL.