Handle404 may occasionally appear in a container in a cluster environment

TYK version: v4.1.0

My tyk production cluster environment uses k8s deployment with 10 Pods deployed. The apps directories are mounted to shared directories by PVC. About 500 apis have been deployed so far. Why request to listen to the path, accidental 404 situation? One pod will appear 404, the rest of the pod will not, Calling /tyk/reload/group will recover and reappear after a whilethe ! How to solve this problem once and for all?principle of tyk is to start loading, where to save the listening path? Memory in the container? Where should I start my investigation?

我的tyk生产集群环境使用了k8s部署,部署了10个pod,apps的目录采用PVC挂载共享目录的方式,目前大概部署了500个API,为什么请求监听路径,偶发性出现404的情况呢?某个pod会出现404,其余pod不会,tyk的原理是启动加载的时候,把监听路径保存在哪里呢?内存里吗?我应该从哪方面开始排查呢?

14:08 request url:/xxx/RMR009 Request Success. http status return 200
14:19 request url:/xxx/RMR009 Request Error. http status return 404 ,Listening path not found .The proxy_muxer.go handle404() method is triggered

My TYK.conf redis configuration:

"storage": {
		"type": "redis",
		"host": "",
		"port": 0,
		"hosts": null,
		"addrs": [
			"xxxxxxx:6379",
			"xxxxxxx:6379",
			"xxxxxxx:6379",
			"xxxxxxx:6379",
			"xxxxxxx:6379",
			"xxxxxxx:6379"
		],
		"master_name": "",
		"sentinel_password": "",
		"username": "",
		"password": "xxxxxxxxxxxxxxxxxxxxx",
		"database": 0,
		"optimisation_max_idle": 100,
		"optimisation_max_active": 0,
		"timeout": 0,
		"enable_cluster": true,
		"use_ssl": false,
		"ssl_insecure_skip_verify": false
	},

Well, typically 404s indicate the requested resource cannot be found. Either the requested details are wrong or they have not been loaded/removed. To get started please follow our troubleshooting guide here

Can you share the gateway logs for pod with the issue? You may first want to follow the link above.

Can you monitor or check that the pod/gateway has loaded all the APIs? The link has a section that mentions this

Considering the gateways use a shared directory, you may want to ensure you only write to one gateway when updating APIs to avoid double-writing or overwriting.

Sorry, I don’t know what this means

The link shared is a very good place to start. Please note that the 404s could be coming from the gateway or even your upstream URL. The gateway logs would indicate

Sorry, the translation may not be accurate. What I mean is, when the TYK program is started, where is the API listening path loaded? Is it loaded into the machine’s memory?