Tyk CE helm chart persistence is confusing and not functional

Sean_Houghton · October 25, 2021, 4:10pm

I’ve opened up a PR which fixes the problems with persistence in the headless chart. However, it still behaves in a way that seems unusable so I’m assuming something is misconfigured or I have a misunderstanding about how it’s supposed to be used.

When creating APIs using the REST API (POST to /tyk/apis) the resulting api object is written to the the app_path as a json file. However, using the list apis endpoint (GET to /tyk/apis) won’t show that API until you send a SIGUSR2 signal to the gateway to request a config reload. However, sending that signal causes the pod to restart which creates a service outage. Additionally, the restart is considered a crash by Kubernetes and causes a CrashLoopBackoff state further extending the outage.

Every part of Tyk headless seems to have major usability problems with state persistence and state consistency. Am I missing something here?

Thanks!

sedky · October 25, 2021, 4:38pm

Sean,

instead of firing off a SIGUSR2 signal to restart the Gateway, you can use the Hot Reload API to refresh the APIs cache whenever you make a change to a Policy or API.

That will get Tyk to respond with the updated configurations on subsequent GET requests.

https://tyk.io/docs/tyk-gateway-api/

I will take a look at your PR, thanks so much for your contribution!

Sean_Houghton · October 25, 2021, 7:16pm

Thanks for the reply. Does this mean I have to call the hot-reload API even after using the API to make a change? How does this work with the operator which is supposed to automatically sync state with Ingress definitions? I don’t see any hot reload API calls in the operator source code:

github.com

TykTechnologies/tyk-operator/blob/master/pkg/client/gateway/api.go#L39


      
          	}
          
          
	return &spec, nil
          }
          
          
func (a Api) Create(ctx context.Context, def *model.APIDefinitionSpec) (*model.Result, error) {
          	return a.createOrUpdate(ctx, def)
          }
          
          
func (a Api) createOrUpdate(ctx context.Context, def *model.APIDefinitionSpec) (*model.Result, error) {
          	var o model.Result
          
          
	err := client.Data(&o)(client.PostJSON(ctx, endpointAPIs, def))
          	if err != nil {
          		return nil, err
          	}
          
          
	return &o, nil
          }
          
          
func (a Api) Update(ctx context.Context, def *model.APIDefinitionSpec) (*model.Result, error) {

sedky · October 25, 2021, 7:26pm

Great observations!

The Gateway in OSS mode does rely on Hot Reload, here’s the Operator’s code which invokes a Hot Reload after an API call.

The Dashboard (Tyk Pro) setup doesn’t use Hot Reload because it has its own synchronization strategy. That’s why you wouldn’t see that method implemented in the Operator code for Tyk pro.

The CRUD methods in the Operator do in fact call the Hot Reload mechanism, this is invoked in a few layers above the code snippet you linked which is lower level.

Sean_Houghton · October 25, 2021, 8:11pm

Thanks for the clarification. My intuition is that new users will expect read-after-write consistency with regular CRUD calls so I would suggest making it very clear in API documentation that a hot reload call is required.