Still a little stuck on implementing an end-to-end CICD story

Support Installation and Deployment
, ,
1

Hey Folks,

In trying to build out a workflow which can be used by multiple teams with multiple repositories. Tyk hybrid is deployed across K8S with a tyk hosted dashboard.

Please help with vetting & fine tuning my instructions to teams here. I’m leaning towards tyk sync over tyk operator because the documentation and tooling seems a little friendlier. (there’s no way I can get away with line one of a doc for 100+ developers that says “step one- compile tyk operator in go”)

(proposed) Developer Workflow

  1. Log in to Tyk Cloud
  2. Click “Deployments → [Environment Name]”
  3. Select Manage APIs
  4. Reload the gateway dashboard page 5 or 6 times until the NGNIX bad gateway & service unavailable errors stop happening (*ideally we fix this at some point as well, super annoying)
  5. Configure your API within the gateway GUI
  6. On the Advanced Options → Segment Tags section add the appropriate keys to deploy to the correct environment (eg: sandbox-external, staging-external, production-external)
  7. save your api
  8. copy and paste the ID of the API configuration you’ve created into your local repository ./gateway-config/tyk-api-ids.txt file. This is a comma separated list of the IDs your team owns.
  9. run the command TYK_API_IDS=$(cat ./gateway-config/tyk-api-ids.txt); docker run -v ${PWD}/gateway-config:/home -it --rm tykio/tyk-sync:v1.2.3 dump -d https://<DASHBOARD-NAME>.cloud-ara.tyk.io/ --secret $TYKGIT_DB_SECRET --target /home --apis $TYK_API_IDS
  10. check the resulting configuration files into your local repo

(proposed) CICD job

  1. add the following code snippet to your ./circleci/config.yaml workflows jobs
  deploy_tyk:
    docker:
      - image: 'tykio/tyk-sync:v1.2.3'
    steps:
      - checkout
      - run:
          # this part of the command is a bit troubling since the tyk-sync container is a little arcane.  I could really use a expert help here
          command: publish --dashboard $TYK_SYNC_DASHBOARD  --secret $TYK_SYNC_DB_SECRET --path ./gateway-config
          name: Send tyk configurations to deployed environment
  1. add the following to your ./circleci/config.yaml workflows
  update_api_definitions:
    when:
      equal:
        - main
        - << pipeline.git.branch >>
      # TODO: update to include a FOLDER limit to the place where tyk configs exist
      # only deploy on change - not blocking
      jobs:
        - deploy_tyk

I’m not married to the tyk-sync requirement, especially because we ARE operating in an environment with K8s and I know that the operator is said to be the preferred way of moving forward on that, but if I’m honest, the operator docs feel a little overwhelming and I’m trying to boil this down to a turnkey solution for dozens to hundreds of developers.

Unfortunately it seems it’s a bit hard to synthesize the correct way to do all this stuff from the various docs that are listed on the topic.

2

Note - after these two - I suppose I’m going to ALSO want a nice method for making sure the API configurations can be tested E2E in a simple and scalable way as well.

If there is any guidance around testing that the TYK gateway can resolve the upstream target and make a simple call through I’d be interested in ways that other people have solved this.

3

Alexander, seems like you should be using Tyk Operator for your use case.

(there’s no way I can get away with line one of a doc for 100+ developers that says “step one- compile tyk operator in go”)

You wouldn’t need to install Tyk Operator except to use snapshot build.

We have a blog in the works to describe a typical CICD workflow. I’ll update here when its ready.

1 Like
4

@Alexander_Chesser - have you seen the docs for the Tyk Operator?

They’re richer than what I can put together in 24 hours. It covers the basics about building APIs, what the CRD is etc. It’s fairly trivial piecing together what needs to happen to integrate it with your devops pipelines.

Can you share some of the gaps you have specifically when working with the tool?

5

Can you share some of the gaps you have specifically when working with the tool?

Overall it just doesn’t seem to flow from a beginning-to-end type scenario and there are gaps in the pre-requisites of things.

A couple examples:

  1. in the tyk operator “how to take a snapshot” guide we’re told to compile out own copy of the operator tyk-operator/pkg/snapshot/README.md at master · TykTechnologies/tyk-operator · GitHub what isn’t mentioned is that the version of go has to be 17. If you build with 19 you get a seg-fault every time you run.
  2. if I get through the “taking a snapshot” part and head over to Publish an API with Tyk Operator and skip to “Step 3: Apply the changes” (I’ve already exported my configurations, should be fine) but when I run kubectl apply -f apifilename.yaml there are errors without much insight or guidance into what I might be missing.
  3. back to the snapshot piece the guidance around ConfigData clearly states the values must be there but without providing context or insights into what values should be set within the values tyk-operator/pkg/snapshot/README.md at master · TykTechnologies/tyk-operator · GitHub

So I guess my general hope for guidance on the docs is that each page I land on either has all of the steps needed to make something happen end-to-end or has the links to the pre-requisite steps.

I don’t want to come off as overly harsh, there are definitely things the docs do help me through, but at the same time there are lots of gaps.

The good news is that I’m also getting some direct help behind the scenes from some excellent support team members :slight_smile:

6

Thank you @Alexander_Chesser! We always appreciate the feedback. This helps all of us so thank you very much! I put a note for me to look at this and improve the flow.

1 Like