Procal on EKS

A later DevOps Institute cycle on the same Procal cost-estimate platform (see the earlier Terraform + Atlantis case study), this time moving the three backend services onto Kubernetes. Built and operated by a small cohort team, the path deliberately went through a self-managed cluster before EKS: local Kubernetes with kind and minikube, then a bare-metal cluster assembled by hand with kubeadm on Mac minis, so the control plane EKS now manages was never a black box to begin with. The real focus of this cycle was what came after the cluster existed: getting to a state where Git, not a person running kubectl, is what the cluster obeys.

Self-managed Kubernetes, then EKS

Production topology

One public door: the NLB is the only thing the internet reaches. Everything behind the gateway, including the ArgoCD UI and Grafana, rides that same load balancer. boots isn't routed at all, it only consumes from SQS. AWS-managed services (Secrets Manager, SQS, ECR) are reached by every pod through Pod Identity, no access keys stored anywhere.

Full AWS architecture

Detailed diagram (2964×2804px) — scroll inside the box to read the small text.

Getting here

1. 1

   Containers

   Docker images, one process each. Build once, run anywhere.

2. 2

   Local Kubernetes

   kind and minikube. Pods, Services, Deployments on a laptop.

3. 3

   Self-managed, on-prem

   kubeadm on Mac minis. Assembled etcd, the API server, scheduler, and controller manager by hand, along with networking, storage, and upgrades.

4. 4

   EKS

   A managed control plane. IAM, VPC design, and load balancers became the real work instead of babysitting etcd.

5. 5

   Helm

   Templating plus a package manager. One chart, many environments.

6. 6

   ArgoCD & Argo Rollouts

   Git became the source of truth; the cluster continuously reconciles itself to match it.

GitOps: pull, not push

ArgoCD runs inside the cluster and continuously compares Git against reality, correcting drift on its own; the CI pipeline can only write to a config repository and never touches the cluster directly. Config is deliberately split across 5 repositories in 3 roles (source per service, one Helm chart, one config repo) so the commit the pipeline writes can never re-trigger the pipeline itself. Dev syncs automatically with self-heal and prune; prod requires a human to press sync, then Argo Rollouts shifts traffic 25% → pause → 50% → pause → 100%.

CI/CD: build once, promote the bytes

1. 1

   Push to main

   Kaniko builds the image inside an ordinary pod, with no Docker daemon and no privileged container.

2. 2

   The image lands in ECR

   Tagged with the short commit SHA. This is the only build that will ever happen for this change.

3. 3

   Dev updates itself

   A pipeline step writes the new tag into the dev values file; ArgoCD notices the commit and syncs on its own.

4. 4

   Manual testing on dev

   The only step before a release that isn't automated.

5. 5

   Anyone can cut a release

   Run the pipeline from the GitLab UI and give it a version. No repo clone, no terminal needed.

6. 6

   The same image moves up

   Retagged in ECR via the AWS CLI, so production runs the exact bytes already tested on dev, not a fresh build.

7. 7

   Prod goes OutOfSync

   The pipeline writes the new repository and tag into the prod values file, then stops. It never touches the cluster.

8. 8

   A human approves, then canary

   Someone clicks sync in the ArgoCD UI, then Argo Rollouts shifts traffic in stages: 25%, a pause, 50%, another pause, 100%.

Tech stack

Highlights

• Went from hand-running kubeadm on Mac minis (etcd, API server, scheduler, controller manager, all by hand) to EKS, before layering GitOps on top — EKS removed kubeadm init, not everything that comes after it.
• GitOps is pull, not push: ArgoCD runs inside the cluster and continuously reconciles it against Git, so the CI pipeline never holds cluster credentials at all.
• Config is split across 5 repositories in 3 roles (source, Helm chart, and environment config) specifically so the commit the pipeline writes can never re-trigger the pipeline itself.
• Production never rebuilds an image, it retags the exact bytes already tested on dev in ECR, then Argo Rollouts shifts traffic in stages instead of all at once.
• Every pod reaches AWS (Secrets Manager, SQS, ECR) through EKS Pod Identity, scoped per ServiceAccount, so there is no static AWS access key stored in the cluster, the pipeline, or any Git repo.
• The full stack, EKS, RDS, ArgoCD, and observability included, was estimated at under $280/month on the AWS Pricing Calculator.