Module 1: Why Kubernetes And The Continuity Challenge

1. Guide introduction: the orchestrator that was missing

Description

Welcome to the third compute piece of NIEVA's AWS Cloud ecosystem. If you made it here, you already completed aws-core-services-guide (single-account IAM, S3, basic VPC/EC2, the DynamoDB Shipments table) and aws-serverless-and-containers-guide (the serverless-vs-containers criteria, Step Functions, EventBridge, API Gateway, and — most relevant for this guide — you built with your own hands the Dockerfile, the image, and the local registry for andes-cargo-status-api, a read-only HTTP service that looks up a shipment's status by shipmentId). This guide — Kubernetes and EKS in Production — repeats none of that. It also doesn't re-explain Docker from scratch: that's docker-essentials-guide, a transitive prerequisite you already used without knowing it every time you ran docker build.

What this guide does do is resolve a concrete frustration left pending at the close of aws-serverless-and-containers-guide: you built a real container image, ran it locally with docker run, published it to a registry — and the orchestrator that was supposed to keep it running continuously, scaled and healthy, never actually ran. The andes-cargo-cluster ECS cluster and the status-api-service service exist only as shown YAML and JSON, verified against official AWS documentation, because LocalStack's free tier doesn't cover ECS. This guide takes that exact image, those exact two names, and gives them, for the first time, a real orchestrator running on your machine: Kubernetes, via kind (kubernetes-in-docker), no AWS account, no credit card, no paid-plan limit whatsoever.

Connection to the module

This Module 1 has four jobs, in order: install the criteria for why Kubernetes when ECS already seemed enough (lesson 2); resolve, with full honesty, the Andes Cargo case's continuity challenge — what's inherited, what's reused, what's genuinely new (lesson 3); install the complete lab, for real, with executed evidence (lessons 4, 5, and 7); and understand the architecture of what you just created before building anything on top of it (lesson 6). Lesson 8 — this module's project — leaves you with a running cluster, the image loaded, and the full map of the seven modules that follow.

What this guide assumes you already know (and won't re-explain)

Four previous guides in the ecosystem left you with the ground prepared. This guide uses them without re-teaching any of them even once:

Single-account IAM, S3, basic VPC/EC2, DynamoDB (aws-core-services-guide) — the Shipments table, with partition key shipmentId, already exists and remains the data source andes-cargo-status-api queries.
The serverless-vs-containers criteria, and the complete Dockerfile for andes-cargo-status-api (aws-serverless-and-containers-guide, Module 6) — five instructions on top of python:3.13-slim, a two-endpoint Flask service (/health and /shipments/<id>). This guide does not rewrite that Dockerfile — it inherits it line by line, which you confirm yourself in lesson 7 of this module.
The ECS/Fargate model — clusters, task definitions, services, launch types (aws-serverless-and-containers-guide, Module 7) — you know what a desiredCount is, what a taskRoleArn does versus an executionRoleArn, and why a loose docker run isn't the same as an orchestrated service. This guide assumes that vocabulary is already installed; you'll use it constantly as a point of comparison against Kubernetes.
Dockerfile, layers, docker build/run, image best practices (docker-essentials-guide, transitive) — if you still haven't done that guide, lesson 7 of this module (rebuilding the image) is going to read strangely.
The push-based vs. pull-based GitOps vocabulary (cicd-and-gitops-on-aws-guide, named in its M7.4) — this module doesn't go deep on it, but Module 5 of this guide picks it up directly, without redefining the terms from scratch.

What you're going to build: from an orphaned image to a complete platform

                    ANDES CARGO — FROM "IMAGE BUILT, NEVER ORCHESTRATED"
                    TO A COMPLETE KUBERNETES PLATFORM IN PRODUCTION

  aws-serverless-and-containers-guide left you here:

    andes-cargo-status-api:1.0 (image, EXECUTED)
              │
              │  docker push to registry:2 (EXECUTED, $0)
              ▼
    localhost:5000/andes-cargo-status-api:1.0
              │
              │  who runs it continuously?
              ▼
    andes-cargo-cluster (ECS) ──── REPRESENTATIVE, never executed
    status-api-service (ECS) ──── (LocalStack Hobby doesn't cover ECS)

  This guide turns it into this:

    andes-cargo-status-api (same Dockerfile, unchanged)
              │
              │  kind load docker-image (M1, EXECUTED)
              ▼
    andes-cargo-cluster (kind — real Kubernetes, EXECUTED)
              │
    ┌─────────┴──────────────────────────────────────────────────┐
    │  Deployment + Service + ConfigMap/Secret + probes + HPA (M2-M3)│
    │  Ingress + NetworkPolicy (M4)                                    │
    │  ArgoCD syncing from Git (M5)                                     │
    │  Gatekeeper + Kyverno + trivy image (M6)                          │
    │  Real EKS: what changes (M7, representative, with honesty)        │
    └────────────────────────────────────────────────────────────────┘

The name andes-cargo-cluster is not a coincidence — it's the same name the previous guide left documented, never actually run. When you see it appear in a real kubectl get nodes in lesson 5 of this module, you'll be looking at the cluster that aws-serverless-and-containers-guide promised and couldn't deliver.

The complete map: this guide's 8 modules

#	Module	What it installs	Andes Cargo piece it builds
1	Why Kubernetes and the continuity challenge	The K8s-vs-ECS criteria, `kind`, `kubectl`, the cluster	`andes-cargo-cluster` running, image loaded
2	Pods, Deployments and Services	The three workload primitives	`andes-cargo-status-api` with N replicas, exposed via `status-api-service`
3	Configuration, secrets, health and autoscaling	`ConfigMap`/`Secret`, probes, `HorizontalPodAutoscaler`	The same Deployment, now production-ready
4	Networking, `Ingress` and `NetworkPolicy`	Kubernetes' networking model, `ingress-nginx`	`status-api-service` exposed over HTTP and protected
5	GitOps with ArgoCD	Pull-based operator, Gitea, `Application`	A change in Git that reflects itself, no `kubectl apply`
6	Runtime security: admission control and scanning	OPA Gatekeeper, Kyverno, `trivy image`	Real guardrails on `andes-cargo`
7	What's specific to EKS	Managed control plane, node groups, IRSA, ALB Controller	The migration plan from `kind` to real EKS
8	Capstone: Andes Cargo on Kubernetes	End-to-end walkthrough, one change that passes and one the gate stops	The complete, documented system

This module's map: the 8 lessons

#	Lesson	What you practice
1	Introduction (this one)	The complete map, what's inherited, what's new
2	Why Kubernetes when ECS was already enough	The market criteria, what you gain and what it costs
3	The continuity challenge: picking up `andes-cargo-status-api`	Honest recap of aws-serverless M6/M7
4	Hands-on: installing `kind` and `kubectl`	Executed: versions confirmed on your machine
5	Hands-on: your first cluster	Executed: `andes-cargo-cluster` running
6	Cluster architecture	`kube-apiserver`, `etcd`, `scheduler`, `kubelet`
7	Hands-on: loading the image into the cluster	Executed: the `status-api` image inside `kind`
8	Project: Andes Cargo's cluster, ready	Executed: final checklist + roadmap for the rest of the guide

What will exist by the end of this guide: the capstone (preview)

Module 8, the complete capstone, is where every piece you build starting from this first module points. It's worth looking at it now, even with seven modules left, because it changes how you read everything that follows: by the end, andes-cargo-status-api will be running inside andes-cargo-cluster, with a Git repository (andes-cargo-k8s/) as the single source of truth that an operator (ArgoCD) syncs on its own, with no one running kubectl apply by hand; every object trying to enter the cluster will first pass through an admission controller that can reject it before it even exists; and the image that Deployment runs will have been scanned by Trivy. An innocuous change — a new label — will cross that whole path frictionlessly; a change that violates a security policy will be rejected, with the real error message, before touching a single Pod.

That's the underlying difference between what aws-serverless-and-containers-guide left (a built image, an orchestrator that was only documented) and what this guide delivers: the same component, now with a complete production system around it.

What this guide does NOT teach (and where it does)

So you know the boundary from the start: this guide builds Kubernetes end to end, but it isn't where every path it touches in passing actually ends.

Terraform/IaC itself (HCL, modules, state) → terraform-and-iac-guide. The only infrastructure YAML that appears here is what's needed for what this guide teaches (Kubernetes manifests, a kind-config.yaml), never a new business resource declared in Terraform.
Push-based CI/CD on AWS (GitHub Actions pipelines, automated plan/apply) → cicd-and-gitops-on-aws-guide, a prerequisite. This guide is the other pole of the same GitOps principle: pull, not push.
Infrastructure and IaC supply-chain security (end-to-end OIDC against real AWS, SBOM, signing with cosign, HCL scanning) → cloud-security-and-guardrails-guide. This guide builds the in-cluster admission control and the container image scanning that guide deferred — two different layers.
Kubernetes FinOps (cost per Pod, node right-sizing, Spot) → finops-and-cost-guardrails-guide. A guardrail in this guide is never "this is expensive" — it's "this violates a policy."
SRE and incident response (SLI/SLO, on-call, postmortem) → sre-and-incident-response-guide (not yet designed). This guide builds the system that prevents the incident, not the discipline of responding when it fails anyway.
GenAI on EKS (GPU on nodes, inference) → genai-on-aws-production-guide (not yet designed).
In-depth cluster observability (Prometheus, Grafana) → monitoring-observability-guide. This guide uses kubectl logs/describe/top as a minimal debugging tool, not as a topic on its own.
Docker from scratch → docker-essentials-guide, a prerequisite.

The analogy that will accompany you through this whole guide: the orchestra conductor

You're going to develop this in depth in lesson 2, but it's worth keeping in mind from the start: a container orchestrator is, literally, an orchestra conductor. Each musician (container) knows how to play their instrument, but without someone to set the tempo, decide when each section comes in, and react in real time if a violinist loses focus, the "orchestra" — your system — falls apart at any unforeseen moment. A manual docker run is a lone musician, playing with no one listening or correcting them. ECS is a conductor capable of leading a quartet — enough for a small hall, with a limited repertoire of available instruments. Kubernetes is a conductor capable of leading a full symphony orchestra — dozens of musicians, entire sections, instruments other composers (the community) keep adding — but that power comes with a real trade-off: conducting a symphony demands far more preparation than conducting a quartet. That trade-off, with name and evidence, is exactly lesson 2's topic.

Common mistakes

Assuming this guide will review Docker, IAM, ECS, or the serverless-vs-containers criteria before moving forward (expectation). What happens: someone arrives from aws-serverless-and-containers-guide expecting a review of those topics before touching Kubernetes, and is surprised when lesson 2 jumps straight into comparing Kubernetes against an ECS model it treats as already known. Why it happens: it's the natural pattern of any staged guide series — but this one deliberately doesn't follow it: aws-serverless-and-containers-guide Module 7 already covered ECS/Fargate in depth, and repeating it here would be the same content twice. How to spot it: if at any point in this module you ask yourself "wait, what was a taskRoleArn again?", it's not that this guide skipped it by accident — it's an explicit design decision. How to fix it: go back to aws-serverless-and-containers-guide, Module 7, specifically for that concept; don't expect this guide to repeat it along the way.

Confusing "this guide uses the same case" with "this guide rebuilds the same work" (conceptual, the central mistake lesson 3 corrects from the root). What happens: someone assumes they're going to rewrite andes-cargo-status-api's Dockerfile, or invent a new component to have "something of their own" in this guide. Why it happens: it's intuitive to think a new guide needs to build something from scratch to justify its existence. How to spot it: if in lesson 7 of this module you find yourself editing app.py or the Dockerfile instead of just rebuilding the image exactly as it is. How to fix it: this guide's design decision, explained in depth in lesson 3, is exactly the opposite — reusing the image without touching it is the point, not a shortcut. Any change in the service's behavior happens, from Module 3 onward, via ConfigMap/Secret, never by rebuilding the image.

Skipping lesson 4 (installation) because "I already have kind or kubectl installed from another project" (workflow). What happens: someone with prior Kubernetes experience assumes their existing installation is already good enough, without confirming the version, and arrives at lesson 5 with an outdated kind that behaves differently from what this guide describes. Why it happens: kind/kubectl are common tools, and it's easy to assume "installed" is enough without checking the exact version. How to spot it: if your kind version reports something older than v0.32.0 or your kubectl version --client reports something very different from v1.36.1. How to fix it: run the two verification commands from lesson 4 anyway — it takes 10 seconds, and it prevents an old version from giving you different behavior than what this guide documents, without you knowing why.

Exercises

Exercise 1 — Trace the continuity boundary. Without looking back at the "What this guide assumes you already know" section, write from memory the four guides (or parts of guides) this guide inherits without repeating, and for each one, the concrete piece of Andes Cargo it picks up.

See solution

aws-core-services-guide → the DynamoDB Shipments table, which andes-cargo-status-api keeps querying without any change.
aws-serverless-and-containers-guide, Module 6 → andes-cargo-status-api's exact Dockerfile, reused without rewriting a single line.
aws-serverless-and-containers-guide, Module 7 → the names andes-cargo-cluster and status-api-service, reused on purpose as this guide's narrative thread — the cluster that only ever existed as YAML now truly exists.
docker-essentials-guide (transitive) → the vocabulary of layers and docker build/run, which lesson 7 of this module uses without re-explaining it.

If your answer recognized that nothing gets rewritten, everything gets reused, you understood this guide's spirit.

Exercise 2 — Prediction: what will change and what won't. With this lesson's "What you're going to build" diagram, predict: will the name andes-cargo-status-api (the image) change in any module of this guide? And andes-cargo-cluster? Justify each answer in one sentence.

See solution

Neither one changes. The andes-cargo-status-api image is rebuilt exactly as-is in lesson 7 of this module (same Dockerfile, no changes) and is never touched again unless a specific module explicitly calls for it (a hard rule of this guide: any change in behavior goes through ConfigMap/Secret, never by rebuilding the image). The name andes-cargo-cluster doesn't change either — it's, literally, the same name aws-serverless-and-containers-guide used for the documented-never-executed ECS cluster; this guide picks it back up on purpose so it's clear this is the same promised cluster, now actually running.

Exercise 3 — Explain the continuity to a colleague. A colleague who did aws-serverless-and-containers-guide but not this guide asks you: "why is the Kubernetes cluster named the same as the ECS cluster we never got to run?" Answer them in two or three sentences.

See solution

A complete answer sounds roughly like this: "It's not a coincidence or a mistake — it's intentional. aws-serverless-and-containers-guide designed an ECS cluster named andes-cargo-cluster with a status-api-service service, but LocalStack's free tier doesn't include ECS, so that cluster never actually ran, it only stayed documented. This guide picks up those exact two names for the Kubernetes cluster that does run on your machine — it's the same infrastructure promise, fulfilled with a different orchestrator."

Summary and next step

In this lesson you got the complete map of the third compute guide in NIEVA's AWS Cloud ecosystem: eight modules that go from the design criteria (here) to Kubernetes primitives, configuration and autoscaling, networking, GitOps with ArgoCD, runtime security, and what's specific to EKS. You confirmed what this guide inherits without repeating it — IAM/S3/VPC/DynamoDB, andes-cargo-status-api's Dockerfile, the ECS/Fargate model, basic Docker — and got the first preview of the guide's central analogy: a container orchestrator is an orchestra conductor, and the difference between ECS and Kubernetes is the difference between conducting a quartet and conducting a full symphony.

Before moving on you should be able to: name this guide's eight modules and which piece of Andes Cargo each one builds; explain why the name andes-cargo-cluster is reused on purpose; and describe, in your own words, the difference between "reusing" and "rebuilding" the Andes Cargo case.

Lesson 2 formalizes the orchestra-conductor analogy with the real market criteria: when Kubernetes beats ECS, and what that win costs.

Resources

aws-core-services-guide (NIEVA) — the ecosystem's first guide, a full prerequisite of this one.
aws-serverless-and-containers-guide (NIEVA), Modules 6 and 7 — the exact origin of andes-cargo-status-api, andes-cargo-cluster, and status-api-service that this guide picks up.
docker-essentials-guide (NIEVA) — transitive prerequisite, Dockerfile and layers from scratch.
Kubernetes — What is Kubernetes? — the project's official definition, lesson 2's starting point.
kind — Quick Start — the official documentation for the tool you install in lesson 4 of this module.

Next →2. Why Kubernetes when ECS was already enough