Module 2: Pods Deployments And Services

3. Deployments: ReplicaSets and the declarative control loop

Next →

Description

Lesson 2 left an open question: if a loose Pod has no self-healing mechanism, which object does, and how does it work under the hood? This lesson answers that precisely: a Deployment doesn't "watch" Pods actively as if it were a separate process running a while true. It does something different, and simpler than it seems — it declares a desired state, and a control plane component (the controller-manager, which you already know by name from Module 1, lesson 6) compares it against reality, in a loop that never stops, acting only when it finds a difference.

Connection to the module

This lesson is the conceptual counterpart to lesson 5 (hands-on): here you understand the exact mechanism behind "I delete a Pod and someone replaces it on its own" — something you're going to see actually run, with andes-cargo-status-api, two lessons from now. It's also the first time in this guide you see, in detail, the pattern that reappears, under another name, across the entire Module 5: GitOps with ArgoCD is exactly this same reconciliation loop, applied to a Git repository instead of a replica count.

Declarative vs. imperative: the distinction that everything else rests on

So far, in this guide, you've used two different styles of telling a tool something, even though they haven't been named explicitly:

  • Imperative: you tell the system what to do, step by step. docker run -d --name andes-cargo-status-api ... is imperative — it's a single order, executed once, with no memory that you gave it. If the container crashes, no one reads that command again to recreate it; you'd have to run it again, yourself, by hand.
  • Declarative: you tell the system what final state you want, without specifying the steps to get there. kind-config.yaml (Module 1) was your first example — you declared "I want one control-plane and two workers," and kind decided how to get there. A Deployment is exactly the same style, but applied to Pods: you declare "I want N replicas of this Pod, running this image," and Kubernetes decides how to maintain that reality, indefinitely, without you asking again.

The difference isn't just one of writing style — it's about who acts when something changes. With an imperative command, if reality drifts from what you asked for (the container crashes), no one does anything until you give another order. With a declarative object, there's a process running permanently inside the cluster whose only job is to notice that drift and correct it — with no one asking again.

The control loop: how the controller-manager reconciles on its own

The component responsible for this behavior is kube-controller-manager, one of the control plane processes you already saw running as a Docker container in Module 1, lesson 6. Inside it run dozens of independent controllers, each responsible for a different object type — the one that matters for this lesson is the Deployment controller (which, in turn, delegates most of the work to a ReplicaSet controller, see the next section).

Every controller implements the same pattern, called a reconciliation loop, and it's worth seeing it explicit, because it's Kubernetes' most important design pattern overall:

                    THE RECONCILIATION LOOP (never stops)

        ┌──────────────────────────────────────────────────────┐
        │                                                          │
        ▼                                                          │
  1. Read the DESIRED state                                        │
     (the manifest you declared: "I want 3 replicas")              │
        │                                                          │
        ▼                                                          │
  2. Read the ACTUAL state                                         │
     (how many Pods with that label really exist, right now)       │
        │                                                          │
        ▼                                                          │
  3. Compare both                                                  │
        │                                                          │
        ├── Do they match? ──▶ Do nothing, wait, go back to step 1
        │
        └── Don't they match? ──▶ 4. Act to bring the actual
                                     state closer to the desired one
                                     (create missing Pods, or
                                     delete extra Pods)
                                        │
                                        └──────────────▶ back to step 1

There's no "notify someone" or "ask for permission" step — the loop acts on its own, all the time, at a frequency of seconds. This explains a behavior you're going to confirm with your own hands in lesson 5: when you delete a Pod managed by a Deployment, there's no perceptible delay before seeing the replacement — the controller was already running the loop, and it detects the difference on the next pass, almost immediately.

Deployment and ReplicaSet: two objects, one decision you make

Lesson 2 already showed the complete hierarchy (Deployment → ReplicaSet → Pod); this section explains why there are two separate objects instead of one, because it isn't a gratuitous complication:

  • ReplicaSet is the object that literally maintains a number of replicas of a Pod with a fixed template. Its reconciliation loop is simple: it counts the Pods with the label you gave it, and creates or deletes until the number matches replicas.
  • Deployment is a layer on top of ReplicaSet, and adds something ReplicaSet can't do on its own: managing changes to the Pod template over time — for example, updating the image to a new version without taking the service down all at once. When you change something in a Deployment's template, it doesn't modify the existing ReplicaSet — it creates a new one, with the updated template, and gradually moves Pods from the old ReplicaSet to the new one (RollingUpdate, the default behavior — you're going to go deeper on deployment strategies in Module 5, lesson 7, when ArgoCD enters the picture).

In practice, you're almost never going to create a ReplicaSet by hand — you declare a Deployment, and Kubernetes creates the ReplicaSet automatically, with a derived name (the Deployment's prefix, plus a hash suffix — you're going to see this exact pattern in lesson 5). This guide never declares a ReplicaSet directly for this exact reason: it's a real object, that exists and you can inspect, but it isn't the interface you use to work with it.

                 WHAT CHANGES IF YOU UPDATE A Deployment's IMAGE

  Before                                  After kubectl apply (new image)

  Deployment                              Deployment
      │                                       │
      ▼                                       ▼
  ReplicaSet-abc123 (3/3)                 ReplicaSet-abc123 (0/3) ── gradually emptied
      │                                       │
      ├── Pod-abc123-x1                   ReplicaSet-def456 (3/3) ── NEW, gradually filled
      ├── Pod-abc123-x2                       │
      └── Pod-abc123-x3                       ├── Pod-def456-y1
                                                ├── Pod-def456-y2
                                                └── Pod-def456-y3

This mechanism — two ReplicaSets, one emptying while the other fills — is exactly what makes a RollingUpdate possible with no service downtime: at every moment there's at least some replicas responding, never all three down at once.

A Deployment's minimal anatomy

apiVersion: apps/v1
kind: Deployment
metadata:
  name: <deployment-name>
  namespace: <namespace>
spec:
  replicas: <N>
  selector:
    matchLabels:
      app: <label>
  template:
    metadata:
      labels:
        app: <label>
    spec:
      containers:
        - name: <container-name>
          image: <image:tag>
          ports:
            - containerPort: <port>

Notice two things that didn't appear in lesson 2's Pod manifest:

  • apiVersion: apps/v1, not v1. Deployment lives in the apps API group, alongside ReplicaSet, StatefulSet, and other workload controllers — a different group from the "core" group (v1) where Pod, Service, and Namespace live.
  • spec.template contains, inside itself, the same structure as a complete Pod manifest (metadata.labels + spec.containers) — not a coincidence. The Deployment doesn't invent a new way to describe a Pod, it reuses exactly the same template you already know, and uses it as a "mold" for every replica it creates.
  • spec.selector.matchLabels must match, exactly, spec.template.metadata.labels. If they don't match, kubectl apply rejects the manifest — it's how Kubernetes guarantees the Deployment always knows how to recognize which Pods are "its own."

Analogy: the thermostat, not the technician manually adjusting the AC

Picking back up the office building analogy: a Deployment is the thermostat, not the person who manually raises and lowers the AC every time someone complains about being cold. You configure the thermostat once — "keep this temperature" — and from then on, the system reacts on its own every time the real temperature drifts, with no one having to notice or ask again. A ReplicaSet, in this same analogy, is the physical compressor the thermostat turns on and off — the piece that carries out the correction, while the thermostat is the one deciding when to correct and by how much. You never talk directly to the compressor; you talk to the thermostat, and the rest happens on its own — exactly the same relationship you have with a Deployment versus its ReplicaSet.

Deep dive: why this is the same pattern you're going to see in GitOps (Module 5)

It's worth previewing this connection, because it changes how you're going to read the entire Module 5: ArgoCD does, over an entire Git repository, exactly the same thing the controller-manager does over a Deployment. It reads the desired state (the manifests in Git), reads the actual state (the objects that really exist in the cluster), compares, and acts only if there's a difference — this lesson's same three-step loop, with "Git" in place of "the Deployment manifest" and "the whole cluster" in place of "a ReplicaSet's Pods." When you get to Module 5 and see ArgoCD "converging the cluster toward what Git declares," you're not going to be learning a new concept — you're going to be seeing this lesson's same reconciliation loop, applied at a much larger scale.

Common mistakes

Believing the Deployment actively "checks" every so many seconds, as if it ran a timer (conceptual, a simplification that's almost always harmless, but breaks intuition in edge cases). What happens: someone imagines the reconciliation loop as a cron running every N seconds, and is surprised by how fast a Deployment reacts to a change — or, the other way around, always expects an instant reaction. Why it happens: "loop that repeats" sounds like a timer, but the real mechanism is more like an event-driven system: controllers react both to changes notified by kube-apiserver (nearly immediate) and to a periodic backup resync (with a longer interval, on the order of minutes, for cases where a notification was missed). How to spot it: if you observe an almost instant reaction to a change and assume "it's always this fast," regardless of the cause of the drift. How to fix it: for what you need in this guide, it's enough to know the loop reacts quickly to most changes (including deleting a Pod, which you're going to confirm in lesson 5) — the exact event-notification mechanism is documented in depth in the official controllers reference (see Resources), if you want to go deeper than what this guide covers.

Editing a ReplicaSet directly, instead of editing the Deployment that created it (workflow, specific to anyone who already has some kubectl experience). What happens: someone finds the ReplicaSet with kubectl get replicasets, and edits its replica count directly there, instead of on the Deployment. The change seems to work for a moment, but the Deployment reverts it shortly after, with no visible error message. Why it happens: the ReplicaSet is a real, editable object — but the Deployment is its source of truth, and as soon as the Deployment's reconciliation loop runs again, it overwrites any change made directly on the ReplicaSet, because from its perspective the ReplicaSet "should" keep showing the template the Deployment declared. How to spot it: a change on the ReplicaSet that reverts on its own, with no one touching it again. How to fix it: any desired-state change — replica count, image, environment variables — is always made on the Deployment, never on the ReplicaSet it creates. This guide never edits a ReplicaSet directly, not even once, for this exact reason.

Assuming "declarative" means Kubernetes validates that the desired state makes business sense (conceptual). What happens: someone declares replicas: 3 with an image that doesn't exist, or with the wrong port, and expects Kubernetes to "reject" the manifest for being incorrect in some deeper way than a YAML syntax error. Why it happens: "declarative" sounds like "the system understands what I want," but in reality the reconciliation loop tries to fulfill the desired state exactly as declared, without judging whether it makes sense — if the image doesn't exist, the result isn't a rejection, it's a Pod stuck in ImagePullBackOff (the exact error you already saw named in Module 1, lesson 7), retrying indefinitely. How to spot it: a Deployment showing 0/3 in READY for much longer than expected. How to fix it: kubectl describe pod <name> (you already used it in lesson 4) always shows, in its Events section, the exact reason a Pod isn't ready — the reconciliation loop tries to fulfill the desired state, but it can't guess that state has an error, only report it.

Exercises

Exercise 1 — Explain the reconciliation loop without the diagram. Without looking back at the corresponding section, describe in your own words the three steps of the reconciliation loop, and what happens on each of the two possible paths of the comparison step.

See solution
  1. Read the desired state (the declared manifest, for example replicas: 3).
  2. Read the actual state (how many Pods with the corresponding label really exist, right now).
  3. Compare: if they match, nothing happens and the loop starts again; if they don't match, the controller acts (creates missing Pods or deletes extra Pods) to bring the actual state closer to the desired one, and the loop starts again immediately.

This loop never stops as long as the Deployment exists — it's the exact technical reason behind "Kubernetes self-heals."

Exercise 2 — Predict the result of a direct edit. With what you learned in "Common mistakes," predict what would happen if you directly edited, with kubectl edit replicaset <name>, the replica count of a ReplicaSet created by a Deployment with replicas: 3 declared. Does the change stick? Justify your answer.

See solution

It doesn't stick, or it only sticks momentarily. The Deployment remains the source of truth for that ReplicaSet — as soon as the Deployment's reconciliation loop runs again (nearly immediate), it notices the ReplicaSet it manages doesn't match what the Deployment declares, and corrects it back to replicas: 3. Any desired-state change has to be made on the Deployment, not on the ReplicaSet it creates.

Exercise 3 — Connect the pattern with GitOps. Without going back to the deep-dive section, explain in two or three sentences why "ArgoCD converges the cluster toward what Git declares" (Module 5) is, essentially, the same mechanism this lesson explains for a Deployment.

See solution

Both are reconciliation loops: they read a desired state (the Deployment manifest, or a Git repository's content), read the actual state (the live Pods, or the cluster's real objects), and act only when there's a difference between the two — with no one explicitly asking each time. The difference is in scale and in what's declared: a Deployment reconciles the replica count of a single Pod type; ArgoCD reconciles the entire cluster against everything a Git repository declares. The underlying principle — compare desired against actual, act with no intervention — is identical.

Summary and next step

In this lesson you understood the exact mechanism behind Kubernetes' self-healing: the distinction between imperative and declarative, the three-step reconciliation loop running permanently inside the controller-manager, and why Deployment and ReplicaSet are two separate objects — one that manages template changes over time, another that maintains a fixed replica count for a given template. You also got a preview of how this same pattern reappears in Module 5 with ArgoCD, at a larger scale.

Before moving on you should be able to: explain the difference between an imperative command and a declarative object with your own example; describe the reconciliation loop's three steps from memory; and explain why editing a ReplicaSet directly doesn't produce a lasting change while the Deployment that created it keeps declaring something different.

Next lesson: hands-on, your first Pod. There you confirm, with real evidence, half of this contrast — a loose Pod, with no Deployment behind it, that no one replaces when you delete it. Lesson 5 confirms the other half: the same behavior, but with a Deployment in the mix, running andes-cargo-status-api.

Resources

  1. Kubernetes — Deployments — the complete official reference, including the RollingUpdate mechanics with two ReplicaSets.
  2. Kubernetes — ReplicaSet — official reference, including the documentation's explicit warning about why one is almost never declared directly.
  3. Kubernetes — Controllers — the official explanation of the control loop pattern, the conceptual foundation of this entire lesson.
← Previous2. What a Pod is, and why you rarely create one directlyNext →4. Hands-on: your first Pod