Module 6: Rollback And Safety Nets
2. What rollback means for infrastructure
Description
This lesson develops, with full precision, the distinction the module's introduction left on the table: infrastructure rollback isn't a Terraform operation, it's a pattern that combines Git (reverting a commit) with the pipeline you already built (letting ci.yml/apply.yml run again, in reverse). You're going to see the complete mechanism, step by step, before running it for real in lesson 3.
Connection to the module
Lesson 1 named the difference between application rollback and infrastructure rollback, in general terms. This lesson develops it in full: what exactly "reverting an HCL commit" means, why that's enough to undo an infrastructure change, and where this guide's responsibility ends against the rollback of an application running on top of that infrastructure. Lesson 3 takes this mechanism and runs it for real, on a real andes-cargo-infra/ commit.
Analogy: undoing a remodel by following the blueprints backward, not pressing Ctrl+Z
Imagine an architect delivers blueprints for an office remodel: move a wall, add a door, close a window. The contractor builds exactly that. A week later, someone decides the wall never should have moved. There's no "undo" button on the physical construction — nobody can press Ctrl+Z on a real wall. What does exist is a second set of blueprints: the earlier version, the one that described the office before the change. The contractor goes back to that office with those old blueprints, and builds exactly what they describe — which, in this case, means undoing the new wall and putting everything back the way it was.
That's, precisely, what git revert does with an infrastructure commit. There's no mechanism that tells AWS (or LocalStack) "forget the last thing you did" — what does exist is the earlier HCL commit, the description of how the infrastructure should have looked before the change. git revert doesn't undo the change directly against the cloud: it creates a new commit, whose content is exactly the opposite of the commit you want to undo, and lets the same pipeline —the same contractor, with the same process— build what that new commit describes.
The complete mechanism, in four steps
① A bad commit is already on main
(e.g., a tag got added that turns out to be wrong)
│
▼
② git revert <bad-commit-hash>
creates a NEW COMMIT, whose diff is the exact OPPOSITE
(doesn't erase the bad commit from history — it compensates for it)
│
▼
③ that new commit goes through ci.yml, like any other
terraform plan calculates: "this is what needs to
change to get back to the earlier state"
│
▼
④ a person reviews that plan (the same process as always)
and, if they approve it, apply.yml applies it
→ the infrastructure goes back to the state it had
before the original commitNotice this diagram's most important detail, the one that sets git revert apart from other ways of "undoing" in Git: the original commit never disappears from the history. git revert isn't the same as git reset (which moves the branch pointer backward, as if the commit never existed) nor the same as editing history by hand (git rebase -i, rewriting past commits). git revert adds a new commit, going forward, that compensates for the earlier one — the complete history, including the mistake, stays intact and auditable forever. This isn't a minor technical detail: it's, literally, the same reason this module insists so much on auditability (lesson 5) — a history that's never rewritten is a history you can trust.
Why this is enough: the plan calculates the way back for you
Here's the point that makes this pattern work without you having to think about "how to undo" every different kind of change: you don't have to manually calculate which commands undo a change. Terraform already knows how to compare the HCL against the state and calculate the difference — that's exactly what terraform plan does on every ci.yml run, since Module 3. If the reverted HCL describes the earlier state, the plan on that reverted HCL automatically calculates "what needs to change to get there" — without anyone having to write, by hand, the inverse command of "add the tag" (which would be "remove the tag") for every possible kind of change.
This is, at bottom, the same declarative advantage you already learned in terraform-and-iac-guide: you describe the desired state, not the sequence of steps to get there. A rollback, under this model, is nothing more than "changing what the desired state is, back to the earlier one" — and letting the same machinery (the same plan, the same apply) do the rest.
Boundary: this is NOT application rollback
It's worth being precise here, because the same term —"rollback"— means something structurally different in the world of applications running on top of this infrastructure:
| Infrastructure rollback (this module) | Application rollback | |
|---|---|---|
| What gets reverted | The HCL that describes resources (buckets, tables, roles) | The running artifact (a container image, a deployed binary) |
| Mechanism | git revert of the HCL commit + the same pipeline (plan/apply) | Pointing the runtime back to an already-built earlier version (v41 instead of v42) |
| What needs to already exist | Nothing new — the reverted HCL describes everything needed | The earlier artifact (the v41 image) has to still be available in some registry |
| Where it's taught | Here (cicd-and-gitops-on-aws-guide) | kubernetes-and-eks-in-production-guide, aws-serverless-and-containers-guide |
Andes Cargo, in this guide, never deploys an "application" in the sense of a running, versioned service — it deploys infrastructure: a bucket, a table, IAM roles, a Lambda function packaged as part of the apply, not as an independently versioned service. If Andes Cargo adds a backend service running in containers on EKS in the future (the topic of kubernetes-and-eks-in-production-guide), that service would have its own rollback mechanism —based on going back to an earlier container image, not on reverting HCL—, coexisting with the mechanism you learn here, without replacing it. Both mechanisms are real, both matter, and neither substitutes for the other: Andes Cargo's HCL would still need its own git revert even if that additional application had a completely different container rollback.
Common mistakes
Thinking git revert "restores" the infrastructure directly (conceptual, this lesson's most important mistake). What happens: someone runs git revert and expects the infrastructure to change at that exact instant, with nothing else happening. How to spot it: if you expect to see anything different in LocalStack/AWS immediately after running git revert, before any pipeline has run. How to fix it: git revert only creates a new commit — it's exactly as "inert" as any other HCL commit until ci.yml calculates a plan on it and apply.yml genuinely applies it. The real change in the infrastructure happens at the same point as always: when apply.yml runs, not when the commit gets created.
Confusing git revert with git reset (Git-based, general but critical in this context). What happens: someone, searching for "undo a commit" in Git, finds git reset --hard HEAD~1 and uses it instead of git revert. How to spot it: if the original commit disappears from git log instead of a new commit appearing that compensates for it. How to fix it: git reset moves the branch pointer backward, as if the commit never existed —destroying the history of what happened, on top of being dangerous on a branch that's already shared/merged—. git revert is the right choice for this pattern because it preserves the complete history: the mistake stays on record, and so does its fix, both with author and date, forever.
Looking for a Terraform command that "undoes" the last apply (conceptual, revisit lesson 1). What happens: someone looks for something like terraform apply --undo or similar. How to fix it: as you already saw in lesson 1, that command doesn't exist. This pattern's complete "undo" lives in Git (the reverted commit), not in any new Terraform flag — Terraform simply calculates the plan/apply on the HCL you give it, regardless of whether that HCL is "new" or "an earlier version recreated by a revert."
Exercises
Exercise 1 — Explain why git revert doesn't erase the mistake from history, and why that's an advantage, not a limitation. In two or three sentences, to a colleague who'd rather "just have the mistake disappear."
See solution
A complete answer sounds, roughly, like this: "If the bad commit disappeared from the history, so would the evidence of what happened, who proposed it, who approved it, and when — exactly the kind of information this module's lesson 5 (and Module 1's lesson 2) already identified as what a pipeline adds on top of a manual change. git revert preserves both commits, the mistake and its fix, as part of the same auditable history — anyone can later reconstruct what happened, without relying on anyone's memory."
Exercise 2 — Predict a revert's plan before running it. If a commit added a Compliance tag to a bucket (like the one you already saw in Module 3), and you revert that commit, what kind of change would you expect to see in the resulting plan: a creation, an update, or a destruction? Justify your answer.
See solution
On a project with infrastructure already applied (a real, completed apply), you'd expect to see an update (~ update in-place): the bucket's tags attribute would change, removing Compliance, without the whole bucket getting created or destroyed — the same ~ symbol you already saw in Module 5 (lesson 7) for an attribute change on an existing resource. Watch out for the trap: on a project with no real apply ever completed (this machine's case, without LOCALSTACK_AUTH_TOKEN), the plan would still show a complete creation (+ create) for every resource, because the state has nothing to compare against — lesson 3 confirms this with literal output.
Exercise 3 — Find an example of application rollback from your own experience (or imagined) and compare it to this pattern. Describe, in a short paragraph, a case where "going back to an earlier version" means reactivating an already-built artifact, and explain why that mechanism wouldn't work for undoing a tag on an S3 bucket.
See solution
A typical example: a web application deployed as a container image on Kubernetes, where kubectl rollout undo points the Deployment back to the earlier version's image, already built and available in the container registry. That mechanism works because a complete, ready-to-activate artifact of the earlier version exists. It wouldn't work for undoing an S3 tag because there's no infrastructure "artifact" at all: a bucket isn't a versioned image you can "reactivate" — it's a live resource, modified in place, whose only description of "how it looked before" lives in an earlier commit's HCL, not in any artifact registry.
Summary and next step
In this lesson you developed infrastructure rollback's complete mechanism: git revert creates a new commit (doesn't erase the original), that commit goes through the same ci.yml/apply.yml as any other change, and terraform plan automatically calculates the way back to the earlier state, without anyone having to write the inverse command by hand. You also traced the exact boundary with application rollback —going back to an already-built artifact—, a different mechanism that lives in kubernetes-and-eks-in-production-guide and aws-serverless-and-containers-guide.
Before moving on you should be able to: explain the difference between git revert and git reset; predict what kind of change (+/~/-) you'd expect to see in a revert's plan, depending on whether infrastructure has already been applied or not; and precisely locate the boundary between this module and application rollback.
Lesson 3 —hands-on— runs all of this for real: a real git revert on andes-cargo-infra/, with ci.yml calculating the revert's plan and apply.yml attempting to apply it.
Resources
- Git Docs —
git revert— official documentation, including the distinction fromgit reset. - Git Docs —
git reset— for a direct comparison of the two mechanisms. - Terraform Docs — Command: plan — the mechanism that automatically calculates the "way back," already cited since Module 3.
- This guide's Module 5 (
07-hands-on-running-the-drift-job-manually.md) — the~ update in-placeexample Exercise 2's solution reuses.