Module 1: Why Cicd And Gitops
2. The trouble with manual `apply`
Description
In terraform-and-iac-guide you ran terraform apply dozens of times. It worked, every time. The andes-cargo-infra/ project exists, correct and tested. But there are three questions that guide never made you answer, because they weren't its topic: who ran that apply? with what credentials, on what machine? and what happens if that person, one day, isn't there? This lesson answers those three questions with as much honesty as possible, and then revisits —from a new angle— the previous guide's hardest case study: the terraform destroy an AI agent ran against real production, with the human approving the plan that had already flagged the risk.
Connection to the module
This is the lesson that justifies the rest of the guide, the same way lesson 2 of terraform-and-iac-guide justified that entire guide. You're not going to install anything yet —that starts in lesson 6— and you're not going to define precisely what "CI" or "CD" is yet —that's lesson 3. All this lesson does is name the real pain of having a human as the only step between the code and the infrastructure, with evidence from your own experience and with this sister guide's most-cited incident.
Analogy: the blank check
Imagine that, at your job, every time something needs to be paid, someone on the team signs a blank check and trusts that whoever fills it in later writes the correct amount. There's no second signature. There's no record of who decided that payment was necessary, or who reviewed the amount before the check left the bank. If the amount is wrong —by mistake, by haste, or because nobody looked closely enough— the money is already gone before anyone else could say "wait, check this again."
That is, precisely, what a terraform apply run by hand, with no process around it, is. It doesn't matter how good you are at reading a plan —and in terraform-and-iac-guide you learned to read it carefully—: as long as the only person reviewing the change is the same person applying it, there's no second signature. The check gets cashed the instant it's signed.
Three questions terraform-and-iac-guide never made you answer
1. Who ran the apply?
Think about the andes-cargo-infra/ you already have. Every apply you ran left, yes, a trace in the state —what resources exist, with what configuration— but no trace of who decided to apply it, when, or why. If tomorrow a teammate asks you "who approved the change that added that policy to the bucket last week?", the only honest answer with Terraform run by hand is: whoever had access to the terminal where that command ran, if anyone even remembers. There's no structured, searchable record that a security audit —or simply a curious teammate— can consult.
2. With what credentials, on what machine?
In this guide, against LocalStack, your credentials are test/test — dummy, with zero real risk. But think about what that same flow would mean against a real AWS account: for terraform apply to work from your laptop, your laptop needs AWS credentials with permission to create, modify, and delete the infrastructure andes-cargo-infra/ declares — typically a long-lived access key (AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY), stored in ~/.aws/credentials or exported as an environment variable. That credential then lives on your personal computer: on your disk, in your shell history if you ever exported it by hand, potentially in a memory dump if your machine gets compromised. It's exactly the antipattern this ecosystem's market audit flags as the competition's most-cited security gap —and the reason this guide's entire Module 4 exists.
3. What happens if that person isn't there?
This is the most uncomfortable question, and the most real one. If the only person who knows how to run terraform apply on andes-cargo-infra/ —the only one with credentials configured, the only one familiar with the project— is on vacation, quit, or simply doesn't answer the phone on a Saturday at 3 a.m. when something breaks, who applies the urgent change? With a manual apply, the answer depends entirely on knowledge and access being correctly distributed across the team — something that, in practice, rarely happens without a process that enforces it.
The four problems, now with a human in the middle
terraform-and-iac-guide (lesson 2) already named four problems with imperative infrastructure — loose commands, with no declarative artifact at all. Terraform solved those problems: now you have a file that describes what should exist. But declaring infrastructure in HCL doesn't, by itself, solve the problem of who decides when that HCL becomes reality:
FOUR PROBLEMS TERRAFORM ALONE, WITHOUT A PIPELINE, DOESN'T SOLVE
1. No mandatory review
Terraform shows you the plan. Nothing forces ANOTHER
person to also read it before you approve your own apply.
2. No approval record
The state says WHAT exists. It doesn't say WHO decided it
should exist, when they approved it, or with what justification.
3. No control over who can apply
If the credential lives on your laptop, anyone with access
to your laptop can run apply — whether they should or not.
4. No single point of locking
Nothing stops two people from running apply almost at the
same time, each from their own terminal, with their own
copy of the state — the same lock risk you already saw in
terraform-and-iac-guide, now multiplied by "how many
different laptops can touch main without coordinating."Notice something important: none of these four problems is a Terraform flaw. Terraform does exactly what it promises: it computes a plan, applies it if you ask it to. The problem lives outside Terraform, in the process —or the absence of process— surrounding whoever decides to type apply.
The case study, revisited from a new angle
terraform-and-iac-guide (Module 8) documented, with verifiable evidence, the March 2026 incident where Claude Code ran terraform destroy against DataTalks.Club's real production infrastructure (Alexey Grigorev), after the agent itself accidentally replaced the active state with an old one from an archived .zip. The destroy deleted the VPC, ECS cluster, load balancers, bastion host, the RDS database, and its automatic snapshots — around 1,943,200 rows and 2.5 years of a course's history, with roughly 24 hours of recovery via AWS support (Alexey Grigorev — How I Dropped Our Production Database · Hacker News #47278720 · incidentdatabase.ai — Incident 1424).
The detail that makes that case pedagogically valuable, not just anecdotal, is this: the agent had flagged the risk before executing, and the human approved anyway. terraform-and-iac-guide used that case to install a habit: no apply or destroy runs without reading the full plan first. This lesson asks the question that guide left open, on purpose, for this one: would a pipeline have prevented that incident?
The honest answer, unembellished, has two parts:
No, a pipeline doesn't prevent negligence. If the flow had gone through an apply.yml with a production Environment requiring human approval (Module 4), and the person who approved that Environment had seen the exact same plan with the exact same risk warning — and approved it anyway — the result would have been identical. A pipeline doesn't replace human judgment; it automates the process around that judgment, not the judgment itself.
Yes, a pipeline leaves a record that, in the real incident, didn't exist explicitly. With an apply.yml triggered by a push to main, protected by branch protection (Module 6) and an Environment with required reviewers (Module 4), there would be a structured, searchable trail, impossible to lose in a terminal's history: who approved the PR that introduced the change, who approved the Environment before the destroy ran, when, and against which exact plan —the same one that was reviewed, not one recalculated afterward. In the real incident, approval happened, but not inside a system that recorded it that way. That difference —from "the approval existed, somewhere, maybe" to "the approval exists, with a date, an author, and exact content, queryable forever"— is exactly what a pipeline adds. It doesn't prevent the bad decision. It makes the bad decision auditable, and that auditability is, in practice, what changes teams' behavior: nobody approves as casually when they know their name stays attached, forever, to that specific approval.
Module 6 of this guide returns to this same question, in more depth, after you've built the real guardrail that would have stopped —not the negligence, but the effect— of a destroy against the Shipments table.
Common mistakes
Believing this lesson says "a pipeline is infallible" (expectation-based, explicitly corrected above). What happens: someone finishes the case study section with the conclusion that, if Andes Cargo had a pipeline, no similar incident could ever happen again. Why it happens: it's tempting to read "leaves a record" as "prevents the mistake." How to spot it: if your summary of this lesson is "with a pipeline, this wouldn't have happened" instead of "with a pipeline, we'd know exactly who approved it." How to fix it: go back to the honest-answer paragraph above — a pipeline automates the process, not the judgment. Module 6's "Common mistakes" section (lesson 5) comes back to this once the real guardrail is already built.
Thinking the credentials problem only applies "if you work with real AWS" (scope-based). What happens: someone, working against LocalStack with test/test, concludes that this lesson's question 2 ("with what credentials?") doesn't apply to them, because their dummy credentials carry zero risk. Why it happens: it's true that in this guide's lab there's no real risk —and that's intentional, to keep the $0 commitment. How to spot it: if you think Module 4 (secrets and OIDC) is "theory with no practical application" for you. How to fix it: the lab uses test/test on purpose, so you can practice the complete pattern without risking anything — but the pattern you build (GitHub Secrets, named OIDC) is exactly the one you'd use the day andes-cargo-infra/ points at a real account, without changing the pipeline's structure.
Confusing "reviewing the plan myself, more carefully" with "having a second review" (conceptual). What happens: someone thinks this lesson's problem #1 (no mandatory review) gets solved simply by being more careful reading the plan before applying. Why it happens: it's a real, valuable habit —in fact, it's the habit terraform-and-iac-guide installed— and it feels sufficient. How to spot it: if your mental solution is "I review better" instead of "someone else also reviews, before I can apply." How to fix it: reading your own work carefully reduces mistakes, but it doesn't eliminate the bias of the person who wrote the change also being the one who approves it — it's the same person validating their own decision. A pull request with someone else's review, a mandatory gate before merging (Module 3 and Module 6), adds a genuinely different point of view, not just a second reading by the same person.
Exercises
Exercise 1 — Audit your own history. Without looking at any log, try to answer: of the apply runs you ran in terraform-and-iac-guide, could you say today, with certainty, the exact date and business reason for each one? If you can't, where would that information live if the same work had been done through a pull request?
See solution
Most people won't be able to reconstruct the exact date and reason for each apply without actively digging through their own shell history (if they even kept it) — and that's precisely the point of the exercise: that information existed only in the moment you ran the command, not in any persistent, structured artifact. If the same work were done via pull request, that information would live in the PR itself: the title, the description, the review comments, and the merge date — all searchable, forever, without depending on anyone's memory.
Exercise 2 — Explain the difference between "preventing" and "auditing" to a skeptical colleague. A colleague tells you: "if a pipeline doesn't stop someone from approving a dangerous destroy, then it's useless — the final result is the same disaster." Answer them in three or four sentences, using this lesson's exact distinction.
See solution
A complete answer sounds, roughly, like this: "You're right that the pipeline doesn't change the decision of the person approving — if someone decides to apply a dangerous change after seeing the warning, the pipeline isn't going to magically stop them. But 'the final result is the same' is only true for the destroyed infrastructure, not for what happens afterward: without a pipeline, nobody can reliably reconstruct who approved what and when; with a pipeline, that record exists, it's immutable, and it radically changes how fast the root cause can be diagnosed and how seriously anyone on the team takes the next approval. Preventing human error is a culture and judgment problem; leaving an auditable record is an engineering problem, and that one a pipeline does solve."
Exercise 3 — Design the question a pipeline can actually answer. Write three concrete questions about an infrastructure change that would be impossible to answer with confidence if andes-cargo-infra/ were only ever applied by hand, and that a pipeline with mandatory PR, published plan, and chained apply.yml can answer with certainty.
See solution
Three reasonable questions: (1) "Who, exactly, approved the change that modified the andes-cargo-shipment-docs bucket's policy last week?" — with a pipeline, the answer is the PR's author and whoever approved it, with a date, in GitHub's history. (2) "Is the apply that ran in production exactly the same plan that was reviewed, or did something different get recalculated between the review and the application?" — with a pipeline (Module 5, needs + artifacts), the answer is verifiable: the same plan file travels from one job to the next. (3) "Was there any direct change, outside this process, that nobody reviewed?" — with a pipeline and scheduled drift detection (Module 5), the answer is a periodic plan that compares real state against what's declared, instead of "there's no way to know until someone notices by accident."
Summary and next step
In this lesson you precisely named the problem underlying the rest of this guide: a terraform apply run by hand leaves no record of who ran it, exposes long-lived credentials on a person's machine, and depends on that specific person being available when needed. You also revisited terraform-and-iac-guide's Claude Code destroy incident from a new angle: a pipeline wouldn't have prevented the negligence —the human approved after seeing the risk— but it would have left an auditable record of that approval, which in the real incident didn't exist explicitly.
Before moving on you should be able to: name the three questions a manual apply can't answer with confidence; explain the difference between "preventing" and "auditing" an infrastructure change; and describe, in one sentence, what a pipeline adds over the manual apply you already master.
Lesson 3 gives the solution a formal name: what continuous integration, continuous delivery, and continuous deployment are, exactly — three things that share an acronym and that this lesson hasn't distinguished yet.
Resources
- Hacker News — discussion of incident #47278720 — the original thread for the
destroyincident cited in this lesson. - incidentdatabase.ai — Incident 1424 — the structured record of the incident in the public AI incident database.
terraform-and-iac-guide, Module 8, lesson 5 — the full source of thedestroycase study, with the technical detail of how thestategot replaced.- AWS Well-Architected Framework — Operational Excellence — the official AWS pillar framing why deployment automation and auditability are an operational requirement, not an aesthetic preference.