Module 8: Capstone The Andes Cargo Pipeline

4. End-to-end walkthrough: a rejected change

Next →

Description

Lesson 3 proved the pipeline approves and applies a legitimate change, end-to-end, with nobody having to intervene by hand in the middle. This lesson proves the other half of the thesis, the most important one for any real security system: that the pipeline stops a dangerous change, also end-to-end, without depending on a person noticing in time. You're going to reproduce Module 6's complete scenario —seeding the state with the fixture that declares Shipments as already existing, and proposing to delete it— but this time as this guide's final proof: confirmation that the safety net still works under the same narrative pressure that already ran through the rest of the pipeline in lesson 3, not just in the isolated example where it was first built.

Connection to the module

This lesson reuses, without changing a single line, Module 6's guardrail and fixture (lesson 7) and the project that integrated them (that module's lesson 8). It builds no new mechanism — its value is exactly confirming it, a third time, within this guide's final walkthrough. Lesson 5 changes nature entirely: from here on, the rest of the module is boundary honesty, with no more execution.

Analogy: the sprinkler system's fire test

A fire sprinkler system gets installed, tested in a controlled environment, and certified — but the test that really matters, the one an inspector requires before approving a building, is the complete simulation: a test fire, under conditions as realistic as possible, with the complete system active, not just the isolated sprinkler on a lab table. This lesson is that fire test. The guardrail already passed its tabletop test in Module 6. Here it gets tested inside the complete building, with the rest of the pipeline —all of ci.yml, not just the guardrail step— around it.

Step 1 — The scenario: someone tries to delete Shipments by mistake

Pick back up exactly Module 6's lesson 7 scenario: someone at Andes Cargo, mistakenly thinking the Shipments table is going to be replaced by a new service (outside this guide's scope), opens a Pull Request that removes the resource entirely. Before touching any file, back up the project's current state so you can restore it precisely afterward:

cd andes-cargo-infra
git status

What to expect (literal):

On branch main
nothing to commit, working tree clean

Confirming you're starting from a clean state is just as important as the guardrail itself — this module's lesson 8 assumes andes-cargo-infra/ ends up exactly like this at this lesson's close.

Step 2 — Seeding the fixture and proposing the destruction

Module 6's fixture is right where you left it, with no change:

ls .github/guardrail-fixtures/

What to expect (literal):

shipments-already-applied.state.json

Empty out dynamodb.tf entirely —the table and the IAM policy that depended on it, in the same file— and remove the output that depended on that resource, exactly like in Module 6, lesson 7:

# dynamodb.tf — the Shipments table was decommissioned here by mistake. The
# file is intentionally left empty so the diff is exactly what this lesson
# describes: the resource is removed, not the whole file.

terraform validate

What to expect (literal):

Success! The configuration is valid.

Step 3 — Running the guardrail inside the complete walkthrough

Seed the fixture and run Module 6's same dedicated workflow —guardrail-demo.yml, deliberately separate from ci.yml, so no real Pull Request can push a fake state into the project—:

rm -f terraform.tfstate
act workflow_dispatch -j destroy-shipments-check -W .github/workflows/guardrail-demo.yml

What to expect (literal output, executed to write this lesson):

[guardrail-demo/destroy-shipments-check] ⭐ Run Set up job
[guardrail-demo/destroy-shipments-check]   ✅  Success - Set up job
[guardrail-demo/destroy-shipments-check] ⭐ Run Main Check out andes-cargo-infra
[guardrail-demo/destroy-shipments-check]   ✅  Success - Main Check out andes-cargo-infra [36.931458ms]
[guardrail-demo/destroy-shipments-check] ⭐ Run Main Set up Terraform
[guardrail-demo/destroy-shipments-check]   ✅  Success - Main Set up Terraform [2.667890417s]
[guardrail-demo/destroy-shipments-check] ⭐ Run Main Terraform init
[guardrail-demo/destroy-shipments-check]   ✅  Success - Main Terraform init [14.8967295s]
[guardrail-demo/destroy-shipments-check] ⭐ Run Main Seed the state with the fixture (Shipments already exists)
[guardrail-demo/destroy-shipments-check]   ✅  Success - Main Seed the state with the fixture (Shipments already exists) [755.256083ms]
[guardrail-demo/destroy-shipments-check] ⭐ Run Main Install tflocal
[guardrail-demo/destroy-shipments-check]   ✅  Success - Main Install tflocal [5.80159425s]
[guardrail-demo/destroy-shipments-check] ⭐ Run Main Terraform plan (offline, against the seeded fixture)
[guardrail-demo/destroy-shipments-check]   | Plan: 10 to add, 0 to change, 1 to destroy.
[guardrail-demo/destroy-shipments-check]   ✅  Success - Main Terraform plan (offline, against the seeded fixture) [4.740371625s]
[guardrail-demo/destroy-shipments-check] ⭐ Run Main Guardrail — block any plan that destroys the Shipments table
[guardrail-demo/destroy-shipments-check]   ❗  ::error::Guardrail failed: this plan destroys aws_dynamodb_table.shipments (Shipments). Blocking before apply.
[guardrail-demo/destroy-shipments-check]   | "address":"aws_dynamodb_table.shipments","mode":"managed","type":"aws_dynamodb_table","name":"shipments","provider_name":"registry.terraform.io/hashicorp/aws","change":{"actions":["delete"]
[guardrail-demo/destroy-shipments-check]   ❌  Failure - Main Guardrail — block any plan that destroys the Shipments table [2.123080458s]
[guardrail-demo/destroy-shipments-check] ⭐ Run Complete job
[guardrail-demo/destroy-shipments-check]   ✅  Success - Complete job
[guardrail-demo/destroy-shipments-check] 🏁  Job failed
Error: Job 'destroy-shipments-check' failed

Plan: 10 to add, 0 to change, 1 to destroy. — the 1 to destroy is the exact signal the guardrail exists to catch: the Shipments table, seeded as already existing by the fixture, and absent from the HCL after Step 2. The guardrail found the exact line —"actions":["delete"]— and stopped the job with exit 1, before any chance existed for this plan to ever become an artifact apply.yml could download.

What did NOT happen, and why it's this lesson's most important part

Notice what the log above does not show, with the same attention you already practiced in Module 5 for reading an honest failure:

  • No Upload the plan for apply.yml to use later step ran. That step lives, in ci.yml, after the guardrail — a job that fails never reaches later steps, not under act, not on real GitHub.
  • No artifact called terraform-plan got uploaded in this run. Confirm it directly:
ls .artifacts/ 2>/dev/null || echo "no .artifacts directory for this run"

What to expect (literal, if you ran this lesson without reusing lesson 3's .artifacts/ folder):

no .artifacts directory for this run
  • apply.yml, if someone triggered it by hand right now, would have nothing to download. It's not that apply.yml "decides" not to apply this change — it's that, technically, there's no terraform-plan artifact from this run to point at. The absence of an action is, here, just as real as the guardrail's exit 1: two independent layers, the explicit one (the guardrail blocking) and the structural one (nothing to apply), protecting the same resource.

This is, with evidence and not just with lesson 2's diagram's logic, the complete proof of this module's thesis: Andes Cargo's safety net works under the same narrative pressure —a numbered Pull Request, a named branch, a change that looks reasonable at first glance— that ran through lesson 3's accepted change, not just in the isolated, controlled example where Module 6 first built it.

Restoring the project

git checkout -- dynamodb.tf outputs.tf
rm -f terraform.tfstate tfplan plan.json
git status

What to expect (literal):

On branch main
nothing to commit, working tree clean

git checkout -- <file> restores each file to its committed version — exactly right for a change that was never approved, never merged, and should never have touched main. The project ends up, again, in the same clean state from Step 1, ready for lesson 5.

Common mistakes

Expecting apply.yml to run and fail "a different way" in this lesson (expectation-based). What happens: someone, after seeing the guardrail block the change, tries running act push -W apply.yml anyway, expecting to see some message related to the Shipments table. How to fix it: there's no terraform-plan artifact from this run — apply.yml, run at this point, would fail at its first step (fetch-reviewed-plan) with an artifact-not-found error, with no relation to the rejected change's content. This lesson's proof ends exactly where it ends: at the guardrail, inside ci.yml/guardrail-demo.yml, before apply.yml ever enters the picture.

Forgetting to restore dynamodb.tf/outputs.tf before moving on to lesson 5 (workflow-based, revisit Module 6). What happens: someone runs this complete lesson, sees the guardrail block the change, and moves on without running the git checkout -- step. How to spot it: git status would show uncommitted changes, or the project would have the Shipments table permanently removed. How to fix it: always close this lesson with the project in a clean state — this module's lesson 8, the final project, assumes andes-cargo-infra/ is exactly as this lesson left it.

Thinking this lesson tests something different from Module 6 (scope-based). What happens: someone expects a new mechanism, different from the guardrail already built. How to fix it: this lesson, on purpose, introduces no new mechanism — its value is confirming, for the third time in this guide (Module 6, lesson 7; Module 6, lesson 8; and now), that the exact same guardrail still works, integrated into the capstone's complete walkthrough, not just in the example where it was built.

Exercises

Exercise 1 — Compare this module's two walkthroughs, side by side. Without looking back, describe the three most important differences between lesson 3's walkthrough (accepted change) and this one's (rejected change), in terms of which steps ran and which didn't.

See solution

(1) In lesson 3, ci.yml's twelve steps ran completely, including Upload the plan for apply.yml to use later; in this lesson, the job stopped at the guardrail step, and no later step ran. (2) In lesson 3, apply.yml genuinely ran (fetch-reviewed-plan succeeded, terraform-apply made a real attempt that only failed for lack of LocalStack); in this lesson, apply.yml never triggered, because there was no artifact to download. (3) In lesson 3, Plan: showed 0 to destroy; in this lesson, it showed 1 to destroy — the signal that, in both cases, the guardrail correctly evaluated before deciding whether to let the change through or block it.

Exercise 2 — Explain why the fixture is still needed in this module, not just Module 6. A colleague asks why this lesson can't simply try removing Shipments from the HCL against this project's real state, with nothing seeded. Answer them with the exact reason.

See solution

This project's state, on this machine, without a valid LOCALSTACK_AUTH_TOKEN, never had a real apply completed — it's still completely empty, the same structural limitation confirmed since Module 3 and Module 5. Without the fixture, removing the table from the HCL on an empty state doesn't produce any "delete" action — it simply produces "one less thing to create," because nothing was ever applied to destroy. The fixture remains, in this module as in Module 6, the only honest way to generate a plan with genuine destruction against this specific LocalStack, with no need for real, applied infrastructure.

Exercise 3 — Design the variant where the guardrail SHOULD fail to catch the change. Describe an HCL change, different from this lesson's, that the current guardrail —as written— would not detect, even though it's also dangerous for the Shipments table.

See solution

The guardrail specifically searches for the text "delete" within the 200 characters following "address":"aws_dynamodb_table.shipments" in the plan's JSON (Module 6, lesson 7). A change the current guardrail wouldn't detect: renaming the Terraform resource (for example, from aws_dynamodb_table.shipments to aws_dynamodb_table.shipments_v2) while keeping the same name = "Shipments" in its arguments — Terraform would treat this as destroying the old resource and creating a new one under a different address (aws_dynamodb_table.shipments_v2), and the grep, which looks for the exact address aws_dynamodb_table.shipments, would never find the associated "delete" action, because that action would show up under an address the guardrail doesn't know about. It's exactly the kind of edge case Module 6's lesson 6 already named as the underlying reason a real policy system (conftest/Rego, evaluating the complete data structure) is more robust than a text grep.

Summary and next step

In this lesson you reproduced Module 6's scenario —the seeded state fixture, the Shipments table removed from the HCL— inside this guide's final walkthrough. The guardrail blocked the change with Plan: 10 to add, 1 to destroy followed by Job failed, exactly like in its original example, confirming no artifact got uploaded and apply.yml never had a chance to run. You restored the project to a clean state, ready for the three boundary-honesty lessons that follow.

Before moving on you should be able to: explain, unaided, the three differences between the accepted walkthrough (lesson 3) and the rejected one (this lesson); justify why the fixture is still needed in this module; and describe a kind of change the current guardrail, as written, wouldn't detect.

With both halves of the final technical proof complete —one change that goes through, one that doesn't—, lesson 5 closes this guide's execution honesty: which part of this whole process you can only live with a real GitHub account.

Resources

  1. Terraform Docs — Command: state push — official reference for the command that seeds the fixture reused in this lesson.
  2. Terraform Internals — JSON Output Format — the JSON plan's schema the guardrail evaluates, with the resource_changes[].change.actions field at the center of this test.
  3. This guide's Module 6 (07-hands-on-a-failing-guardrail-example.md) — the guardrail's and fixture's exact origin, reused unchanged in this lesson.
  4. This guide's Module 6 (08-project-andes-cargos-safety-net.md) — the sister project that first integrated the guardrail into the complete pipeline, the same pattern this lesson confirms once more.
← Previous3. End-to-end walkthrough: a real change crossing the pipelineNext →5. What only a real GitHub account shows