Module 8: Capstone The Andes Cargo Security Gate
2. Architecture review: the full security gate
Description
Before touching a single line of YAML, this lesson draws the complete map: where each new control lives, in which file, triggered by which event, and what exactly happens when one of them fails. It's a reading lesson, not an execution one — the map you're going to build here is the one lesson 3 turns into a real ci.yml.
Connection to the module
cicd-and-gitops-on-aws-guide left two files — ci.yml (triggers on every Pull Request, runs terraform plan) and apply.yml (triggers on every push to main, runs terraform apply). This guide's Modules 5 and 6 already touched both: Module 5, lesson 7, added a Trivy step inside ci.yml's terraform-checks job; Module 6, lesson 8, added a complete verify-artifact job to apply.yml. This capstone module does something different from both: it reorganizes the policy control and the scan control into their own named jobs, chained with needs:, inside ci.yml — the complete gate, visible as a single chain, not as scattered steps inside a job with a different purpose.
Analogy: the airport's blueprint, before building the counters
Lesson 1 compared the gate to an airport's chained checkpoints. This lesson is that airport's architectural blueprint: before installing the first metal detector, someone decides where each checkpoint goes, what happens if someone doesn't pass it, and why the order is that one and not another. A poorly thought-out blueprint would put passport control after luggage control — technically it works, but it wastes the time of every passenger whose passport wasn't valid anyway. This lesson is that planning exercise, done deliberately before lesson 3.
The complete diagram: plan → conftest → Trivy → cosign → apply
PULL REQUEST (ci.yml)
┌──────────────────────────────────────────────────────────────────────┐
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌───────────────────┐ │
│ │ terraform │ │ conftest │ │ Trivy │ │
│ │ plan + show │ ───► │ policy-check│ ───► │ iac-scan │ │
│ │ -json │ │ (Module 4) │ │ (Module 5) │ │
│ └─────────────┘ └──────┬──────┘ └──────────┬─────────┘ │
│ │ FAIL │ FAIL │
│ ▼ ▼ │
│ 🛑 stops here 🛑 stops here │
│ │
│ ┌───────────────┐ │
│ │ cosign │ │
│ │verify-artifact│ │
│ │ (Module 6) │ │
│ └───────┬───────┘ │
│ │ FAIL │
│ ▼ │
│ 🛑 stops here │
└──────────────────────────────────────────────────────────────────────┘
│ all three PASS
▼
MERGE to main → push (apply.yml)
┌──────────────────────────────────────────────────────────────────────┐
│ fetch-reviewed-plan ──┐ │
│ ├──► needs: [both] ──► terraform-apply │
│ verify-artifact ───────┘ (Module 6.8, already inherited) │
│ (second verification, defense in depth) │
└──────────────────────────────────────────────────────────────────────┘
Two observations that govern everything that follows:
verify-artifact appears twice, and it isn't a mistake. Module 6, lesson 8, already added a verify-artifact job to apply.yml, running right before terraform-apply — the last possible check, at the exact moment before something gets deployed. This module adds the same verification, earlier, inside ci.yml, as part of the Pull Request chain. It isn't wasted redundancy: it's defense in depth, the same principle you already saw with conftest/Trivy evaluating different layers of the same plan. Verifying the signature in the PR gives the reviewing team an early signal ("this artifact is already broken, don't even bother approving this PR"); verifying again in apply.yml protects against the unlikely but real case that something changes between the PR moment and the merge moment.
Every 🛑 is literal, not decorative. needs: in GitHub Actions — and in act, which respects it with the same semantics — means a job doesn't trigger at all if the one it needs failed. It's not that it runs and fails fast: it doesn't run. This module's lesson 5 confirms it with a real log where iac-scan and verify-artifact don't appear even once, at any point, after policy-check fails.
The same diagram, as a sequential flow
flowchart LR
A[terraform plan] --> B[terraform show -json]
B --> C{policy-check<br/>conftest}
C -->|FAIL| X1[🛑 stopped]
C -->|PASS| D{iac-scan<br/>Trivy}
D -->|FAIL| X2[🛑 stopped]
D -->|PASS| E{verify-artifact<br/>cosign}
E -->|FAIL| X3[🛑 stopped]
E -->|PASS| F[merge to main]
F --> G[apply.yml: verify-artifact + terraform-apply]
ci.yml, before and after this module
cicd-and-gitops-on-aws-guide left ci.yml with a single job, terraform-checks, with nine steps running in sequence inside that one job. This guide's Module 5 inserted two more steps (install Trivy, run trivy config) inside that same job, between Terraform format check and Terraform init. This module changes the shape, not just the content:
| Before this module | After this module | |
|---|---|---|
Jobs in ci.yml | 1 (terraform-checks, with Trivy as two internal steps) | 4 (policy-check, iac-scan, verify-artifact, plus terraform-checks unchanged) |
| Trivy lives in... | A step inside terraform-checks | Its own job, iac-scan |
conftest lives in... | Nowhere in ci.yml — only run by hand in M4 | Its own job, policy-check |
cosign lives in... | Only in apply.yml (M6.8) | In apply.yml and in its own ci.yml job, verify-artifact |
| Visibility in the GitHub Actions interface | A single success/failure icon for the whole job | Four independent icons, one per control |
Why separate Trivy from terraform-checks instead of leaving it where Module 5 put it? The same reason Module 6, lesson 8's Exercise 2 already anticipated about verify-artifact: a separate job gives independent visibility — if iac-scan fails, it shows up as a scanning failure, not "the Terraform job failed, we need to investigate which of its fifteen steps it was" — and enables real parallelism where needs: allows it. terraform-checks (the original job, with fmt/init/validate/plan) keeps existing unchanged — this module doesn't touch it, only adds the three new jobs alongside it.
What exactly each job checks
This isn't an empty repeat of M4/M5/M6 — it's the "what runs in CI" version of each, with the exact command lesson 3 is going to put inside each job:
| Job | Tool | What it evaluates | Central command |
|---|---|---|---|
policy-check | conftest 0.69.0 | Terraform's plan, converted to JSON | conftest test tfplan.json -p policy/ |
iac-scan | Trivy 0.74.0 | The raw HCL, no plan needed at all | trivy config --exit-code 1 --severity CRITICAL,HIGH . |
verify-artifact | cosign v3.1.3 | The deployment artifact (lambda/function.zip) against its signature | cosign verify-blob --key cosign.pub --bundle manifest.sig --insecure-ignore-tlog=true lambda/function.zip |
Notice a real asymmetry: policy-check needs terraform plan/terraform show -json to run first, inside the same job (they're its own prior steps) — it depends on the project's computed state. iac-scan needs no plan at all: it evaluates the HCL as it stands in the repository, with no Terraform step involved (the same reason, already explained in Module 5, lesson 7, why that control can run before terraform init). verify-artifact doesn't need Terraform at all either: it only needs lambda/function.zip, cosign.pub, and manifest.sig to exist in the repository, all three already committed since Module 6. Three controls, three completely different dependencies — the exact technical reason they can live in separate jobs instead of being forced into a single sequential job.
Common mistakes
Assuming needs: [policy-check, iac-scan] (a list) and needs: policy-check followed by needs: iac-scan in separate jobs are the same thing. What happens: someone, designing lesson 3's ci.yml from memory, writes verify-artifact with needs: [policy-check, iac-scan] instead of chaining it only to iac-scan (which itself depends on policy-check). How to spot it: with the list, verify-artifact would wait for both to finish — functionally equivalent in this specific case, because iac-scan already depends on policy-check, but conceptually different: a linear chain (A → B → C) communicates a sequence; a dependency list (C needs: [A, B]) communicates a sync point between parallel branches. How to fix it: for a sequential chain like this gate, each job depends only on the one that directly precedes it — iac-scan on policy-check, verify-artifact on iac-scan — never on all the earlier ones at once, exactly as lesson 3 builds it.
Forgetting that a job with no needs: at all runs in parallel with the others, not before them. What happens: someone adds a fourth job to this module's ci.yml, with no needs:, assuming "it's going to run after the other three because I wrote it further down in the file." How to spot it: if your reasoning about execution order is based on the job's position within the YAML file. How to fix it: GitHub Actions — and act — don't execute jobs in the order they appear in the file; they execute according to the dependency graph needs: explicitly declares. A job with no needs: starts as soon as the workflow triggers, in parallel with any other job with no needs:, regardless of where it's written in the file.
Thinking the second verify-artifact (in apply.yml) is redundant and can be removed. What happens: someone, noticing ci.yml already verifies the signature in the PR, proposes removing the equivalent job from apply.yml to "not repeat work." How to spot it: if your argument is "it was already verified once, no need again." How to fix it: this lesson's defense-in-depth section already explains it — they're two different moments (PR review vs. right before deploying), and the cost of running cosign verify-blob again (seconds) is negligible compared to the risk of deploying an artifact that changed between those two moments without anyone noticing.
Exercises
Exercise 1 — Sketch, in text, what would happen if iac-scan didn't have needs: policy-check. Without looking at lesson 3's ci.yml yet, describe the pipeline's behavior if iac-scan declared no dependency: in what order would the three jobs run, and what would happen if policy-check fails?
See solution
Without needs:, all three jobs (policy-check, iac-scan, verify-artifact) would run in parallel, all triggered at the same time by the pull_request event, with no relationship between them. If policy-check fails, iac-scan and verify-artifact would keep running anyway — already in progress, with no signal stopping them — each finishing with its own independent result. The complete pipeline would still report as failed (GitHub Actions marks the workflow as failed if any job fails), but runner time would have been spent on iac-scan and verify-artifact over a change policy-check had already rejected anyway — exactly the waste lesson 1's "why order matters" section identified, now confirmed with the precise technical mechanism that prevents it.
Exercise 2 — Explain why terraform-checks (the original job, with Trivy as an internal step since Module 5) doesn't get removed or merged with iac-scan. A colleague asks: if iac-scan already runs Trivy in its own job, why leave the Trivy step duplicated inside terraform-checks too?
See solution
It actually doesn't stay duplicated if done correctly: this module's lesson 3 moves the Trivy step out of terraform-checks into the new iac-scan job, it doesn't copy it. terraform-checks keeps fmt/init/validate/plan — the work that does need Terraform's complete engine, including the representative connection to LocalStack for awslocal s3 ls — while iac-scan is left with just the scan, which never needed that engine in the first place. Merging them back into one job would lose exactly the advantage this lesson's "why separate Trivy...?" section already explained: independent visibility and the ability for the gate to fail fast, in the cheapest job, without dragging along all the Terraform work behind it.
Exercise 3 — Predict the result of running act pull_request -j verify-artifact directly (scoped to a single job), without having run policy-check or iac-scan first. Does act respect needs: when you ask for one specific job with -j, or does it run it anyway?
See solution
act -j <job> runs only the requested job, ignoring its declared needs: dependencies — it's, deliberately, a way to isolate a job for debugging, the same one Module 6, lesson 8 already used (act push -j verify-artifact -W .github/workflows/apply.yml) to test that job in isolation. This means act pull_request -j verify-artifact would run that job with neither policy-check nor iac-scan having even executed, giving a potentially misleading result if you confuse it with a real complete-pipeline run. To observe the real behavior of the chained needs: — the one this module's lesson 5 depends on demonstrating — you need to run act pull_request without the -j flag, letting act resolve the complete dependency graph on its own.
Summary and next step
This lesson drew the complete map before building anything: three new jobs (policy-check, iac-scan, verify-artifact) chained with needs: inside ci.yml, each with a different dependency (the plan, the raw HCL, the committed artifact), and a fourth piece — apply.yml's verify-artifact, inherited from Module 6 — that isn't touched, but understood as defense in depth, not redundancy. You confirmed, with the diagram and the before/after table, why Trivy moves from an internal step to its own job, and why the three new jobs' order — policy before scanning, scanning before signature — follows the same increasing-cost criterion you already saw in Module 5.
Lesson 3 turns this diagram into real YAML, and runs it with act pull_request against andes-cargo-infra/.
Resources
- GitHub Docs —
jobs.<job_id>.needs— the official reference for the job-dependency mechanism that governs this entire module. - This course, Module 5, lesson 7 — the origin of the "fail fast, cheapest step first" criterion this module applies one level up, to complete jobs.
- This course, Module 6, lesson 8 — the
apply.ymlwithverify-artifactthis module doesn't modify, only references as defense in depth. cicd-and-gitops-on-aws-guide, Module 3, lesson 8 — the original nine-stepci.yml, this module's exact starting point.