Module 8: Capstone The Andes Cargo Security Gate

3. Hands-on: chaining the gate into `ci.yml`

Next →

Description

This lesson turns lesson 2's diagram into real YAML, and runs it for real, twice, with act pull_request against a real Docker runner. The first run confirms the three jobs exist and are correctly chained; the rest of the module (lessons 4 and 5) is going to reuse this exact same ci.yml against two different changes — one that passes, one that doesn't. Everything you see here ran to write this lesson, with the same runner image (catthehacker/ubuntu:act-latest) cicd-and-gitops-on-aws-guide and this guide's Modules 5 and 6 already used.

Isolation note. As in Module 5, lesson 7, and Module 6, lesson 8, this extended ci.yml runs on a new, disposable lab (its own git init), not on the andes-cargo-infra/ you've been accumulating since Module 1 — act running against Docker touches .git/ in ways you don't want mixed with your main project.

Connection to the module

This guide's lessons 4 through 6 already tested conftest, Trivy, and cosign in isolation, each by hand, in your terminal. This lesson doesn't repeat any of those tests — it takes the three already-verified commands and puts each one inside its own ci.yml job, with needs: declaring the exact chain lesson 2 drew.

Analogy: installing the airport's checkpoints, one by one, and testing the complete chain

Lesson 2 was the blueprint. This is the construction: install the first checkpoint, test it works alone; install the second, test the first feeds it correctly; install the third, test the complete chain end to end. No real airport tests its three checkpoints for the first time on opening day — it tests them, together, before the first passenger arrives. This lesson is that acceptance test.

Step 1 — The lab, with M1-M7's inherited pieces

mkdir andes-cargo-infra && cd andes-cargo-infra
git init -b main

Copy the complete HCL, hardened end to end by Modules 1 through 7 (the root files, modules/, lambda/, policy/, .trivyignore), and Module 6's three supply-chain artifacts (sbom.cyclonedx.json, cosign.pub, manifest.sig — cosign.key stays out of the repository, gitignored since Module 6, lesson 5):

cat > .gitignore <<'EOF'
cosign.key
tfplan
tfplan.json
.terraform/
EOF

cat > .actrc <<'EOF'
-P ubuntu-latest=catthehacker/ubuntu:act-latest
EOF

git add -A && git commit -m "Bootstrap the extended security gate: policy-check + iac-scan + verify-artifact chained in ci.yml"

Step 2 — ci.yml: three new jobs, each with its needs:

This is the ci.yml inherited from cicd-and-gitops-on-aws-guide (the original terraform-checks job, unchanged) with three new jobs added alongside. Notice the Trivy step: it moves from inside terraform-checks (where Module 5, lesson 7, had put it) into its own job, iac-scan — exactly the change lesson 2 justified.

name: ci

on:
  pull_request:
    branches: [main]

jobs:
  policy-check:
    runs-on: ubuntu-latest
    steps:
      - name: Check out andes-cargo-infra
        uses: actions/checkout@v4

      - name: Set up Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: "1.15.8"

      - name: Terraform init
        run: terraform init -input=false

      - name: Terraform plan
        run: terraform plan -out=tfplan -input=false

      - name: Convert plan to JSON
        run: terraform show -json tfplan > tfplan.json

      - name: Install conftest
        run: |
          curl -sL -o conftest.tar.gz \
            https://github.com/open-policy-agent/conftest/releases/download/v0.69.0/conftest_0.69.0_Linux_x86_64.tar.gz
          tar -xzf conftest.tar.gz conftest
          sudo mv conftest /usr/local/bin/

      - name: Evaluate the policy library against the plan
        run: conftest test tfplan.json -p policy/

  iac-scan:
    needs: policy-check
    runs-on: ubuntu-latest
    steps:
      - name: Check out andes-cargo-infra
        uses: actions/checkout@v4

      - name: Install Trivy
        run: |
          curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin v0.74.0

      - name: IaC security scan (Trivy)
        run: trivy config --exit-code 1 --severity CRITICAL,HIGH .

  verify-artifact:
    needs: iac-scan
    runs-on: ubuntu-latest
    steps:
      - name: Check out andes-cargo-infra
        uses: actions/checkout@v4

      - name: Install envsubst (required by the cosign installer, missing on act's runner image)
        run: apt-get update -qq && apt-get install -y -qq gettext-base

      - name: Install cosign
        uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6

      - name: Verify function.zip against manifest.sig
        run: |
          cosign verify-blob \
            --key cosign.pub \
            --bundle manifest.sig \
            --insecure-ignore-tlog=true \
            lambda/function.zip

Three decisions, none arbitrary:

  • policy-check runs terraform init/plan from scratch, inside the same job. Unlike terraform-checks (which also runs them, for its own purpose), policy-check needs its own copy of the plan because conftest can't evaluate anything without the JSON terraform show -json produces. It's duplicated work in appearance — two jobs running terraform init — but it's the correct price for each job being independent and able to run on a different runner, sharing no state with each other (the alternative, passing the tfplan from one job to another with actions/upload-artifact/download-artifact, is exactly the pattern ci.yml/apply.yml already use between different workflows — within the same workflow, each job in this gate deliberately stays self-sufficient).
  • sudo mv conftest /usr/local/bin/, a new detail in this module. Module 4, lesson 3, installed conftest in the student's working directory, where it was already on that terminal session's effective PATH. Inside a GitHub Actions runner, the PATH doesn't include the working directory by default — moving it to /usr/local/bin/ (which is on the PATH) is the step that makes the next step, conftest test, find the binary without needing a relative path.
  • The Install envsubst step is identical, line by line, to the one Module 6, lesson 8, already documented for apply.yml. It's the same real incompatibility between act's medium image and sigstore/cosign-installer — it repeats here because verify-artifact is, literally, the same job that module wrote, now also in ci.yml.

Step 3 — First run: the three jobs, chained, all green

act pull_request -e .github/act-events/pr-event.json

What to expect (literal — run to write this lesson, with real Docker and the catthehacker/ubuntu:act-latest image; trimmed to the lines showing progress and result, filtering the repeated docker pull/docker exec noise act prints on every step):

[ci/policy-check] ⭐ Run Main Terraform init
[ci/policy-check]   ✅  Success - Main Terraform init [16.926710125s]
[ci/policy-check] ⭐ Run Main Terraform plan
[ci/policy-check]   ✅  Success - Main Terraform plan [5.83613625s]
[ci/policy-check] ⭐ Run Main Convert plan to JSON
[ci/policy-check]   ✅  Success - Main Convert plan to JSON [2.367656084s]
[ci/policy-check] ⭐ Run Main Install conftest
[ci/policy-check]   ✅  Success - Main Install conftest [2.035634042s]
[ci/policy-check] ⭐ Run Main Evaluate the policy library against the plan
[ci/policy-check]   | 
[ci/policy-check]   | 4 tests, 4 passed, 0 warnings, 0 failures, 0 exceptions
[ci/policy-check]   ✅  Success - Main Evaluate the policy library against the plan [267.45375ms]
[ci/policy-check] 🏁  Job succeeded

[ci/iac-scan    ] ⭐ Run Main Install Trivy
[ci/iac-scan    ]   ✅  Success - Main Install Trivy [4.488681208s]
[ci/iac-scan    ] ⭐ Run Main IaC security scan (Trivy)
[ci/iac-scan    ]   | Report Summary
[ci/iac-scan    ]   | ┌───────────────────────────┬───────────┬───────────────────┐
[ci/iac-scan    ]   | │          Target           │   Type    │ Misconfigurations │
[ci/iac-scan    ]   | ├───────────────────────────┼───────────┼───────────────────┤
[ci/iac-scan    ]   | │ .                         │ terraform │         0         │
[ci/iac-scan    ]   | ├───────────────────────────┼───────────┼───────────────────┤
[ci/iac-scan    ]   | │ dynamodb.tf               │ terraform │         0         │
[ci/iac-scan    ]   | ├───────────────────────────┼───────────┼───────────────────┤
[ci/iac-scan    ]   | │ lambda.tf                 │ terraform │         0         │
[ci/iac-scan    ]   | ├───────────────────────────┼───────────┼───────────────────┤
[ci/iac-scan    ]   | │ modules/s3-bucket/main.tf │ terraform │         0         │
[ci/iac-scan    ]   | ├───────────────────────────┼───────────┼───────────────────┤
[ci/iac-scan    ]   | │ secrets.tf                │ terraform │         0         │
[ci/iac-scan    ]   | └───────────────────────────┴───────────┴───────────────────┘
[ci/iac-scan    ]   ✅  Success - Main IaC security scan (Trivy) [1.837377541s]
[ci/iac-scan    ] 🏁  Job succeeded

[ci/verify-artifact] ⭐ Run Main Install cosign
[ci/verify-artifact]   ✅  Success - Main Install cosign [4.038418125s]
[ci/verify-artifact] ⭐ Run Main Verify function.zip against manifest.sig
[ci/verify-artifact]   | WARNING: Skipping tlog verification is an insecure practice that lacks transparency and auditability verification for the blob.
[ci/verify-artifact]   | Verified OK
[ci/verify-artifact]   ✅  Success - Main Verify function.zip against manifest.sig [87.011458ms]
[ci/verify-artifact] 🏁  Job succeeded

Three 🏁 Job succeeded, in the chain's exact order. act respected needs: with no need to be explicitly told: it started policy-check first (no declared dependency), waited for its success, started iac-scan, waited for its own, and only then started verify-artifact. The same 4 tests, 4 passed you already saw running conftest by hand in Module 4, the same Misconfigurations: 0 across the five files you already saw with Trivy in Module 5 (after that lesson's .trivyignore was calibrated), the same Verified OK from Module 6 — all three, now, running inside a real pipeline, not in your terminal.

A real detail worth documenting: tfplan.json confuses Trivy if it's left in the directory

While preparing this lesson, running trivy config . outside a CI job, directly against andes-cargo-infra/, with tfplan.json still present in the directory from an earlier terraform plan, produced this:

ERROR   [terraform parser] Error parsing file  module="root" file_path="main.tf" cause="<nil>" err="main.tf:137,7-8: Invalid expression..."

A confusing error — it mentions main.tf, a file that doesn't even exist in this project — because Trivy, when scanning a directory, tries to automatically detect a Terraform plan snapshot alongside the .tf files, and tfplan.json matches that pattern closely enough for Trivy to try parsing it as if it were HCL. The final result (Misconfigurations: 0 on the real files) doesn't change, but the error message is pure noise, with no relation to a real finding. Inside iac-scan, this problem never shows up — the job does a clean checkout on every run, with no leftover tfplan.json from an earlier terraform plan —, but it's worth knowing about if you ever run trivy config . by hand, in the same directory where you already ran terraform plan -out=tfplan.

Common mistakes

Installing conftest/Trivy/cosign into the runner's working directory, without moving them to a PATH directory. What happens: someone copies the corresponding lesson's install command (Module 4, 5, or 6) as-is, without the extra sudo mv .../usr/local/bin/ a CI runner needs. How to spot it: the install step finishes successfully, but the next step (conftest test, for example) fails with conftest: command not found. How to fix it: on your local terminal, the working directory is usually on the session's effective PATH; inside an Actions runner, it isn't by default — always move the downloaded binary to /usr/local/bin/ (or add the directory to PATH with echo "$dir" >> $GITHUB_PATH) before using it in a later step.

Confusing needs: policy-check (a single name) with needs: [policy-check] (a one-element list). What happens: someone, reviewing this ci.yml's YAML syntax, wonders whether needs: iac-scan (no brackets) works the same as needs: [iac-scan]. How to spot it: both forms are valid and equivalent in the GitHub Actions specification — it isn't an error, but it can raise unnecessary doubts when reading the file. How to fix it: use the bracketless form when depending on a single job (as in this ci.yml), and the list form only when depending on more than one — it's a readability convention, not a functional difference, but keeping it consistent helps anyone reading the file tell, at a glance, a simple dependency from a real sync point.

Running act pull_request without the -e flag and being surprised by an empty event. What happens: someone runs act pull_request without specifying .github/act-events/pr-event.json, and act synthesizes a minimal Pull Request event, without the fields a real workflow might expect (PR number, base branch, etc.). How to spot it: if your workflow doesn't use any of those fields — like this lesson's ci.yml, which doesn't need them — the result doesn't visibly change; but if some future job did need them, it would fail confusingly. How to fix it: always pass -e with an explicit event, even if the current ci.yml doesn't need it — it's the same discipline cicd-and-gitops-on-aws-guide already established, and it avoids surprises as the workflow grows.

Exercises

Exercise 1 — Run act pull_request -j iac-scan in isolation, and explain why it works without policy-check having run first. Using the -j flag you already saw in lesson 2, run only the iac-scan job. Why doesn't it fail, despite needs: policy-check being declared?

See solution

act -j <job> runs the requested job in isolation, ignoring any declared needs: — it's a debugging mode, not a faithful simulation of the complete pipeline (the same behavior lesson 2's Exercise 3 already predicted). iac-scan doesn't depend, at execution time, on any file or state policy-check produces — unlike verify-artifact inside the original apply.yml, which did need the tfplan downloaded from an earlier job — so it runs with no problem in isolation. This is useful for quickly debugging a specific job, but it does not confirm needs: is correctly chained — for that, the only valid test is running the complete workflow, without -j, like this lesson's Step 3.

Exercise 2 — Calculate the complete gate's approximate total time, adding up this lesson's three real "evaluation" times (not counting installation). Using the bracketed times from Step 3's output (Evaluate the policy library..., IaC security scan (Trivy), Verify function.zip...), what's the combined time of the three controls, not counting the time to install each tool?

See solution

267.45375ms (policy-check) + 1.837377541s (iac-scan) + 87.011458ms (verify-artifact) ≈ 2.19 seconds of real evaluation, across all three controls combined — a tiny fraction next to the 16.9 seconds terraform init alone took in the same job. The point of the exercise is noticing that this security gate's real cost, once the tools are installed, is nearly negligible — most of a real pipeline's time goes into installing dependencies and initializing Terraform, not evaluating the security policies themselves. It's a strong argument, with real numbers, against the common objection that "a security gate slows down the pipeline."

Exercise 3 — Design a hypothetical fourth job, sbom-check, that validates sbom.cyclonedx.json still exists and isn't empty, and decide where to chain it. Without writing the complete YAML, describe in prose: which job would it depend on (needs:), and why does that specific spot in the chain make sense under lesson 2's increasing-cost criterion?

See solution

A reasonable answer: sbom-check doesn't depend on any Terraform plan or any costly external tool — it only needs to confirm a file exists and has content (test -s sbom.cyclonedx.json, for example) —, so, following the increasing-cost criterion, it should run first, even before policy-check, or in parallel with it (with no needs: at all, since it doesn't depend on any of the other three's results). Chaining it at the end, after verify-artifact, would be the opposite design mistake lesson 1 already warned about with Trivy and terraform init: spending the most expensive controls' time before an almost-instant check that could have failed fast, first.

Summary and next step

In this lesson you built ci.yml with three new jobs — policy-check, iac-scan, verify-artifact —, each with needs: pointing at the previous one, and ran them end to end with act pull_request against real Docker: all three finished with 🏁 Job succeeded, in the exact order the chain declares. You confirmed act respects needs: with no extra flag needed, and documented a real detail — tfplan.json confusing Trivy's automatic detector — that doesn't affect the gate's result but is worth knowing.

Lesson 4 reuses this exact ci.yml, with no change, against a real, small business change — proof that the gate lets through what should pass, not just stops what should stop.

Resources

  1. GitHub Docs — jobs.<job_id>.needs — official reference, already cited in lesson 2.
  2. nektosact.com — User Guide — complete act reference, including the -j flag used in Exercise 1.
  3. This course, Module 4, lesson 3; Module 5, lesson 3; Module 6, lesson 5 — the origin of each install command this lesson adapts to a CI runner.
  4. cicd-and-gitops-on-aws-guide, Module 3, lesson 8 — the original nine-step ci.yml, this file's starting point.
← Previous2. Architecture review: the full security gateNext →4. End-to-end walkthrough: a change that passes the complete gate