Module 5: Apply On Merge The Cd Half

3. Chaining jobs with `needs` and passing the exact `plan`

Next →

Description

This lesson solves the module's central problem: apply.yml has to apply the same plan a person already reviewed in ci.yml —not a new one, recalculated on the spot—. You're going to learn needs:, the field that orders jobs within a single workflow, and actions/upload-artifact/download-artifact, the real mechanism that moves a file from one job to another —and even from one workflow to another—. Along the way you're going to run into two findings verified today, running this guide: one about how act simulates artifacts on your machine, and another about a very specific Docker Desktop networking limitation you have to resolve before any of this works.

Connection to the module

Lesson 2 gave you apply.yml's trigger. This lesson builds its internal structure: how many jobs it has, in what order they run, and how the second one gets the file the first one (or, more precisely, ci.yml) generated. Lesson 4 puts all of this together in the complete file, run end-to-end.

Analogy: signing the exact contract you reviewed, not a reprint

Imagine you review and sign a paper contract, page by page, initialing every clause. If the other party, before filing the contract, rewrites it from scratch —even swearing the content is "the same"— you no longer have any guarantee that what gets filed is what you approved: a comma, a number, an entire clause could have changed without anyone noticing. The only honest way to file a contract is to keep exactly the pages you already signed, not a reprint. actions/upload-artifact/download-artifact is that physical file: ci.yml "signs" a plan (calculates it and saves it as a binary file), and apply.yml, instead of calculating a new one, downloads exactly that same file and applies it as-is.

Step 1 — needs:, within a single workflow

You already saw, in Module 2 (lesson 2), that by default every job in a workflow runs in parallel, with no guaranteed order between them — order only exists if you declare it explicitly with needs:. Before building the real file, confirm it with a minimal example:

name: needs-demo

on: workflow_dispatch

jobs:
  fetch-reviewed-plan:
    runs-on: ubuntu-latest
    steps:
      - run: echo "Step 1: this job runs first, nothing depends on it"

  terraform-apply:
    needs: fetch-reviewed-plan
    runs-on: ubuntu-latest
    steps:
      - run: echo "Step 2: this job only starts after fetch-reviewed-plan succeeds"
act workflow_dispatch -j terraform-apply

act -j terraform-apply asks act for a specific job — but notice what happens when that job declares needs::

What to expect (literal output, executed to write this lesson — act runs the job it depends on first, even though you didn't request it explicitly):

[needs-demo/fetch-reviewed-plan] ⭐ Run Main echo "Step 1: this job runs first, nothing depends on it"
[needs-demo/fetch-reviewed-plan]   | Step 1: this job runs first, nothing depends on it
[needs-demo/fetch-reviewed-plan]   ✅  Success - Main echo "Step 1: this job runs first, nothing depends on it" [64.4995ms]
[needs-demo/fetch-reviewed-plan] 🏁  Job succeeded
[needs-demo/terraform-apply    ] ⭐ Run Main echo "Step 2: this job only starts after fetch-reviewed-plan succeeds"
[needs-demo/terraform-apply    ]   | Step 2: this job only starts after fetch-reviewed-plan succeeds
[needs-demo/terraform-apply    ]   ✅  Success - Main echo "Step 2: this job only starts after fetch-reviewed-plan succeeds" [59.2ms]
[needs-demo/terraform-apply    ] 🏁  Job succeeded

act -l also reflects the chaining, with a column you haven't seen used yet: Stage.

act -l

What to expect (literal) — notice the Stage column: 0 for the job with no dependencies, 1 for the one that declares needs::

Stage  Job ID                Job name              Workflow name  Workflow file    Events
0      fetch-reviewed-plan   fetch-reviewed-plan   needs-demo     needs-demo.yml   workflow_dispatch
1      terraform-apply       terraform-apply       needs-demo     needs-demo.yml   workflow_dispatch

A higher Stage means "runs later, and only if the previous Stage succeeded" — if fetch-reviewed-plan failed, terraform-apply wouldn't run at all, not under act, not on real GitHub. This is exactly the behavior apply.yml needs: if something goes wrong downloading the reviewed plan, applying anything would be worse than applying nothing.

An important limit: needs: doesn't cross files

Here's where it's worth being precise, because it's easy to draw the wrong conclusion from Module 3 (lesson 2). That module established, with a solid security reason, that ci.yml and apply.yml are separate files, triggered by different events (pull_request vs push to main). needs:, however, only orders jobs within the same file, in the same workflow run — there's no GitHub Actions syntax that lets a job in apply.yml say needs: ci.yml/terraform-checks. They're completely independent runs, at different times, potentially separated by hours or days (however long a Pull Request takes to get reviewed).

That means this lesson's word "chaining" has two distinct layers, and both are real:

  1. Within apply.yml, needs: orders its own jobs — the mechanism you just tested above.
  2. Between ci.yml and apply.yml, the chaining isn't with needs: — it's with a shared artifact, which one workflow uploads and the other, in a completely separate run, downloads. That's the piece you build next.

Step 2 — Uploading the plan from ci.yml

For apply.yml to have something to download, ci.yml has to save the plan as an applicable binary file —not the readable text you already know from plan-output.txt—. terraform plan accepts a flag you haven't used until now: -out=, which saves the calculated plan to a file terraform apply can pick up and apply exactly as-is, without recalculating anything.

Extend ci.yml's Terraform plan step (Module 3, lesson 6) with that flag, keeping the text file you already use for the summary:

      - name: Terraform plan
        run: tflocal plan -input=false -no-color -out=tfplan | tee plan-output.txt

-out=tfplan doesn't change a single line of what you see on screen —the text tee captures in plan-output.txt is identical to before—; it just adds, in parallel, a binary file (tfplan) with the complete plan, ready to be applied.

Now add the new step, at the end of ci.yml, after publishing the summary:

      - name: Upload the plan for apply.yml to use later
        uses: actions/upload-artifact@v4
        with:
          name: terraform-plan
          path: tfplan
          retention-days: 5

actions/upload-artifact is an official GitHub Action —from the same family as actions/checkout, already familiar since Module 2— that packages the file specified in path: and saves it as an artifact, associated with this specific workflow run, under the name you give it in name: (here, terraform-plan — the identifier apply.yml is going to use to find it). retention-days: 5 limits how long GitHub keeps the artifact before automatically deleting it — five days is more than enough for the typical time a Pull Request takes to get reviewed and merged.

A real finding: the --artifact-server-addr that doesn't resolve by default

Before running this with act, there's a networking obstacle worth naming with the same honesty Module 3 applied to skip_requesting_account_id. act simulates GitHub's artifact service with a local HTTP server, activated with the --artifact-server-path <folder> flag — without this flag, act doesn't even try to start it, and any upload-artifact/download-artifact step fails. Run it as-is, with nothing else:

act pull_request -e .github/act-events/pr-event.json -j terraform-checks --artifact-server-path ./.artifacts

What to expect (literal, verified today against act 0.2.89 running on Docker Desktop):

Attempt 1 of 5 failed with error: Request timeout: /twirp/github.actions.results.api.v1.ArtifactService/CreateArtifact. Retrying request in 3000 ms...
Attempt 2 of 5 failed with error: Request timeout: /twirp/github.actions.results.api.v1.ArtifactService/CreateArtifact. Retrying request in 6263 ms...
❗  ::error::Failed to CreateArtifact: Failed to make request after 5 attempts: Request timeout

The cause, confirmed by testing both configurations: by default, act's local artifact server tries to listen on the address 172.16.0.2 (--artifact-server-addr, with that fixed default value) — an address that, on Docker Desktop for macOS, the job's ephemeral container can't reach back to act's process on the host, even with network="host" declared (a limitation of how Docker Desktop emulates host-type networking on macOS, different from real Linux). The result is the same kind of connection failure you already know —a real attempt, with real retries, that never gets anywhere—, except this time the problem isn't LocalStack: it's act's own artifact server, unreachable from inside the container.

The verified fix: pass your machine's real IP address on the local network, instead of trusting the default value:

# macOS
export ARTIFACT_ADDR=$(ipconfig getifaddr en0)

# Linux
export ARTIFACT_ADDR=$(hostname -I | awk '{print $1}')

act pull_request -e .github/act-events/pr-event.json -j terraform-checks \
  --artifact-server-path ./.artifacts \
  --artifact-server-addr "$ARTIFACT_ADDR"

What to expect (literal output, executed to write this lesson, excerpt from the new step):

[ci/terraform-checks] ⭐ Run Main Upload the plan for apply.yml to use later
[ci/terraform-checks]   | With the provided path, there will be 1 file uploaded
[ci/terraform-checks]   | Artifact name is valid!
[ci/terraform-checks]   | Beginning upload of artifact content to blob storage
[ci/terraform-checks]   | Uploaded bytes 12484
[ci/terraform-checks]   | Finished uploading artifact content to blob storage!
[ci/terraform-checks]   | Artifact terraform-plan.zip successfully finalized. Artifact ID 2119430229
[ci/terraform-checks]   | Artifact terraform-plan has been successfully uploaded! Final size is 12484 bytes.
[ci/terraform-checks]   | Artifact download URL: https://github.com/nektos/act/actions/runs/1/artifacts/2119430229
[ci/terraform-checks]   ✅  Success - Main Upload the plan for apply.yml to use later [1.280550167s]
[ci/terraform-checks] 🏁  Job succeeded

This isn't an arbitrary address this guide asks for "just because": it's your own machine, on the network interface it already uses for all your other local traffic — the same kind of address you'd see with ipconfig/ifconfig, different on every machine, which is why it's calculated with a command instead of hardcoded in .actrc (which is portable across machines, and is exactly why this guide doesn't add an IP there that only makes sense on yours).

A second finding: why download-artifact finds the file without asking for a run-id

--artifact-server-path saves uploaded artifacts to disk, organized by run number — and here's a direct consequence of something you already know from Module 2 (lesson 2): github.run_id is fixed at 1 under act, always, on every separate invocation. That means when ci.yml uploads the artifact (run with run_id=1) and, later, in a completely separate invocation of act, apply.yml downloads it (also with run_id=1), both are writing to and reading from the same folder inside --artifact-server-path — without you having to tell download-artifact which specific run to fetch it from.

.artifacts/
└── 1/                          ← always "1" under act, whatever the workflow
    └── terraform-plan/
        └── tfplan.zip          ← uploaded by ci.yml, found by apply.yml

It's honest to name this precisely, so you don't draw a false conclusion: on real GitHub, every workflow run has a genuinely unique run_id —never 1 twice—, so apply.yml, running on a real repository, would not find ci.yml's artifact with a simple download-artifact like the one you just saw. It would need actions/download-artifact@v4's run-id: parameter, explicitly pointing to the ci.yml run that generated the approved plan —typically obtained by querying GitHub's API (gh run list, filtering by the merged commit's SHA) or, in more elaborate pipelines, with the workflow_run trigger, which automatically exposes github.event.workflow_run.id—. This guide doesn't build that lookup mechanism —it would add a call to the GitHub API that doesn't make sense under act, without a real account—; it names it here, precisely, so you know exactly what this lesson simplifies and why that simplification is specific to act, not a valid shortcut in production.

Step 3 — Downloading the plan in apply.yml

With that understood, apply.yml's fetch-reviewed-plan job is deliberately simple: download the artifact and confirm it arrived, without doing anything else.

jobs:
  fetch-reviewed-plan:
    runs-on: ubuntu-latest
    steps:
      - name: Download the plan reviewed in the pull request
        uses: actions/download-artifact@v4
        with:
          name: terraform-plan

      - name: Confirm the plan file arrived intact
        run: |
          test -s tfplan
          echo "tfplan is present: $(wc -c < tfplan) bytes"

actions/download-artifact@v4, with just name: (no run-id:), searches the current run —under act, always 1, the exact reason for the finding above—. test -s tfplan is a minimal shell check: it fails if the file doesn't exist or is empty, a cheap safety net before the next job tries to apply a plan that never arrived.

The second job, terraform-apply, also needs the file —every job in a workflow runs in its own container, sharing no filesystem with other jobs, exactly as you already saw in Module 3 (lesson 7) for steps from different workflows—, so it downloads the same artifact a second time, inside its own container:

  terraform-apply:
    needs: fetch-reviewed-plan
    runs-on: ubuntu-latest
    env:
      AWS_ACCESS_KEY_ID: test
      AWS_SECRET_ACCESS_KEY: test
      AWS_DEFAULT_REGION: us-east-1
      AWS_ENDPOINT_URL: http://host.docker.internal:4566
    steps:
      - name: Check out andes-cargo-infra
        uses: actions/checkout@v4

      - name: Set up Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: "1.15.8"

      - name: Download the plan reviewed in the pull request
        uses: actions/download-artifact@v4
        with:
          name: terraform-plan

      - name: Terraform init
        run: terraform init -input=false

It might look redundant to download the same file twice —once in fetch-reviewed-plan, again in terraform-apply—, but it serves a real purpose: the first job is a cheap, fast check that can fail early, without spending time installing Terraform, if the artifact never existed. The second one does the real work. It's the same "cheap before expensive" principle you already saw in ci.yml with fmt/validate before plan.

Common mistakes

Thinking needs: can reference a job in another file (this lesson's central mistake). What happens: someone writes, in apply.yml, something like needs: ci.yml/terraform-checks, expecting that to chain both workflows. How to spot it: GitHub Actions rejects the YAML —needs: only accepts job IDs declared in the same file—. How to fix it: remember this lesson's two layers — needs: orders jobs within a file; the chaining between ci.yml and apply.yml is done with a shared artifact, never with needs: directly.

Forgetting -out= and wondering why there's nothing to upload (workflow-based). What happens: someone adds the upload-artifact step pointing to tfplan, but the Terraform plan step is still Module 3's, without the -out=tfplan flag. How to spot it: upload-artifact fails with a "no such file or directory" error — no binary file was ever generated, only plan-output.txt's text. How to fix it: confirm the plan step's run: includes -out=tfplan, not just the tee redirect.

Confusing the default --artifact-server-addr with a LocalStack problem (diagnosis-based). What happens: someone sees a Request timeout error in an upload-artifact/download-artifact step, and assumes it's the same kind of connection failure they already saw with awslocal against LocalStack —they check whether LocalStack is running, wasting time there—. How to spot it: the message explicitly mentions ArtifactService, not any AWS endpoint or host.docker.internal:4566. How to fix it: this is a networking problem between the job's container and act's own process, not between the job and LocalStack — the fix is passing --artifact-server-addr with your real IP, as you saw in this lesson, with LocalStack having nothing to do with it.

Exercises

Exercise 1 — Explain why needs: doesn't solve this whole lesson's problem. Without looking at this lesson, explain to a colleague why, even though needs: exists and works, apply.yml still needs upload-artifact/download-artifact to get ci.yml's plan.

See solution

A complete answer sounds, roughly, like this: "needs: only orders jobs within the same workflow file, in the same run — ci.yml and apply.yml are separate files, triggered by different events, at different times (one when a Pull Request opens, the other when it merges, potentially hours later). There's no needs: syntax that crosses that boundary. The only real mechanism for moving a file from one workflow run to another is a shared artifact: ci.yml uploads it, apply.yml downloads it, each in its own independent run."

Exercise 2 — Diagnose the --artifact-server-addr finding. A colleague, on Linux (not macOS), tells you --artifact-server-path worked for them without needing to pass --artifact-server-addr explicitly. Is this inconsistent with what you learned in this lesson?

See solution

Not necessarily — the problem verified in this lesson is specific to how Docker Desktop emulates host-type networking on macOS, where network="host" doesn't give the container real access to the Mac's network interface. On real Linux, where network="host" does share the host's network stack natively, it's plausible the default address (172.16.0.2) resolves correctly with no adjustment at all. The underlying lesson —verify instead of assuming, and know how to diagnose the exact error (ArtifactService, not an AWS endpoint) if it shows up— applies regardless of the operating system.

Exercise 3 — Explain the fixed run_id simplification to someone who's going to use this in a real repository. A colleague wants to copy this lesson's apply.yml directly into a real GitHub repository. What would you warn them about the Download the plan reviewed in the pull request step?

See solution

You'd warn them that, on a real repository, actions/download-artifact@v4 with only name: (no run-id:) searches apply.yml's current run — which never has an artifact called terraform-plan, because that artifact gets uploaded by a separate, earlier ci.yml run. Under act, this works with no adjustment because run_id is always 1 in both runs — a coincidence of the simulation, not a real mechanism. On a real repository, they'd need to add run-id: explicitly pointing to the ci.yml run that generated the approved plan, typically obtained with the GitHub API or the workflow_run trigger.

Summary and next step

In this lesson you learned needs: as an ordering mechanism within a single workflow, confirmed with act -l showing two distinct Stage values. You extended ci.yml with -out=tfplan and actions/upload-artifact@v4, and built apply.yml's fetch-reviewed-plan job with actions/download-artifact@v4. Along the way, you verified two real findings: act's default --artifact-server-addr doesn't resolve reliably under Docker Desktop on macOS (fix: your real IP), and why download-artifact without run-id: works under act specifically because run_id is always 1 —a simulation simplification, not something valid on a real repository without adjustment—.

Before moving on you should be able to: explain why needs: can't cross workflow files; write the upload-artifact/download-artifact pair from memory; and diagnose an ArtifactService error without confusing it with a LocalStack problem.

Lesson 4 —hands-on— puts it all together: lesson 2's on, this lesson's job structure, and the final step that really runs terraform apply.

Resources

  1. GitHub Docs — needs context — official reference for the needs: field.
  2. GitHub — actions/upload-artifact and actions/download-artifact — the official repositories for both Actions, including documentation for the run-id: parameter for cross-run downloads.
  3. nektosact.com — Artifacts — official documentation for --artifact-server-path/--artifact-server-addr.
  4. Terraform Docs — Command: plan (-out) — the flag that turns the plan into an applicable file.
← Previous2. The merge trigger: `push` to `main`Next →4. Hands-on: building `apply.yml`