Module 8: Capstone The Andes Cargo Pipeline

3. End-to-end walkthrough: a real change crossing the pipeline

Next →

Description

This is the lesson that delivers, with evidence, the promise this guide opened with back in Module 1: stopping running terraform apply by hand, from a laptop, and starting to let a pipeline do it for you, with review in between. You're going to add a small, real change to Andes Cargo's HCL —a second tag on the andes-cargo-shipment-docs bucket—, open a branch, simulate a Pull Request, watch ci.yml calculate and approve the plan, simulate the merge to main, and watch apply.yml attempt to apply that exact plan against LocalStack. At no point in this lesson are you going to type terraform apply — not once, in any terminal.

Connection to the module

This lesson walks, left to right, exactly the diagram you assembled in lesson 2. It introduces no new workflow — it uses ci.yml and apply.yml exactly as they stood at the close of Module 6, with the guardrail already integrated. Lesson 4 walks the same diagram a second time, with a change the guardrail does stop.

Analogy: the assembly line, with a real piece put to the test

Earlier modules tested each assembly-line station separately —the one that checks format, the one that calculates the plan, the one that applies—, each with its own test piece. This lesson puts a real piece at the start of the line and follows it all the way to the end, without pulling it out at any intermediate point to check it by hand. If the complete line works, the piece comes out the other side exactly as designed, with nobody having had to intervene in the middle of the process — proof that the stations, together, do the work each one promised to do separately.

Step 1 — The real change: a second tag on the manifest bucket

Recall the andes-cargo-shipment-docs bucket, as it stood at the close of Module 3: it already has the Compliance = "manifest-retention-required" tag, added with merge() over local.common_tags. Now, imagine Andes Cargo's finance team asks for every storage resource to be tagged with its cost center, to be able to split S3 spend across areas — a real, small business request, with no relation at all to the Shipments table or any destructive resource.

Open s3.tf and extend the same merge():

module "shipment_docs_bucket" {
  source = "./modules/s3-bucket"

  bucket_name        = var.bucket_name
  enable_versioning  = true
  bucket_policy_json = data.aws_iam_policy_document.require_https.json
  tags = merge(local.common_tags, {
    Compliance = "manifest-retention-required"
    CostCenter = "logistics-andes"
  })
}

One new line, CostCenter = "logistics-andes", inside the same map merge() already combined since Module 3. Format and confirm the diff:

terraform fmt s3.tf
git diff --stat

What to expect (literal):

 s3.tf | 1 +
 1 file changed, 1 insertion(+)

Step 2 — The branch and the simulated Pull Request

On a real repository, this change would live on a feature branch, with a Pull Request open against main — exactly the same pattern that held for every change in this guide since Module 2. Write the event that simulates that PR, with a new number and a branch name that honestly describe the intent:

.github/act-events/pr-event-52.json:

{
  "action": "opened",
  "number": 52,
  "pull_request": {
    "number": 52,
    "title": "Add cost center tag to the shipment-docs bucket",
    "head": {
      "ref": "feature/add-cost-center-tag",
      "sha": "d3e4f5a6b7c8d9e0f1a2b3c4d5e6f7a8b9c0d1e2"
    },
    "base": {
      "ref": "main",
      "sha": "2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c"
    },
    "html_url": "https://github.com/andes-cargo/andes-cargo-infra/pull/52",
    "user": {
      "login": "andes-cargo-dev"
    }
  },
  "repository": {
    "name": "andes-cargo-infra",
    "full_name": "andes-cargo/andes-cargo-infra",
    "default_branch": "main"
  }
}

PR #52 —higher numbers than #42 (Module 2) and #45 (revert, Module 6), consistent with the time that's passed since then—, branch feature/add-cost-center-tag. The shas in head/base are still, like every one in this guide, hand-made-up values with the correct format of a Git SHA — no external API generated them.

Step 3 — ci.yml: the plan, calculated and reviewed

Run the complete CI pipeline, exactly as it would run on a real Pull Request:

export ARTIFACT_ADDR=$(ipconfig getifaddr en0)   # Linux: hostname -I | awk '{print $1}'
rm -rf .artifacts && mkdir -p .artifacts

act pull_request -e .github/act-events/pr-event-52.json -j terraform-checks --secret-file .secrets \
  --artifact-server-path ./.artifacts --artifact-server-addr "$ARTIFACT_ADDR"

What to expect (literal output, executed to write this lesson — twelve steps, in order):

[ci/terraform-checks] ⭐ Run Set up job
[ci/terraform-checks]   ✅  Success - Set up job
[ci/terraform-checks] ⭐ Run Main Check out andes-cargo-infra
[ci/terraform-checks]   ✅  Success - Main Check out andes-cargo-infra [36.820792ms]
[ci/terraform-checks] ⭐ Run Main Set up Terraform
[ci/terraform-checks]   ✅  Success - Main Set up Terraform [2.737736916s]
[ci/terraform-checks] ⭐ Run Main Terraform format check
[ci/terraform-checks]   ✅  Success - Main Terraform format check [139.219791ms]
[ci/terraform-checks] ⭐ Run Main Terraform init
[ci/terraform-checks]   ✅  Success - Main Terraform init [15.859424583s]
[ci/terraform-checks] ⭐ Run Main Terraform validate
[ci/terraform-checks]   ✅  Success - Main Terraform validate [1.936238084s]
[ci/terraform-checks] ⭐ Run Main Install awslocal
[ci/terraform-checks]   ✅  Success - Main Install awslocal [10.97458975s]
[ci/terraform-checks] ⭐ Run Main Confirm the runner can reach LocalStack on the host
[ci/terraform-checks]   |
[ci/terraform-checks]   | Could not connect to the endpoint URL: "http://host.docker.internal:4566/"
[ci/terraform-checks] Failed but continue next step
[ci/terraform-checks]   ❌  Failure - Main Confirm the runner can reach LocalStack on the host [6.085954667s]
[ci/terraform-checks] ⭐ Run Main Install tflocal
[ci/terraform-checks]   ✅  Success - Main Install tflocal [2.358674125s]
[ci/terraform-checks] ⭐ Run Main Terraform plan
[ci/terraform-checks]   | Plan: 12 to add, 0 to change, 0 to destroy.
[ci/terraform-checks]   ✅  Success - Main Terraform plan [4.452404s]
[ci/terraform-checks] ⭐ Run Main Guardrail — block any plan that destroys the Shipments table
[ci/terraform-checks]   | Guardrail passed: no destroy action found for aws_dynamodb_table.shipments.
[ci/terraform-checks]   ✅  Success - Main Guardrail — block any plan that destroys the Shipments table [2.094138167s]
[ci/terraform-checks] ⭐ Run Main Publish the plan to the job summary
[ci/terraform-checks]   ✅  Success - Main Publish the plan to the job summary [56.640833ms]
[ci/terraform-checks] ⭐ Run Main Upload the plan for apply.yml to use later
[ci/terraform-checks]   | Uploaded bytes 12316
[ci/terraform-checks]   | SHA256 digest of uploaded artifact zip is ccb8f6766aa01964aaaf15436a93c6f05800b6d4461fc0f045fe00f3158ed9de
[ci/terraform-checks]   | Artifact terraform-plan.zip successfully finalized. Artifact ID 2119430229
[ci/terraform-checks]   | Artifact terraform-plan has been successfully uploaded! Final size is 12316 bytes. Artifact ID is 2119430229
[ci/terraform-checks]   | Artifact download URL: https://github.com/nektos/act/actions/runs/1/artifacts/2119430229
[ci/terraform-checks]   ✅  Success - Main Upload the plan for apply.yml to use later [769.993ms]
[ci/terraform-checks] ⭐ Run Complete job
[ci/terraform-checks]   ✅  Success - Complete job
[ci/terraform-checks] 🏁  Job succeeded

Three things to read with the same care you already practiced in every earlier module:

  • Plan: 12 to add, 0 to change, 0 to destroy. — the same count as always, because a new tag modifies an attribute inside a resource that's already being created for the first time in this local state anyway, it doesn't add or remove any complete resource (the same lesson from Module 3, lesson 8).
  • Guardrail passed — Module 6's grep genuinely ran, against this plan's real JSON, and found no delete action on Shipments. This lesson's change doesn't touch that table at all; the guardrail confirms it with evidence, doesn't assume it.
  • /runs/1/artifacts/2119430229 — notice the 1 inside the URL: it's github.run_id, fixed under act (confirmed since Module 2), not a number that went up because you already ran this pipeline many times throughout this guide.

Confirm the new tag's exact content, the same "don't trust just the summary" discipline you already learned in Module 3:

grep -A2 "CostCenter" plan-output.txt

What to expect (literal, two occurrences — tags and tags_all, both inside module.shipment_docs_bucket.aws_s3_bucket.this):

      + tags                        = {
          + "Compliance"  = "manifest-retention-required"
          + "CostCenter"  = "logistics-andes"
          + "Environment" = "dev"
...
      + tags_all                    = {
          + "Compliance"  = "manifest-retention-required"
          + "CostCenter"  = "logistics-andes"
          + "Environment" = "dev"

All four tags show up together, in alphabetical order (how Terraform orders a map when printing it) — Compliance is still there, untouched by this change, and CostCenter shows up exactly where PR #52's title promised. This is the moment where, on a real repository, someone on the team would come in to review this summary and approve the merge.

Step 4 — The (simulated) merge and apply.yml

Simulate PR #52's merge by running apply.yml directly, the same command from Module 5:

act push -W .github/workflows/apply.yml \
  --artifact-server-path ./.artifacts --artifact-server-addr "$ARTIFACT_ADDR"

What to expect (literal output, executed to write this lesson):

[apply/fetch-reviewed-plan] ⭐ Run Main Download the plan reviewed in the pull request
[apply/fetch-reviewed-plan]   | Preparing to download the following artifacts:
[apply/fetch-reviewed-plan]   | - terraform-plan (ID: 2119430229, Size: 96, Expected Digest: undefined)
[apply/fetch-reviewed-plan]   | Redirecting to blob download url: http://192.168.100.35:34567/twirp/github.actions.results.api.v1.ArtifactService/DownloadArtifact
[apply/fetch-reviewed-plan]   | SHA256 digest of downloaded artifact is ccb8f6766aa01964aaaf15436a93c6f05800b6d4461fc0f045fe00f3158ed9de
[apply/fetch-reviewed-plan]   | Total of 1 artifact(s) downloaded
[apply/fetch-reviewed-plan]   ✅  Success - Main Download the plan reviewed in the pull request [397.6365ms]
[apply/fetch-reviewed-plan] ⭐ Run Main Confirm the plan file arrived intact
[apply/fetch-reviewed-plan]   | tfplan is present: 15650 bytes
[apply/fetch-reviewed-plan]   ✅  Success - Main Confirm the plan file arrived intact [63.144542ms]
[apply/fetch-reviewed-plan] 🏁  Job succeeded
[apply/terraform-apply    ] ⭐ Run Main Check out andes-cargo-infra
[apply/terraform-apply    ]   ✅  Success - Main Check out andes-cargo-infra [33.099708ms]
[apply/terraform-apply    ] ⭐ Run Main Set up Terraform
[apply/terraform-apply    ]   ✅  Success - Main Set up Terraform [2.583884542s]
[apply/terraform-apply    ] ⭐ Run Main Download the plan reviewed in the pull request
[apply/terraform-apply    ]   | SHA256 digest of downloaded artifact is ccb8f6766aa01964aaaf15436a93c6f05800b6d4461fc0f045fe00f3158ed9de
[apply/terraform-apply    ]   ✅  Success - Main Download the plan reviewed in the pull request [427.469417ms]
[apply/terraform-apply    ] ⭐ Run Main Terraform init
[apply/terraform-apply    ]   ✅  Success - Main Terraform init [15.485153542s]
[apply/terraform-apply    ] ⭐ Run Main Install tflocal
[apply/terraform-apply    ]   ✅  Success - Main Install tflocal [5.882883292s]
[apply/terraform-apply    ] ⭐ Run Main Terraform apply
[apply/terraform-apply    ]   | module.shipment_docs_bucket.aws_s3_bucket.this: Creating...
[apply/terraform-apply    ]   | module.lambda_manifest_processor_role.aws_iam_role.this: Creating...
[apply/terraform-apply    ]   | aws_dynamodb_table.shipments: Creating...
[apply/terraform-apply    ]   | module.app_server_role.aws_iam_role.this: Creating...
[apply/terraform-apply    ]   | aws_dynamodb_table.shipments: Still creating... [00m10s elapsed]
[apply/terraform-apply    ]   | module.app_server_role.aws_iam_role.this: Still creating... [00m10s elapsed]
[apply/terraform-apply    ]   | module.shipment_docs_bucket.aws_s3_bucket.this: Still creating... [00m10s elapsed]
[apply/terraform-apply    ]   | module.lambda_manifest_processor_role.aws_iam_role.this: Still creating... [00m10s elapsed]
[apply/terraform-apply    ]   | ╷
[apply/terraform-apply    ]   | │ Error: creating AWS DynamoDB Table (Shipments): operation error DynamoDB: CreateTable, exceeded maximum number of attempts, 9, ... connect: connection refused
[apply/terraform-apply    ]   | ╵
[apply/terraform-apply    ]   | ╷
[apply/terraform-apply    ]   | │ Error: creating IAM Role (AppServerRole): operation error IAM: CreateRole, exceeded maximum number of attempts, 9, ... connect: connection refused
[apply/terraform-apply    ]   | ╵
[apply/terraform-apply    ]   | ╷
[apply/terraform-apply    ]   | │ Error: creating IAM Role (LambdaManifestProcessorRole): operation error IAM: CreateRole, exceeded maximum number of attempts, 9, ... connect: connection refused
[apply/terraform-apply    ]   | ╵
[apply/terraform-apply    ]   | ╷
[apply/terraform-apply    ]   | │ Error: creating S3 Bucket (andes-cargo-shipment-docs): operation error S3: CreateBucket, exceeded maximum number of attempts, 9, ... connect: connection refused
[apply/terraform-apply    ]   | ╵
[apply/terraform-apply    ]   ❗  ::error::Terraform exited with code 1.
[apply/terraform-apply    ]   ❌  Failure - Main Terraform apply [54.804567333s]
[apply/terraform-apply    ] 🏁  Job failed
Error: Job 'terraform-apply' failed

The SHA256 matches again, across the two appearances that matter: ccb8f6766aa0... in ci.yml's Upload and in apply.yml's two downloads. The file that almost got applied is, bit for bit, the same one calculated and reviewed in Step 3 — no reprint, no recalculation in between. The same four Andes Cargo root resources, with no dependencies between them, try to get created in parallel, and all four fail the same honest way you already know from Module 5: connection refused, because LocalStack isn't running on this machine right now.

Confirm it independently:

docker ps -a --filter name=localstack_main

What to expect (literal): no rows at all — the same root cause as always, not a new problem this change introduced.

Step 5 — Verified with awslocal (representative)

With a valid LOCALSTACK_AUTH_TOKEN and LocalStack running, this apply would have ended in success, and this command would confirm the tag directly against the real bucket:

awslocal s3api get-bucket-tagging --bucket andes-cargo-shipment-docs

What to expect (representative — same format already confirmed in earlier guides in this ecosystem, without a live run against a valid token at this moment):

{
    "TagSet": [
        {"Key": "Project", "Value": "andes-cargo"},
        {"Key": "Environment", "Value": "dev"},
        {"Key": "ManagedBy", "Value": "terraform"},
        {"Key": "Compliance", "Value": "manifest-retention-required"},
        {"Key": "CostCenter", "Value": "logistics-andes"}
    ]
}

Five tags, the four inherited ones plus this lesson's CostCenter — the final confirmation, against the real resource, that the complete pipeline did exactly what Step 3's plan promised. The difference between this block and what you really ran in Steps 3 and 4 isn't in any pipeline mechanism —the three workflows are identical in both cases— but exclusively in whether there's a real LocalStack on the other side of host.docker.internal:4566.

Committing the close

git add .github/act-events/pr-event-52.json s3.tf
git commit -m "Add CostCenter tag to the shipment-docs bucket"
git log --oneline -3

What to expect (representative for the hashes, literal for the structure):

<hash> Add CostCenter tag to the shipment-docs bucket
<hash> Add doc/adr/0001-choosing-cicd-tooling.md: ADR for GitHub Actions over Jenkins/GitLab CI
<hash> ci.yml: add a guardrail step that blocks any plan destroying the Shipments table; add guardrail-demo.yml to test it against a seeded fixture

Common mistakes

Looking for terraform apply at some point in this lesson (the central mistake, expectation-based). What happens: someone, following terraform-and-iac-guide's habit, looks for the terraform apply command written somewhere in this lesson, and doesn't find it. How to fix it: it's not there, on purpose — Step 4 runs act push -W apply.yml, which inside its own container executes tflocal apply -auto-approve tfplan, but the student never types that command directly in their terminal. That's, literally, this entire guide's thesis, fulfilled in this lesson.

Expecting Plan: 12 to add to change to 13 because a tag got added (conceptual, revisit Module 3). What happens: someone sees the same number as every earlier run and wonders whether the change applied to the file. How to fix it: a tag is an attribute inside a resource, not a new resource — Plan:'s count never changes because of a tag, no matter how many get added. The correct evidence is the plan's content (Step 3's grep), not the last line's number.

Confusing Step 4's failure with a problem specific to this change (diagnosis-based, revisit Module 5). What happens: someone sees Job failed and suspects the CostCenter tag broke something. How to spot it: the message —connection refused after real retries, on the four root resources, identical to every earlier apply.yml in this guide— has no relation to the change's content. How to fix it: docker ps -a --filter name=localstack_main confirms, again, that the cause is LocalStack's absence, not this lesson's HCL.

Exercises

Exercise 1 — Reconstruct this guide's complete PR thread, from memory. Without looking back, list the three simulated Pull Requests that went through andes-cargo-infra/ throughout this guide, with their number and purpose.

See solution

PR #42 (Module 2, lesson 6; made real in Module 3, lesson 8): feature/add-shipment-tags → main, adds the Compliance tag to the manifest bucket. PR #45 (Module 6, lesson 3): revert-shipment-tags → main, reverts that same tag over a compliance policy change. PR #52 (this lesson): feature/add-cost-center-tag → main, adds the CostCenter tag to the same bucket, at the finance team's request. All three share the exact same mechanism —a hand-written pr-event.json, act pull_request -e—, only the content of the HCL change each one describes changes.

Exercise 2 — Explain this lesson's SHA256 verification chain, without looking at the text. At how many points was the same SHA256 verified in this lesson, and what does each one confirm?

See solution

Two points: (1) when ci.yml uploads the artifact in Step 3 (ccb8f6766aa0..., calculated for the first time); (2) when apply.yml, in Step 4, downloads it twice —once in fetch-reviewed-plan, again in terraform-apply, each job in its own container—, with the exact same value in both downloads. Together, the three appearances confirm the file terraform-apply almost applied is, bit for bit, the same one calculated and reviewed in ci.yml — never a new plan, recalculated along the way.

Exercise 3 — Predict what would happen if someone added the tag directly in dynamodb.tf instead of s3.tf. Without running anything, would Module 6's guardrail trigger if someone, by mistake, added a new tag to the Shipments table instead of the bucket?

See solution

No. The guardrail specifically looks for a "delete" action in the aws_dynamodb_table.shipments resource's "actions" array — adding a tag to that table would produce, at most, an "update" action (or, in this project, with no real apply ever completed, it would still show up as "create", just like the rest of the empty state), never "delete". The guardrail is designed to specifically detect destruction, not any modification to the table — a new tag on Shipments, even though it would be a change out of place in terms of where that business logic should live, wouldn't trigger any security alarm.

Summary and next step

In this lesson you followed a real HCL change —a second tag on the manifest bucket— from a feature branch, through a simulated Pull Request, ci.yml calculating and approving the plan (with Module 6's guardrail explicitly confirming there was no destruction at all), a simulated merge, and apply.yml genuinely attempting to apply that exact plan against LocalStack —verified by SHA256 at two points, with the same honest failure as always for lack of a token—. At no point did you type terraform apply.

Before moving on you should be able to: reconstruct the complete walkthrough from memory, from the HCL change to the apply attempt; explain why the guardrail didn't interfere with this specific change; and precisely describe what you'd see with a valid LOCALSTACK_AUTH_TOKEN at each of this lesson's four steps.

Lesson 4 walks exactly the same diagram, with a change that shouldn't go through — this guide's final proof's other half.

Resources

  1. nektosact.com — User Guide — complete reference for act pull_request and act push -W, used end-to-end in this lesson.
  2. HashiCorp Developer — Automate Terraform with GitHub Actions — the complete pattern this walkthrough demonstrates working end-to-end.
  3. Terraform Docs — the merge function — the function reused for this lesson's second tag.
  4. This guide's Module 3 (08-project-andes-cargos-ci-workflow.md) — the first real change's origin (Compliance) this walkthrough extends.
← Previous2. Architecture review: the complete pipelineNext →4. End-to-end walkthrough: a rejected change