Module 4: Bedrock Guardrails And Defense In Depth

8. Project: Andes Cargo's guardrails layer

Description

The previous seven lessons built, piece by piece, extract-shipment-manifest-fields's complete defense in depth: the six policies explained (lesson 2) and declared in real HCL (lesson 3), the exact reason that isn't enough alone (lesson 4), and the two custom, deterministic checks (lessons 5 and 6), with the honest limit of what neither can prove (lesson 7). This project integrates the three pieces — managed guardrail, PII scrubber, schema validator — into a single runnable flow, run for real, and closes with the honesty ledger precisely distinguishing what in this module ran for real from what stayed representative.

Connection to the module

Like every closing project in this guide, this deliverable introduces no new concept — it reuses, integrated, exactly what the previous seven lessons already built and tested separately. Module 5 picks back up this same guardrail and these same two scripts to extend them with the security gate inherited from cloud-security-and-guardrails-guide.

Step 1 — The managed guardrail, in its final form

bedrock.tf (lesson 3), unchanged from what's already built — the manifest_extractor_guardrail module with all six active inputs:

module "manifest_extractor_guardrail" {
  source = "./modules/bedrock-guardrail"

  name                      = "andes-cargo-manifest-extractor-guardrail"
  blocked_input_messaging   = "This input is not allowed due to content policy violations."
  blocked_outputs_messaging = "This output is not allowed due to content policy violations."

  content_filters = [
    { type = "PROMPT_ATTACK", input_strength = "HIGH", output_strength = "NONE" }
  ]

  pii_entities = [
    { type = "EMAIL", action = "ANONYMIZE" },
    { type = "PHONE", action = "ANONYMIZE" }
  ]

  denied_topics = [
    {
      name       = "ProhibitedShipmentGuidance"
      definition = "Guidance, instructions, or advice about smuggling, evading customs inspections, or shipping illegal, prohibited, or undeclared goods."
      examples = [
        "How do I hide undeclared goods from customs inspection?",
        "What is the best way to avoid a customs check on this shipment?",
      ]
    }
  ]

  grounding_filters = [
    { type = "GROUNDING", threshold = 0.75 },
    { type = "RELEVANCE", threshold = 0.75 },
  ]

  managed_word_lists = ["PROFANITY"]
  custom_words       = ["undisclosed cargo", "avoid inspection"]

  tags = local.common_tags
}

The complete modules/bedrock-guardrail/ — two policies inherited from Module 3, three added in this module's lesson 3 —, verified with the same sequence as always:

cd andes-cargo-infra/
terraform fmt -check -recursive; echo "fmt exit: $?"
terraform validate
terraform plan -input=false -no-color -out=tfplan-m4-final

What to expect (literal — run for real, with no LocalStack, no AWS account, in this environment, to close this module):

fmt exit: 0

Success! The configuration is valid.

Plan: 17 to add, 0 to change, 0 to destroy.

The exact same number Module 3 and this module's lesson 3 already confirmed — no new infrastructure, just policy depth inside the resource already declared.

Step 2 — The two custom checks, integrated into a single flow

guardrails/pre_invoke_checks.py and guardrails/post_invoke_checks.py (lessons 5 and 6) are, each, independent and tested separately. This project joins them in guardrails/defense_in_depth_flow.py, the script reflecting the exact order extract-shipment-manifest-fields would use them in — with the call to Bedrock in the middle represented, never executed:

#!/usr/bin/env python3
"""defense_in_depth_flow.py -- ties pre_invoke_checks.py and post_invoke_checks.py
around the exact point where extract-shipment-manifest-fields would call
bedrock:InvokeModel, for Module 4's capstone project (lesson 8).

This script demonstrates INTEGRATION, not invocation. The "model response" is
always a fixed, hardcoded, representative dict supplied by the caller -- never
the output of a real Bedrock call. See Module 4, lesson 7 for why no real
call happens anywhere in this guide. Everything BEFORE and AFTER that
hardcoded stand-in -- the PII scrub, the ShipmentFields schema validation, and
the final decision of whether the candidate would be written to Shipments --
runs for real, is 100% deterministic, and is covered by pytest below.
"""

from __future__ import annotations

from dataclasses import dataclass

from post_invoke_checks import validate_shipment_fields
from pre_invoke_checks import scrub_pii


@dataclass(frozen=True)
class ExtractionOutcome:
    scrub_found_pii: bool
    redacted_text: str
    schema_valid: bool
    schema_errors: tuple[str, ...]
    would_write_to_shipments: bool


def run_defense_in_depth(raw_manifest_text: str, representative_model_response: dict) -> ExtractionOutcome:
    """Mirrors the order extract-shipment-manifest-fields would run in:

    1. pre_invoke_checks.scrub_pii() over the raw manifest text -- REAL, always.
    2. [NOT RUN HERE] bedrock:InvokeModel, with the guardrail of lesson 3
       referenced -- represented by the caller-supplied
       `representative_model_response` dict, never invoked.
    3. post_invoke_checks.validate_shipment_fields() over that response --
       REAL, always. This function has no idea whether its input came from a
       real invocation or a fixture; that independence is the entire point of
       testing it in isolation in lesson 6.
    4. The write-to-Shipments decision: only if step 3 passed.
    """
    scrub_result = scrub_pii(raw_manifest_text)
    validation_result = validate_shipment_fields(representative_model_response)

    return ExtractionOutcome(
        scrub_found_pii=scrub_result.found_pii,
        redacted_text=scrub_result.redacted_text,
        schema_valid=validation_result.is_valid,
        schema_errors=validation_result.errors,
        would_write_to_shipments=validation_result.is_valid,
    )


def format_outcome(outcome: ExtractionOutcome) -> str:
    lines = [
        f"pre_invoke_checks:  {'PII FOUND (redacted)' if outcome.scrub_found_pii else 'CLEAN'}",
        f"post_invoke_checks: {'PASS' if outcome.schema_valid else 'FAIL'}",
    ]
    for err in outcome.schema_errors:
        lines.append(f"  - {err}")
    decision = "WRITE to Shipments" if outcome.would_write_to_shipments else "DO NOT WRITE to Shipments"
    lines.append(f"DECISION: {decision}")
    return "\n".join(lines)

Notice step 2's comment inside run_defense_in_depth(): [NOT RUN HERE] — the most explicit possible label, in the code itself, for exactly the point where a real invocation would happen in production, and where this project deliberately stops.

Step 3 — Two scenarios, run for real

if __name__ == "__main__":
    # Scenario A: the deterministic path's own manifest (4471), no PII, and a
    # representative model response that DOES match ShipmentFields exactly.
    scenario_a = run_defense_in_depth(
        raw_manifest_text=(
            "shipmentId=4471\noriginCountry=Peru\ndestinationCountry=Chile\n"
            "carrier=AndesExpress\nweightKg=120"
        ),
        representative_model_response={
            "shipmentId": "4471", "originCountry": "Peru",
            "destinationCountry": "Chile", "carrier": "AndesExpress",
            "weightKg": "120",
        },
    )
    print("=== Scenario A: well-formed manifest, valid representative response ===")
    print(format_outcome(scenario_a))
    print()

    # Scenario B: a free-text manifest carrying an incidental email, and a
    # representative model response missing weightKg -- the exact Brecha 2
    # case from lesson 4/6.
    scenario_b = run_defense_in_depth(
        raw_manifest_text=(
            "Hi team, following up on shipment AC-4471. Contact me at "
            "ana.rojas@andescargo.com or call +51 987 654 321 if you need "
            "anything. AndesExpress handles pickup Thursday."
        ),
        representative_model_response={
            "shipmentId": "4471", "originCountry": "Peru",
            "destinationCountry": "Chile", "carrier": "AndesExpress",
        },
    )
    print("=== Scenario B: free-text manifest with PII, incomplete representative response ===")
    print(format_outcome(scenario_b))

python3 guardrails/defense_in_depth_flow.py

What to expect (literal — run for real, same environment as this guide):

=== Scenario A: well-formed manifest, valid representative response ===
pre_invoke_checks:  CLEAN
post_invoke_checks: PASS
DECISION: WRITE to Shipments

=== Scenario B: free-text manifest with PII, incomplete representative response ===
pre_invoke_checks:  PII FOUND (redacted)
post_invoke_checks: FAIL
  - missing required field(s): weightKg
DECISION: DO NOT WRITE to Shipments

Scenario B is the practical confirmation of a rule worth stating out loud: finding PII never, by itself, stops the decision to write — the email gets redacted and the flow continues, exactly as lesson 5 designed pre_invoke_checks.py —; what does stop it is the response's incorrect shape, with no relation whatsoever to PII's presence in the input. The two layers evaluate different things, and this run demonstrates it with real data, not in the abstract.

Step 4 — The complete `pytest` suite, all three pieces together

cd guardrails/
pytest -v -p no:randomly

What to expect (literal — run for real; first and last cases shown, 34 total):

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 34 items

test_defense_in_depth_flow.py::test_well_formed_manifest_with_valid_response_writes_to_shipments PASSED [  2%]
test_defense_in_depth_flow.py::test_pii_in_input_never_blocks_a_valid_response_from_being_written PASSED [  5%]
test_defense_in_depth_flow.py::test_incomplete_response_is_never_written_regardless_of_input PASSED [  8%]
test_defense_in_depth_flow.py::test_pii_and_incomplete_response_together_still_blocks_the_write PASSED [ 11%]
test_defense_in_depth_flow.py::test_redacted_text_never_contains_the_original_email PASSED [ 14%]
test_defense_in_depth_flow.py::test_format_outcome_reports_write_decision PASSED [ 17%]
test_defense_in_depth_flow.py::test_format_outcome_reports_do_not_write_decision PASSED [ 20%]
test_post_invoke_checks.py::test_valid_candidate_passes PASSED           [ 23%]
...
test_pre_invoke_checks.py::test_cli_end_to_end_with_pii PASSED           [ 97%]
test_pre_invoke_checks.py::test_cli_end_to_end_clean PASSED              [100%]

============================== 34 passed in 0.02s ==============================

Seven new integration cases (test_defense_in_depth_flow.py) added to pre_invoke_checks.py's eleven (lesson 5) and post_invoke_checks.py's sixteen (lesson 6) — 34 total, zero failures, in under three hundredths of a second. It's worth naming the most important of the seven new cases: test_pii_and_incomplete_response_together_still_blocks_the_write — it runs, in a single test, exactly Step 3's Scenario B, confirming in code what that example already showed narratively.

Step 5 — This module's honesty ledger

Just as Module 3 closed with a six-row table, this project closes with the same discipline:

Piece	Command	Status	Exact reason
Managed guardrail, 6 mechanisms / 5 policies	`terraform fmt`/`validate`/`plan`	Executed	New resource — never needs network (Module 3, lesson 1)
`pre_invoke_checks.py` (PII scrubber)	`pytest` (11 cases)	Executed	Pure Python, regex, zero external dependencies
`post_invoke_checks.py` (`ShipmentFields` validator)	`pytest` (16 cases)	Executed	Pure Python, zero external dependencies
`defense_in_depth_flow.py` (integration of both)	`pytest` (7 cases) + direct run	Executed	The orchestration is real; the dict representing the model's response is fixed, labeled data
Real blocking of a prompt attack	—	Representative	Requires actually invoking `bedrock:InvokeModel`/`Converse` (lesson 7)
Real masking of a PII leak by the managed guardrail	—	Representative	Same reason — only a real invocation would confirm it (lesson 7)
`tflocal apply` of the guardrail against LocalStack	—	Representative	Bedrock — "Included in Plans: Ultimate" (Module 3, lesson 6; reconfirmed this module's lesson 7)

No row in this table invokes a Bedrock model, at any point — this guide's strictest rule, with no exception, not even in this closing project.

Common mistakes

Presenting defense_in_depth_flow.py as if it proved Andes Cargo's real extraction works (losing sight of the input dict being fixed, not generated, mistake). What happens: someone, showing this project in a portfolio, describes Step 3 as "the extractor working end to end." How to spot it: if your description of this project doesn't distinguish between "the orchestration of the two checks is real" and "the response dict is fixed, hand-written data." How to fix it: the script's own comment already says so, in the code — representative_model_response, with representative in the parameter's own name, not an accident. The correct phrasing: "I built and tested the integration of the two custom checks around the point where Bedrock would get invoked" — never "I tested the complete extractor."

Concluding, from Step 3's Scenario B, that finding PII "doesn't matter" because it didn't block anything (over-simplifying a rule with a specific intent mistake). What happens: someone, seeing that pre_invoke_checks: PII FOUND didn't change the final decision in Scenario B, concludes the PII check is cosmetic. How to spot it: if your summary of this project is "the PII check doesn't affect anything, the schema one is the one that really decides." How to fix it: reread lesson 5 — the PII check does act: it redacts the text before it moves forward, protecting any log or intermediate storage of that sensitive data. That it doesn't block the final write is an explicit design decision (avoiding duplicating what ANONYMIZE already does in the managed guardrail), not evidence the check does nothing.

Forgetting Step 1's 17 resources are the COMPLETE andes-cargo-infra/ project, not just this module's (losing the cumulative context mistake). What happens: someone, reading Plan: 17 to add, assumes this module, on its own, adds 17 new resources. How to spot it: if your mental count of "what did Module 4 build" includes S3 buckets, DynamoDB tables, or Lambda functions. How to fix it: as Module 3, lesson 8 already established, 17 to add is the count for the entire project, inherited from the previous three modules — this specific module added no new resource, only three policy blocks inside an already-existing resource (lesson 3). The number repeats exactly the same, module after module, precisely because nothing broke or got duplicated along the way.

Exercises

Exercise 1 — Verify, yourself, that each row in Step 5's honesty ledger matches this module's specific lesson backing it.

See solution

Row 1 (managed guardrail) → lesson 3. Rows 2 and 3 (the two custom checks) → lessons 5 and 6, respectively. Row 4 (the integration) → this same project, Steps 2-4. Rows 5 and 6 (real blocking/masking) → lesson 7. Row 7 (apply against LocalStack) → Module 3, lesson 6, reconfirmed in this module's lesson 7. If any row doesn't find its exact lesson, check that a new, unsupported claim wasn't introduced — the same traceability discipline Module 3, lesson 8 already demanded of itself.

Exercise 2 — Modify, yourself, Step 3's Scenario A so the representative response has an additional confidenceScore field. Predict the result before running it.

See solution

schema_valid would become False, with unexpected_fields = ("confidenceScore",) — exactly the test_unexpected_extra_field_fails case lesson 6 already tested in isolation, now visible through the complete integration. pre_invoke_checks would still report CLEAN (Scenario A's input manifest didn't change), but the final decision would change to DO NOT WRITE to Shipments — confirmation that the schema check, not the PII one, is the one governing the final decision in this specific case.

Exercise 3 — Explain, to a hypothetical technical interviewer, what this project demonstrates about the defense-in-depth architecture, without using the word "representative" more than once.

See solution

A complete answer sounds, roughly, like this: "This project integrates an AWS-managed guardrail, with six content-security mechanisms declared and verified with terraform plan, alongside two custom, deterministic checks running entirely outside AWS's control — one before the call to the model, one after. The 34 tests that run for real demonstrate the orchestration between the three layers works exactly as designed: finding sensitive information in the input never, by itself, blocks the final decision; a response with the wrong shape does block it, no matter how clean the input is. The one thing this specific environment can't prove is a language model's real classification when facing a genuine attack — a lab limit, stated with the same honesty in every lesson, never hidden."

Summary and next step

This project integrated this module's three pieces — the six-mechanism managed guardrail (lessons 2-3), the custom PII scrubber (lesson 5), the custom schema validator (lesson 6) — into a single runnable flow, run for real across two complete scenarios, with 34 pytest cases confirming correct behavior, including this architecture's central rule: PII in the input gets redacted and the flow continues, a response with the wrong shape gets rejected without exception. You closed with a seven-row ledger precisely distinguishing four pieces that ran for real from three representative ones, each with its own exact reason.

Before moving on you should be able to: run defense_in_depth_flow.py with your own test data and predict the result before executing it; explain, from memory, why PII in the input never blocks the final decision on its own; and defend, against any question, the difference between "orchestration tested" and "extractor tested end to end."

With the guardrail declared and the two custom checks integrated, the complete Module 4 — eight lessons, from the "layers, not substitutes" thesis to this project — is behind you. Module 5 picks back up exactly this same bedrock.tf and this same BedrockManifestExtractorRole role (Module 3) to extend them with the security gate inherited from cloud-security-and-guardrails-guide: bedrock-least-privilege.rego, evaluated against the same plan this module already confirmed clean.

Resources

This guide's Module 3, lesson 8 (08-project-andes-cargos-ai-infrastructure-declared.md) — the same closing-project pattern and honesty ledger this project reapplies.
This module, lessons 2 through 7 — the complete source for every claim in Step 5's honesty ledger.
Terraform Registry — aws_bedrock_guardrail — official documentation for this module's central resource.
AWS — Amazon Bedrock Guardrails — official landscape for the six policies this entire module developed.
pytest Docs — general reference for the tool used across this project's 34 cases.

← Previous7. The exact limit: why Bedrock Guardrails' real blocking can't be tested here Next →1. Introduction: the gate already exists, this workload enters it

8. Project: Andes Cargo's guardrails layer

Description

Connection to the module

Step 1 — The managed guardrail, in its final form

Step 2 — The two custom checks, integrated into a single flow

Step 3 — Two scenarios, run for real

Step 4 — The complete pytest suite, all three pieces together

Step 5 — This module's honesty ledger

Common mistakes

Exercises

Summary and next step

Resources

Step 4 — The complete `pytest` suite, all three pieces together