Module 1: Genai In Production Vs A Notebook
8. Project: Andes Cargo's AI workload map
Description
The previous seven lessons left, scattered, every piece of an architecture decision: process-shipment-manifest can't read free text (lesson 3); an AI-backed guardian is never cheaper or simpler than one without AI, so it's used only when needed (lessons 2, 6); the boundary with AI Engineering defines what this guide does NOT build (lesson 4); the real inventory confirms where each piece lives today (lesson 5); and the honest attempt against Bedrock confirms exactly what this guide can and cannot execute (lesson 7). This final project turns those seven scattered pieces into a single ADR-formatted document (Architecture Decision Record): ADR-001-llm-as-escalation-path.md, the document that's going to govern, with explicit criteria, the seven modules that follow.
Connection to the module
This is Module 1's close, and the one document this entire guide needs to justify, with evidence, why it builds things in the order it builds them — the same kind of document cloud-security-and-guardrails-guide (RISK-MAP.md) and finops-and-cost-guardrails-guide (COST-PROFILE.md) already left at the root of andes-cargo-infra/. Every module from M2 to M8 opens by citing this ADR's row that corresponds to it.
Step 1 — Why an ADR, not just a decision mentioned in prose
You already saw, in cloud-security-and-guardrails-guide M1.8, why an ADR is worth more than a loose table: it documents a decision's context and justification, not just its outcome. This guide has an additional, specific reason to need one: it's the only guide in the ecosystem that declares, from its first lesson, that it's going to leave a central piece — a model's real invocation — unable to run. An ADR is exactly the right format for that limitation to read not as a hidden flaw, but as a documented design decision, with its reasoning, in the same place any other decision in this guide gets documented.
Step 2 — The complete document
At the root of andes-cargo-infra/, create ADR-001-llm-as-escalation-path.md:
# ADR-001 — The LLM Is an Escalation Path, Not the Default
**Status:** Accepted · **Date:** this module's close · **Supersedes:** none
**Governs:** Modules 2 through 8 of `genai-on-aws-production-guide`
**Source:** Module 1, lessons 2, 3, 5, 6, and 7 (this same guide)
## Context
`process-shipment-manifest`, inherited from `aws-core-services-guide` and wired into the
event-driven system by `aws-serverless-and-containers-guide`, parses `key=value` manifests with a
deterministic, free, always-consistent parser. Some logistics partners send free-text manifests
instead -- the body of an email, a note copied from their own system -- that the deterministic
parser cannot read. Today, those manifests fail with no automated recovery path.
Amazon Bedrock can read free text and extract structured fields. It is also fundamentally
different from every other piece of infrastructure this ecosystem has built so far: it is not
deterministic (the same input can produce a slightly different output across calls), it is billed
per token instead of per hour provisioned, and it requires a managed guardrail plus a
defense-in-depth check that no prior module of this ecosystem needed. Module 1, lesson 2
established that a correct response in Bedrock's playground confirms none of this by itself.
The inherited infrastructure, confirmed live in Module 1, lesson 5: three active Lambda functions
(`process-shipment-manifest`, `validate-shipment-manifest`, `notify-shipment-partner`), five IAM
roles, one EventBridge rule on the default bus, and zero rules on the custom bus
`andes-cargo-events` -- which already publishes `ShipmentProcessed`/`ShipmentDelayed` events that
nothing consumes yet. Module 1, lesson 7 confirmed, by direct attempt in this sandbox, that no
free-tier LocalStack plan can invoke Bedrock for real -- the service is documented as
`"Included in Plans: Ultimate"` only, one tier above the free plan this entire ecosystem runs on.
## Decision
**The deterministic parser remains the default path for every manifest, with zero change to its
cost, speed, or reliability. The LLM is invoked only as an escalation path, triggered exclusively
by a `ManifestParseFailed` event that `process-shipment-manifest` publishes when, and only when,
its own `key=value` parsing already failed.** A new function, `extract-shipment-manifest-fields`,
listens for that specific event and attempts extraction via a Bedrock model -- the cheapest model
family that a real evaluation shows is reliable enough for this specific, narrow task (Amazon Nova
or a compact Mistral model are the starting candidates named in lesson 6; the final choice, with
real cost numbers, is Module 2's decision, not this one). Only on a successful extraction does
`extract-shipment-manifest-fields` write to `Shipments`, using the exact same schema the
deterministic path already uses.
This decision is deliberately conservative: it adds a new capability without touching a single
line of the code path that already works for the majority of Andes Cargo's manifests, and it
gives this guide a reliability metric -- the **escalation rate**
(`ManifestParseFailed` events / total manifests processed) -- that is calculable with zero real
invocations of any model, entirely from events this ecosystem's free-tier lab can already produce.
## The map: how Modules 2 through 8 implement this decision
| Order | Module | Implements this ADR by |
|--:|---|---|
| 1 | M2 — The Bedrock cost model | Prices the escalation path per token before a single resource is declared, so cost is a known input, not a surprise |
| 2 | M3 — IaC for an AI endpoint | Declares only the escalation path's infrastructure (`bedrock.tf`); `process-shipment-manifest` is not touched |
| 3 | M4 — Bedrock guardrails and defense in depth | Guards specifically the one path that touches a non-deterministic model; the default path needs no such guard |
| 4 | M5 — Securing the AI workload | Extends the inherited security gate to the escalation path's new Terraform, least privilege scoped to one model ARN |
| 5 | M6 — FinOps for tokens | Extends the inherited cost gate with a budget specific to the escalation path's usage-based cost |
| 6 | M7 — Observability, latency and evals | Measures the escalation rate itself as the primary SLI -- the number that proves this ADR's decision is holding |
| 7 | M8 — Capstone | Proves, end to end, that a well-formed manifest never triggers the escalation path at all |
## Consequences
Every module from M2 to M8 builds around the escalation path, never around replacing the default
path. The deterministic parser's cost, latency, and reliability guarantees -- the ones the first
seven guides of this ecosystem already established -- remain unchanged for the majority of
Andes Cargo's traffic. The escalation rate becomes the single most important reliability number
this guide can report without ever invoking a real model, and it is the first number Module 7
computes for real. If that rate ever climbs unexpectedly (for example, if a large logistics
partner permanently switches away from `key=value` format), it is the signal -- with evidence, not
intuition -- that extending the deterministic parser's coverage is worth more engineering time than
scaling the LLM path further; that trade-off is named here, not resolved here.
## Alternatives considered
**Route every manifest through the LLM, and use it to validate the deterministic parser's output.**
Rejected: this would make the most expensive, least predictable path in the entire system the
default for the majority case that the free, deterministic parser already solves perfectly --
exactly the antipattern Module 1, lesson 3 named directly.
**Replace `process-shipment-manifest`'s parser with an LLM-first design from the start.** Rejected:
it would discard the reliability guarantee (same input, same output, always) that every other
guide in this ecosystem depends on for its own "Qué esperar (literal)" blocks, in exchange for
capability the majority of manifests never needed in the first place.
**Build GPU-backed self-hosted inference instead of a managed model via Bedrock.** Rejected, and
out of scope for this guide entirely: no $0 path exists for dedicated GPU capacity in any LocalStack
plan, and Andes Cargo's manifest volume does not justify the cost of provisioned GPU infrastructure.
Named, not built, in Module 8, lesson 7 of this guide, in contrast with `kubernetes-and-eks-in-
production-guide`.Step 3 — Verifying the document
grep -c '^| [1-7] ' ADR-001-llm-as-escalation-path.md
grep -o '^| [1-7] | M[2-8]' ADR-001-llm-as-escalation-path.md | wc -lWhat to expect (literal — you wrote the content, the shape is deterministic):
7
7
Seven rows in the "The map" table, seven distinct modules (M2 through M8) referenced exactly once each — no module in this guide got left out of the ADR, and none appears duplicated or repeated.
How to read this document, six months later
Facing this ADR, a new reader — a technical interviewer, a teammate joining the project late — should be able to answer, without asking anyone: what's the decision? (the Decision section's first sentence: the deterministic parser is the default, the LLM is escalation); why? (the Context section: process-shipment-manifest can't read free text, and a model brings cost/risk/non-determinism the default doesn't have); how do you measure whether the decision still holds? (the escalation rate, explicitly named in Consequences); and what alternatives were ruled out, and why? (the final section, with the three options considered and rejected). If any of those four questions requires re-reading a whole lesson of this guide to answer, the document didn't do its job.
Module 1's close
With ADR-001-llm-as-escalation-path.md written, this module delivers exactly what it promised in lesson 1: no AI infrastructure built yet — neither bedrock.tf exists, nor does extract-shipment-manifest-fields have a single line of code —, but the architecture decision that's going to govern how it gets built is fixed, in writing, with its context, its reasoning, and its rejected alternatives. You enter Module 2 with an explicit criterion for evaluating every cost decision that follows, instead of a general intuition of "add AI carefully."
Common mistakes
Writing the ADR in Spanish, breaking the ecosystem's convention for technical identifiers and documents (language mistake). What happens: someone, comfortable writing the rest of the guide in Spanish, also translates the ADR's sections. How to spot it: if your ADR-001-llm-as-escalation-path.md mixes "Decision"/"Consequences" in English with Spanish prose inside those same sections. How to fix it: andes-cargo-infra/'s technical documents — just like HCL, Python, and JSON — always go in English, exactly as you already saw with THREAT-MODEL.md, RISK-MAP.md, and COST-PROFILE.md from the previous guides. Only this lesson's prose, the part explaining the ADR, goes in your working language.
Treating the "Alternatives considered" section as optional or decorative (scope mistake). What happens: someone writes Context and Decision carefully, but leaves Alternatives considered as a short list with no real reasoning. How to spot it: if your rejected alternatives don't each carry their own specific reason for being rejected. How to fix it: the alternatives section is the one that proves the decision had criteria behind it, not that it was the only option anyone thought of — it's exactly the difference between "we did this" and "we did this, instead of these two other real options, for these specific reasons," the real standard of an ADR a technical interviewer would expect to see defended.
Updating the escalation rate in Consequences with a made-up number before M7 actually calculates it (forward-looking expectation mistake). What happens: someone, eager to complete the document, adds a specific escalation-rate percentage to this ADR, without having yet run M7.4's real calculation. How to spot it: if your document has an escalation-rate number before completing Module 7. How to fix it: this ADR names the metric — what's going to be measured, and why it matters — not its value. The real value, calculated with a fixed, deterministic set of test events, is M7.4's specific job; adding it here ahead of time would be, exactly, the same "escalate before the deterministic parser actually fails" antipattern this entire ADR exists to prevent.
Exercises
Exercise 1 — Identify which row of "The map" corresponds to a specific module, without looking at the table. From memory, which module in this guide extends the inherited cost gate with a token budget? And which one measures the escalation rate for the first time?
See solution
M6 — FinOps for tokens — extends the inherited cost gate (cost-estimate/cost-check/cost-tags) with bedrock-budget.rego, the new budget policy for token usage. M7 — Observability, latency, and evals — calculates the escalation rate for the first time, literally, from a fixed set of test events, without invoking any model. If you identified both without looking at the table, you have a clear grasp of the relationship between the ADR and the modules that implement it.
Exercise 2 — Defend this ADR's decision against a concrete objection. A colleague says: "Why not just use the LLM for everything? It's simpler to maintain one processing path instead of two." Respond by citing this ADR's "Alternatives considered" section.
See solution
A complete answer sounds, roughly, like this: "The first alternative considered in this ADR is exactly that option, and it's rejected for a concrete reason: it would turn the most expensive, slowest, least predictable path in the system into the default for the majority case that the free, deterministic parser already solves perfectly. 'One path' sounds simpler in the abstract, but in practice it would mean paying per token, and accepting result variability, for shipments that today get processed for free and always with the same result. Two paths, with a clear rule for when to use each, is simpler to operate than one expensive path applied to everything equally."
Exercise 3 — Predict what should happen if the escalation rate, measured in M7, turns out surprisingly high. Based on this ADR's Consequences section, what decision should Andes Cargo make if, six months after deploying this AI workload, the escalation rate climbed from a typical 7% to 40%?
See solution
According to this ADR's Consequences section, an unexpectedly high escalation rate is, explicitly, the signal that it's worth investing in expanding what the deterministic parser recognizes directly — for example, if a large logistics partner permanently switched its shipping format to free text — instead of simply accepting a growing volume of more expensive model invocations. The ADR names this decision as something to evaluate with evidence when it happens, not something it resolves in advance — it's, literally, "a trade-off named here, not resolved here." The right response to that scenario isn't "scale up Bedrock infrastructure to handle more volume" by default, but first asking whether the underlying problem — the format arriving — changed, and whether extending parse_manifest is still the cheaper option.
Summary and next step
In this Module 1 final project you wrote ADR-001-llm-as-escalation-path.md, the document that fixes, in writing, this guide's complete architecture decision: the deterministic parser remains the default, no exceptions; the LLM enters only as an escalation path, triggered by ManifestParseFailed; and the escalation rate is the reliability metric that proves, with evidence and without invoking any model, that decision still holds. You verified the document with grep, and confirmed that "The map"'s seven rows cover, exactly once each, the seven modules that follow.
Before moving on you should be able to: explain the ADR's central decision in one sentence, without looking at the document; name the three rejected alternatives and the reason for each; and locate, for any module from M2 to M8, which row of "The map" corresponds to it and why.
With this document closed, the complete Module 1 — eight lessons, from the guide's map to this formal decision — is behind you. Module 2 opens with the question this ADR explicitly leaves pending: if the escalation path costs per token, how much does it really cost, and why can't Infracost — the tool the rest of the ecosystem already trusts — answer that question on its own?
Resources
cloud-security-and-guardrails-guide, Module 1, lesson 8 (08-project-andes-cargos-risk-map-and-guide-roadmap.md) — the same ADR format, applied to a different domain (risk resolution order).finops-and-cost-guardrails-guide, Module 1, lesson 8 —COST-PROFILE.md, the sibling cost document this ADR complements, not replaces.- Michael Nygard — Documenting Architecture Decisions — the original source of the ADR format this document follows.
- This module, lessons 3, 5, 6, and 7 — the full source for every claim in this ADR's Context section.