3. Andes Cargo's AI workload: when the deterministic parser isn't enough

Description

process-shipment-manifest, the Lambda function aws-core-services-guide built and aws-serverless-and-containers-guide connected to Andes Cargo's event bus, does exactly one thing: it reads a text file uploaded to andes-cargo-shipment-docs, parses it with a simple rule — one line, an = separator, a key and a value —, and if that rule finds the fields it needs, it writes a record to Shipments. It works perfectly, always, for the format it expects. The problem this guide exists to solve isn't that this function has a bug. It's that some of Andes Cargo's logistics partners don't send manifests in that format — they send the body of an email, a note copied from their own system, free text that no = separator can read. Today, those manifests simply fail.

This lesson does three things: it shows you the real parser running against both types of input, so you see the failure with your own eyes instead of accepting it as a claim; it shows you the architecture decision that resolves the problem without rewriting the function that already works; and it introduces, by name, the new piece this guide builds in the following modules — extract-shipment-manifest-fields — without building it yet.

Connection to the module

Lesson 2 gave you the theory: why a model that answers well in the playground says nothing about production. This lesson gives that theory a concrete, real case, the same one that carries the rest of this guide. Lesson 4, immediately after, traces the exact boundary between what this guide builds around that case (infrastructure, guardrails, cost) and what it doesn't build (the prompt that does the extraction itself — that's AI Engineering).

The inherited parser, exactly as aws-core-services-guide left it

This is the real code for process-shipment-manifest, with no modification — the same code that ran in aws-core-services-guide, Module 6:

import json
import urllib.parse
import boto3

s3 = boto3.client("s3")


def lambda_handler(event, context):
    processed = []

    for record in event["Records"]:
        bucket_name = record["s3"]["bucket"]["name"]
        object_key = urllib.parse.unquote_plus(record["s3"]["object"]["key"])

        response = s3.get_object(Bucket=bucket_name, Key=object_key)
        content = response["Body"].read().decode("utf-8")

        manifest = parse_manifest(content)
        # write_shipment_record(manifest, object_key) is still here,
        # inherited from aws-core-services-guide Module 7 -- unchanged.

        processed.append(manifest)

    return {"statusCode": 200, "manifestsProcessed": len(processed)}


def parse_manifest(text):
    fields = {}
    for line in text.strip().splitlines():
        if "=" in line:
            key, _, value = line.partition("=")
            fields[key.strip()] = value.strip()
    return fields

parse_manifest is deliberately simple: for every line of text, if there's an =, everything before it is the key, everything after it is the value. It doesn't know what a "shipment" is — it only knows how to read key=value. That simplicity is precisely what makes it fast, free to run, and 100% predictable.

The same parser, against two real types of input

Run this script — it's exactly parse_manifest, with no modification, against two manifests: one in the format Andes Cargo always expected, and one in the format a real logistics partner would send if it described the same shipment in its own words.

structured_manifest = """shipmentId=4471
originCountry=Peru
destinationCountry=Chile
carrier=AndesExpress
weightKg=120"""

free_text_manifest = """Hi team,

Following up on the shipment we discussed on the call. We're sending
120kg of textile goods from our Lima warehouse to the distribution
center in Santiago. AndesExpress is handling the pickup this Thursday.
Shipment reference on our side is AC-4471.

Let us know if you need anything else.

Regards,
Logistics Team
"""

print(json.dumps(parse_manifest(structured_manifest), indent=2))
print(json.dumps(parse_manifest(free_text_manifest), indent=2))

What to expect (literal — this is exactly the same parse_manifest from process-shipment-manifest, run to write this lesson, with no changes):

{
  "shipmentId": "4471",
  "originCountry": "Peru",
  "destinationCountry": "Chile",
  "carrier": "AndesExpress",
  "weightKg": "120"
}
{}

The first manifest — the usual format — produces exactly the five fields write_shipment_record needs to write to Shipments. The second — a real email, with the same underlying information (120kg, Lima to Santiago, AndesExpress, shipment AC-4471) — produces an empty dictionary. There's no = anywhere in the email text, so parse_manifest never enters the if "=" in line block, and fields ends up exactly as it started: with no fields at all. This isn't a parser bug — the parser did exactly what it promises. It's that the question we asked it ("does this have the key=value shape?") has an honest "no" answer for this text, and the parser has no plan B for that "no."

What happens today when the dictionary arrives empty

With no code change, the next step of process-shipment-manifest — inherited from aws-core-services-guide, Module 7 — tries to read manifest_data["originCountry"] to write the record to Shipments. With manifest = {}, that read raises an uncaught Python KeyError. The Lambda function's invocation ends in an error. ShipmentManifestWorkflow — the Step Functions workflow aws-serverless-and-containers-guide Module 3 built, with real Retry/Catch — retries a couple of times, and if the manifest still can't be parsed (which is exactly what's going to happen, since the text doesn't change between retries), the step ends up marked as failed. There's no automatic recovery path. A real logistics partner's manifest, with perfectly valid shipment information, simply never reaches Shipments. Someone, at some point, has to notice the failure and process that shipment by hand.

The architecture decision: evolution, not a rewrite

This guide's answer isn't to replace parse_manifest with a language model. It's to add to process-shipment-manifest an honest exit for the case that today ends in an uncontrolled KeyError: when key=value parsing doesn't produce the minimum fields it needs, instead of trying to write anyway and failing silently, the function publishes a new event — ManifestParseFailed — on the same andes-cargo-events bus already used by ShipmentProcessed, with the manifest's raw text attached. A new function, extract-shipment-manifest-fields, listens specifically for that event, and tries to extract the same fields by invoking a Bedrock model — and only if that extraction succeeds does it write to Shipments exactly the way the deterministic parser would.

   TODAY (without this guide)              AFTER (with this guide, M3-M4)

   free-text manifest                      free-text manifest
          │                                      │
          ▼                                      ▼
   parse_manifest() → {}                  parse_manifest() → {}
          │                                      │
          ▼                                      ▼
   uncontrolled KeyError                  publishes ManifestParseFailed
          │                                      │
          ▼                                      ▼
   Retry/Catch exhausts attempts          extract-shipment-manifest-fields
          │                                      │
          ▼                                      ▼
   fails, no automatic                    attempts extraction via Bedrock
   recovery                                      │
                                                  ▼
                                        success → Shipments (same as
                                        the deterministic path)

The ManifestParseFailed event follows exactly the same shape as ShipmentProcessed, to avoid introducing a new pattern where the existing one already suffices:

{
    "Source": "andescargo.shipments",
    "DetailType": "Manifest Parse Failed",
    "Detail": "{\"manifestKey\": \"manifests/year=2026/month=08/shipment-4474-manifest.txt\", \"rawText\": \"Hi team,\\n\\nFollowing up on the shipment...\", \"reason\": \"no key=value pairs found\"}",
    "EventBusName": "andes-cargo-events"
}

Notice the name of the reason field. It's not a cosmetic detail: the exact reason the parsing failed — "no key=value pairs found" — is information extract-shipment-manifest-fields doesn't need, but that any human reviewing the event bus does. No part of this guide builds that function yet — the HCL, the handler, and the guardrails arrive in modules 3 and 4. This lesson installs the decision and the name; the rest of the guide builds it piece by piece.

The architecture lesson, said in one sentence

The cheap, deterministic path stays the default; the LLM is an escalation path, not the main path. process-shipment-manifest remains the first and only stop for every well-formed manifest — zero changes to its cost, speed, or reliability. extract-shipment-manifest-fields only comes into play once the cheap path has already failed, exactly the same pattern as a service counter with a standard form and, only if the form doesn't apply, a person who reads the case carefully — slower, more expensive, used deliberately and sparingly. This is the decision that this lesson's M1.8 will fix in writing, as an ADR, and the one M8.4 will test live: a well-formed manifest never touches Bedrock.

This decision also gives you, without invoking Bedrock a single time, the most honest reliability metric this guide can offer: the escalation rate — how many out of every hundred manifests end up publishing ManifestParseFailed, out of the total processed. It's a real number, calculable with events that do run in this guide's $0 lab, and M7 turns it into the first SLI of an AI workload.

Common mistakes

Thinking this lesson already built extract-shipment-manifest-fields (expectation mistake). What happens: someone finishes this lesson looking for the handler code that invokes Bedrock. How to spot it: if you're searching this lesson for a real call to bedrock-runtime invoke-model. How to fix it: this lesson introduces the name, the event that triggers it, and the architecture decision behind it — the real handler, with the Bedrock call isolated behind a simple interface, gets built in M3 (infrastructure) and M4 (guardrails). It's exactly the same pattern as the previous guides: the architecture lesson first, the build after.

Concluding that process-shipment-manifest "has a bug" because it doesn't handle free text (scope mistake). What happens: someone reads that a free-text manifest produces a KeyError and concludes the inherited function is poorly written. How to spot it: if your reaction is "this should have been handled back in aws-core-services-guide." How to fix it: process-shipment-manifest fulfilled its original contract exactly — parsing key=value manifests, a format that, at the time, was the only one Andes Cargo received. The requirement to read free text is new, not a retroactive defect; it's exactly the kind of production evolution — adding a new path without breaking the one that already works — this entire guide models.

Assuming every free-text manifest now needs to go through Bedrock (diagram-reading mistake). What happens: someone reads the "before/after" diagram and concludes that, from now on, every manifest first goes through an AI check. How to spot it: if your explanation of the new flow starts with "first the model checks the manifest." How to fix it: the order is the opposite, and it's this lesson's central architecture decision — parse_manifest() remains the first and only attempt for every manifest. extract-shipment-manifest-fields is only invoked once that first attempt has already failed and published ManifestParseFailed. A system that consulted the model first, "just in case," would be exactly the antipattern this same module's M1.6 names by its own name: the LLM as default, instead of as escalation.

Exercises

Exercise 1 — Run the parser yourself, with a third manifest. Take this lesson's parse_manifest code and test it with a manifest that mixes both formats: two key=value lines followed by a free-text paragraph. Before running it, predict how many fields you'll get.

shipmentId=4474
originCountry=Bolivia
Please process this one as priority, the client called twice already.

Exercise 2 — Explain why the event is called ManifestParseFailed, not ManifestNeedsAI. The new event's name describes what happened (parsing failed), not what should happen next (use AI). Why is that naming choice correct, according to this lesson's architecture lesson?

Exercise 3 — Calculate, with made-up but coherent numbers, an escalation rate. If Andes Cargo processes 850 manifests in a month, and 62 of them publish ManifestParseFailed, what's the escalation rate? What does that number tell you about whether the deterministic default is still the right call?

Summary and next step

In this lesson you saw, with the real process-shipment-manifest parser running against two inputs — one key=value, one free text — exactly where and why Andes Cargo's deterministic path falls short: zero fields extracted from a real email with perfectly valid shipment information. You saw the architecture decision that resolves the problem without rewriting the function that already works: a new event, ManifestParseFailed, published on the existing bus, and a new function, extract-shipment-manifest-fields, that escalates to the model only after the deterministic parser has already tried and failed. And you saw this entire guide's central thesis, said in one sentence: the cheap path stays the default; the LLM is an escalation path.

Before moving on you should be able to: explain, using this lesson's literal parse_manifest output, why a free-text manifest produces an empty dictionary, not a syntax error; name the new event and which field carries the exact reason for the failure; and explain, without hesitating, why the LLM enters at the end of the flow, not the beginning.

Lesson 4 traces the exact boundary between what this guide builds around extract-shipment-manifest-fields — infrastructure, guardrails, cost, observability — and what it doesn't build — the prompt that does the extraction itself —, with a direct quote from the ecosystem's own design.

Resources

1. aws-core-services-guide, Module 6 (04-hands-on-deploying-your-first-function.md) and Module 7 (07-connecting-lambda-to-dynamodb.md) — the complete, inherited code for process-shipment-manifest, with no changes in this lesson.
2. aws-serverless-and-containers-guide, Module 3 (Step Functions) and Module 4 (04-custom-events-and-event-patterns.md) — ShipmentManifestWorkflow and ShipmentProcessed's exact contract, the pattern ManifestParseFailed follows.
3. AWS CLI — events put-events Command Reference — official reference for the event shape (Source/DetailType/Detail) used in this lesson.
4. AWS Docs — Amazon EventBridge event patterns — official reference for how a rule, in this guide's M3, would specifically filter for ManifestParseFailed.