Module 6: The Deterministic Shell
Project: build the deterministic shell for Mercado's agent
Overview
This is the module's capstone. In the seven previous lessons you built, piece by piece, the deterministic shell that contains an AI component's actions: the model proposes instead of executing (L2), in a structured format (L3), from a bounded capabilities menu (L4), validated against the business rules (L5), with a core kept small (L6), in an ordered and audited pipeline (L7). Now you bring them all together into a single deliverable: the complete deterministic shell of Mercado's support agent, running over a realistic batch of model proposals, measured end to end.
The deliverable has four parts, like every architecture capstone: the diagram of the shell (where the AI component lives and what gates surround its actions), the executed code (the complete pipeline running over twelve model proposals, with its literal output), an ADR (the record of the architectural decision to contain the model's actions, with its context, its alternatives, and its consequences), and the justification of why, in this design, the LLM never touches the money directly and the non-determinism stays contained. When you finish, you have an artifact you can take with you and apply to any AI feature that takes actions.
Connection with the module. This lesson closes module 6 by bringing everything before it together into a real, measured case, and connects it with what follows. Toward module 7 (the data loop): the audit log the shell produces is the raw material of the data flywheel you'll see next. Toward module 8 (the guide's capstone): the deterministic shell is one of the pieces you'll architect in Mercado's complete feature, together with the budget, the eval gate, the guardrails, and the fallback. And the usual boundaries: the business logic itself belongs to the domain guides; how to build the agent belongs to AI Engineering; here, the pattern of containing its actions.
The case: the support agent that wants to refund
Let's recap the case that has accompanied the whole module, now complete. Mercado's support agent handles customer messages: "I want my money back", "the order arrived broken", "refund me". The LLM reads each message and proposes an action —almost always a refund, sometimes an escalate_to_human or a send_message—. The agent is useful because the model understands the customer's language; it's safe because it never executes a refund directly. Each proposal passes through the deterministic shell, which validates it and executes it only if it meets all the rules.
The batch of twelve proposals we're going to run is a realistic sample of what a model produces in production: most are legitimate attempts, but there's everything that can go wrong —absurd amounts, hallucinated orders, out-of-policy refunds, double refunds, actions outside the agent's menu, and free text instead of a command—. The shell has to let the good through and contain everything else, and we have to be able to measure how much it contained.
Before the code, the shell diagram —the artifact that shows where the AI component lives and what surrounds it—:
flowchart TD
C["customer message"] --> LLM
subgraph nucleo["PROBABILISTIC CORE"]
LLM["the LLM (simulated)<br/>PROPOSES a structured action"]
end
LLM -->|"{action, order_id, amount}"| S1
subgraph shell["DETERMINISTIC SHELL"]
S1["1 · STRUCTURE<br/>menu command, with fields?"]
S2["2 · CAPABILITY<br/>action of THIS agent?"]
S3["3 · POLICY<br/>meets the business rules?"]
S1 -->|pass| S2 -->|pass| S3
end
S1 -->|fail| B["BLOCK<br/>+ audit log"]
S2 -->|fail| B
S3 -->|fail| B
S3 -->|pass| EX["EXECUTE<br/>money moves ONLY here"]
EX --> LOG["audit log<br/>(raw material of M7)"]
B --> LOG
Read it this way: the AI component —the probabilistic core— lives inside the shell, and its only responsibility is to propose. Its output doesn't go to the money: it goes to the first gate. The three gates in series decide; the money moves only at the end, and only for what passed the three. Everything —what was executed and what was blocked— lands in the audit log. That's the shape of an AI-native system that takes actions: a drop of uncertainty (the core) surrounded by a deterministic pipeline (the shell) that guarantees nothing dangerous reaches the money.
The code: the complete shell, executed
Here's the complete deterministic shell of the support agent, running over the twelve proposals. Note that the execution mutates the state: when a refund is executed, the order is marked as refunded, so that a second refund proposal for the same order —like the last one in the batch— is blocked by double refund. It's the lesson 7 pipeline, with the batch expanded and the containment metrics.
# Module 6, Lesson 8 (project): the support agent's deterministic shell.
# Complete pipeline -structure + capabilities + business rules + auditing-
# over a batch of 12 LLM proposals (SIMULATED): legitimate, hallucinated,
# out of policy, absurd amounts and ungranted capabilities.
# Only what passes the three gates is EXECUTED. No network, no API. Fixed data.
ORDERS = {
"A-1001": {"total": 50.00, "days_since_delivery": 3, "refunded": False},
"A-1002": {"total": 120.00, "days_since_delivery": 45, "refunded": False},
"A-1003": {"total": 30.00, "days_since_delivery": 5, "refunded": True},
"A-1005": {"total": 75.00, "days_since_delivery": 8, "refunded": False},
}
MAX_REFUND = 100.00
RETURN_WINDOW_DAYS = 30
ACTION_MENU = {
"refund": {"order_id", "amount"},
"escalate_to_human": {"reason"},
"send_message": {"text"},
"issue_store_credit": {"amount"},
"change_price": {"product_id", "price"},
}
GRANTED = {"refund", "escalate_to_human", "send_message"}
def stage_structure(p):
if not isinstance(p, dict):
return (False, "free text, not a command")
if p.get("action") not in ACTION_MENU:
return (False, f"unknown action: {p.get('action')!r}")
if ACTION_MENU[p["action"]] - p.keys():
return (False, "missing required fields")
return (True, "")
def stage_capability(p):
if p["action"] not in GRANTED:
return (False, f"{p['action']} is not in the agent's capabilities")
return (True, "")
def stage_policy(p):
if p["action"] != "refund":
return (True, "")
oid, amount = p["order_id"], p["amount"]
if oid not in ORDERS:
return (False, f"order {oid} does not exist")
o = ORDERS[oid]
if o["refunded"]:
return (False, f"order {oid} already refunded")
if o["days_since_delivery"] > RETURN_WINDOW_DAYS:
return (False, "outside the return window")
if amount > o["total"]:
return (False, f"amount {amount:.2f} > total {o['total']:.2f}")
if amount > MAX_REFUND:
return (False, f"amount {amount:.2f} > limit {MAX_REFUND:.2f}")
return (True, "")
def execute(p):
if p["action"] == "refund":
ORDERS[p["order_id"]]["refunded"] = True
return p["amount"]
return 0.0
PIPELINE = [("STRUCTURE", stage_structure),
("CAPABILITY", stage_capability),
("POLICY", stage_policy)]
# Batch of 12 proposals from the probabilistic core (SIMULATED).
PROPOSALS = [
{"action": "refund", "order_id": "A-1001", "amount": 50.00},
{"action": "refund", "order_id": "A-1002", "amount": 120.00},
{"action": "refund", "order_id": "A-1003", "amount": 30.00},
{"action": "refund", "order_id": "A-9999", "amount": 40.00},
{"action": "refund", "order_id": "A-1005", "amount": 5000.00},
{"action": "refund", "order_id": "A-1005", "amount": 75.00},
"I'll give you your money back right away, don't worry",
{"action": "issue_store_credit", "amount": 200.00},
{"action": "change_price", "product_id": "P-1", "price": 0.01},
{"action": "delete_account", "user_id": "U-7"},
{"action": "escalate_to_human", "reason": "customer asks for a supervisor"},
{"action": "refund", "order_id": "A-1001", "amount": 50.00},
]
print(f"{'#':<3}{'action':<20}{'result':<12}{'stage':<12}reason")
print("-" * 96)
executed = 0
money_moved = money_blocked = 0.0
stopped = {"STRUCTURE": 0, "CAPABILITY": 0, "POLICY": 0}
for i, p in enumerate(PROPOSALS):
name = p["action"] if isinstance(p, dict) and "action" in p else "(free text)"
blocked_at = reason = None
for stage_name, stage_fn in PIPELINE:
ok, why = stage_fn(p)
if not ok:
blocked_at, reason = stage_name, why
stopped[stage_name] += 1
break
if blocked_at is None:
money_moved += execute(p)
executed += 1
print(f"{i:<3}{name:<20}{'EXECUTE':<12}{'-':<12}approved")
else:
if isinstance(p, dict) and p.get("action") == "refund":
money_blocked += p.get("amount", 0.0)
print(f"{i:<3}{name:<20}{'BLOCK':<12}{blocked_at:<12}{reason}")
blocked = len(PROPOSALS) - executed
print()
print(f"LLM proposals : {len(PROPOSALS)}")
print(f"Executed : {executed}")
print(f"Blocked (contained) : {blocked}")
print(f" by STRUCTURE : {stopped['STRUCTURE']} "
f"by CAPABILITY : {stopped['CAPABILITY']} by POLICY : {stopped['POLICY']}")
print(f"Containment rate : {blocked / len(PROPOSALS) * 100:.1f}%")
print(f"Money moved (approved) : {money_moved:.2f}")
print(f"Money contained : {money_blocked:.2f} (refunds the LLM proposed that didn't go out)")What to expect. When you run the file, the output is exactly this:
# action result stage reason
------------------------------------------------------------------------------------------------
0 refund EXECUTE - approved
1 refund BLOCK POLICY outside the return window
2 refund BLOCK POLICY order A-1003 already refunded
3 refund BLOCK POLICY order A-9999 does not exist
4 refund BLOCK POLICY amount 5000.00 > total 75.00
5 refund EXECUTE - approved
6 (free text) BLOCK STRUCTURE free text, not a command
7 issue_store_credit BLOCK CAPABILITY issue_store_credit is not in the agent's capabilities
8 change_price BLOCK CAPABILITY change_price is not in the agent's capabilities
9 delete_account BLOCK STRUCTURE unknown action: 'delete_account'
10 escalate_to_human EXECUTE - approved
11 refund BLOCK POLICY order A-1001 already refunded
LLM proposals : 12
Executed : 3
Blocked (contained) : 9
by STRUCTURE : 2 by CAPABILITY : 2 by POLICY : 5
Containment rate : 75.0%
Money moved (approved) : 125.00
Money contained : 5240.00 (refunds the LLM proposed that didn't go out)
Read the complete output, because it's the whole module's shell working over a realistic case.
Three of twelve were executed; the shell contained nine. The executed ones are the legitimate: two valid refunds (A-1001 for $50, A-1005 for $75) and an escalation to a human. Everything else —75% of the batch— was contained, each block at its gate. That number, 75% containment, is the module's measure: of every four model proposals, the shell let one through and stopped three, without the model being "bad" —it simply proposed the normal range of actions an LLM proposes, good and bad, and the shell filtered—.
The blocks distribute across the three gates, each catching its type of problem. Two at STRUCTURE: the free text (row 6, which isn't a command) and the delete_account (row 9, a hallucinated action that doesn't even exist in the platform menu). Two at CAPABILITY: issue_store_credit and change_price (rows 7 and 8, real platform commands but outside the support agent's capabilities). Five at POLICY: the well-formed and in-menu refunds that violate the rules —out of window (row 1), already refunded (row 2), nonexistent order (row 3), absurd amount (row 4), and double refund (row 11)—. Each gate did its job, and the distribution tells you what type of error each one contained.
Row 11 shows the shell protecting the state, not just the money. Notice the last proposal: refund of A-1001 for $50 —the same action as row 0, which was executed—. But this time it's blocked, with the reason "order A-1001 already refunded". Why? Because row 0 executed that refund and mutated the state (marked A-1001 as refunded), so when row 11 proposes refunding it again, the not_refunded rule catches it. Without this protection, a model that proposes the same refund twice —something that happens, from retries or confusion— would pay twice. The shell doesn't only validate against fixed rules: it validates against the current state of the system, which changes with each action it executes.
The final number: $5240 contained. The model proposed, in refund, a total of $5365; the shell executed $125 (the legitimate ones) and contained $5240 that never touched any customer's account. As in lesson 1, the absurd $5000 amount dominates that figure, and that's exactly the point: the shell protects not only against small and frequent errors, but against the large and rare error that, executed just once, would be a disaster. Put the three metrics together —75% containment, $125 moved with judgment, $5240 protected— and you have the quantitative justification of the whole module: the model's non-determinism stayed contained, and the money only moved for what met every rule.
The ADR: recording the decision to contain the actions
An architecture capstone delivers an ADR (Architecture Decision Record): the record of a design decision, with its context, the decision, the alternatives considered, and its consequences. Here's the ADR of the support agent's deterministic shell.
# ADR-006: Deterministic shell for the support agent's actions
## Status
Accepted
## Context
Mercado's support agent uses an LLM to understand customer messages and
resolve cases, many of which end in a refund -an action that moves real and
irreversible money-. The LLM is non-deterministic: it can hallucinate an
order, propose an absurd amount, refund out of policy or twice the same
order. We need the agent to be useful (understand the customer) without its
non-determinism being able to compromise the money or the system's state.
## Decision
The LLM NEVER executes an action on money or state directly. It PROPOSES a
structured action -a named command with typed fields- and a deterministic
shell validates it in a pipeline of three gates in order before executing:
1. STRUCTURE - the proposal is a known menu command, with its fields.
2. CAPABILITY - the action is among those granted to THIS agent (minimal menu).
3. POLICY - the action meets ALL the current business rules.
Only the proposals that pass the three gates are executed. Everything -approved
and blocked- lands in an audit log. The policy lives in CODE, not in the
prompt.
## Alternatives considered
- Execute the LLM's output directly (naive): rejected; a hallucination moves
money with nothing to stop it. Measured: it would pay 5315 vs 125 with a shell.
- Put the policy in the prompt: rejected; it gives a probability, not a
guarantee. The model proposes out of policy anyway every so often.
- Validate the content with a format guardrail (M4) and execute: insufficient;
impeccable JSON can be an action that violates the policy.
## Consequences
+ The LLM can't touch the money directly; the guarantee is given by the code.
+ Bounded blast radius: minimal capabilities -> dangerous operations out of
reach by definition.
+ Auditable system: each decision is recorded (basis of the data loop, M7).
+ Measured containment: 75% of proposals contained, 5240 protected in the batch.
- The shell has to be maintained when the business policy changes (but it's a
change of parameters, not of architecture).
- The concrete policy (limits, window) is domain content, not this shell's;
it has to be coordinated with the business team.
This ADR is the artifact that justifies the decision to the team and to whoever reviews it in the future. Note that it doesn't document how each gate was implemented (that's in the code), but why it was decided to contain the model's actions, what was rejected, and what consequences it brings. A well-written ADR lets someone who arrives in six months understand the decision without having to reconstruct it.
The justification: why the non-determinism stays contained
Close the capstone with the argument that brings the whole module together: why, in this design, the LLM never touches the money and the non-determinism stays contained.
The argument has three legs. The first: the LLM only proposes; it doesn't execute. Its output isn't connected to any API that moves money —it's connected to the shell's first gate—. However much the model "wants" to refund $5000, the only thing it can do is propose it; the execution happens at the end of a pipeline that proposal has to pass entirely, and $5000 doesn't pass the policy gate. The guarantee "no out-of-policy refund goes out" doesn't depend on the model behaving well: it's given by the code's hard conditions, which hold 100% of the time.
The second: the model's surface over the irreversible is minimal. The agent has a menu of three capabilities, of which only one (refund) touches money, and that one is wrapped in five policy rules plus the validation against the current state. The model can't change prices, deactivate accounts, or issue credits —those operations aren't on its menu, so they don't exist for it—. The blast radius of any hallucination is bounded to "propose a refund the policy will reject", which is exactly what we saw contained nine times.
The third: the system is observable and accountable. Each decision —executed or blocked, and why— lands in the audit log. There are no phantom actions: if the money moved, there's a record of which proposal moved it and why it passed the gates; if it didn't move, there's a record of which gate stopped it. This not only lets you operate the system; it's the basis of module 7's data loop, where those same recorded decisions feed the system's continuous improvement.
Put the three legs together and you have the module's thesis demonstrated over a real case: the model proposes, the system disposes. The model contributes the intelligence of understanding the customer; the shell contributes the guarantee that no irreversible action happens without meeting the rules. The non-determinism wasn't eliminated —the model is still as fallible as ever—; it was contained, so its fallibility can't escape to the money. That's the shape of an AI-native system that takes actions, and now you've built it, run it, and measured it.
Exercises
Exercise 1 — Extend the shell. Mercado's business team wants the support agent to also be able to resend the invoice of an order to the customer (a risk-free action, it just sends an email with an existing document). Describe what changes you'd make in each part of the shell —ACTION_MENU, GRANTED, and whether it needs policy rules— and justify why this action needs less containment than a refund.
See solution
Changes to the shell to add resend_invoice:
ACTION_MENU: add"resend_invoice": {"order_id"}—it's a known platform command, with a required field (the order whose invoice to resend)—.GRANTED: add"resend_invoice"to the support agent's menu, because resending invoices is part of its job (unlikechange_price, which isn't).- Policy rules (
stage_policy): a minimal rule —that the order exists (order_id in ORDERS)—, and nothing more. It doesn't need to check amounts, windows, or refund state, because the action doesn't touch money or change state: it just resends a document that already exists.
Why it needs less containment than a refund: resend_invoice is reversible and with no effect on money or critical state. The worst that can happen if the model hallucinates is that an extra invoice is resent (a redundant email), which is a minor and recoverable inconvenience —very different from refunding $5000, which is irreversible money—. The amount of containment an action needs is proportional to its blast radius: refund touches irreversible money and needs five rules + state validation; resend_invoice touches only an email and needs one rule (that the order exists). This illustrates a design principle: not every menu action needs the same weight of shell; the policy is calibrated to each action's risk. Even so, resend_invoice passes through the same three gates —structure, capability, policy—; it's just that its policy gate is lighter.
Exercise 2 — The manipulation attempt. A malicious customer discovers that if they write very insistent messages, they sometimes get the model to propose a refund for an amount greater than the order's. Explain, using the capstone's output, why this attack doesn't work against the shell, and which gate stops it. Then say why putting the policy in the model's prompt would be vulnerable to this attack.
See solution
The attack doesn't work against the shell because, even if the customer gets the model to propose an inflated refund, the proposal has to pass the POLICY gate before executing, and there the amount > o["total"] rule (or amount > MAX_REFUND) stops it. We see it in row 4 of the capstone: the refund of A-1005 for $5000 —an amount far above the order's, exactly the result an attacker would seek— is blocked with "amount 5000.00 > total 75.00". It doesn't matter how the attacker convinced the model to propose it (insistence, manipulation, prompt injection); the shell validates the resulting action against the rules, not the model's reasons for proposing it. The attacker can manipulate the model (the probabilistic core), but can't manipulate the shell's hard condition if amount > total.
Putting the policy in the prompt would be vulnerable because the prompt is part of the probabilistic core —the same component the attacker is manipulating—. If the only barrier against an inflated refund is an instruction in the prompt ("don't refund more than the total"), then convincing the model to ignore that instruction is the attack, and models can be convinced (that's the essence of module 4's prompt injection). The policy in the prompt is a barrier inside the thing the attacker controls; the policy in the deterministic shell is a barrier outside their reach. That's why the module insists: the guarantee lives in the deterministic code, not in the prompt. The attacker can have the last word over what the model proposes; never over what the shell executes.
Exercise 3 — Apply it to another feature. Choose another Mercado AI feature that takes actions —for example, an agent that helps sellers manage their inventory (it could propose lowering a product's price, marking it out of stock, or creating a promotion)—. Design its deterministic shell: the capabilities menu, an example of a structured action, two policy rules, and a dangerous operation you would not give it. Justify each decision with a module principle.
See solution
Deterministic shell for the seller's inventory-management agent:
- Capabilities menu (
GRANTED):{update_stock, mark_out_of_stock, propose_discount}. Each is part of the job of managing inventory. Principle: bounded capabilities (L4) —the menu is derived from the agent's job, not from "what could be useful"—. - Example of a structured action:
{action: "propose_discount", product_id: "P-42", percent: 15}. A named command with typed fields, not free text. Principle: the structured action (L3) —the model names a menu command, it doesn't execute code or prose—. - Two policy rules (for
propose_discount): (1) the discount can't exceed 30% without authorization (percent <= 30), to protect the margin; (2) the product must belong to this seller (product.seller_id == agent.seller_id), so that an agent doesn't touch another's products. Principle: validate against business rules (L5) —the validation is conjunctive and protects against distinct risks: margin and ownership—. - Dangerous operation I would NOT give it:
delete_product(delete a product from the catalog). It's not easily reversible and isn't necessary for routine inventory management —if a product must be deleted, that's a decision that goes through a human via escalation—. Principle: least privilege (L4) and small core (L6) —the irreversible and rare moves out of the menu; the blast radius stays small—.
Design note: just like in the support agent, this agent proposes (lower price, mark out of stock) and the shell disposes (validates the discount against the margin and the ownership before applying it). The price isn't set freely by the model; the model proposes a percentage the rule bounds. And decisions like "does this product qualify for a featured promotion?" would be deterministic rules (over sales, stock), not model judgments —shrinking the core, L6—. The same module shape, applied to a different case: the model contributes the intelligence of managing; the shell guarantees that no action violates the business rules.
Summary and next step
In this capstone you built the complete deterministic shell of Mercado's support agent and measured it end to end. You delivered the four parts: the diagram (the probabilistic core inside the three gates, with the money moving only at the end), the executed code (the pipeline over twelve proposals: 3 executed, 9 contained, 75% containment, $5240 protected, including a double refund caught by the validation against the current state), the ADR (the decision to contain the model's actions, with its context, alternatives, and consequences), and the justification of why the non-determinism stays contained —the LLM only proposes, its surface over the irreversible is minimal, and the system is observable and accountable—. You demonstrated, over a real case, the module's thesis: the model proposes, the system disposes.
With this you close the deterministic shell of the actions. You already know how to contain what the model produces (M4, output guardrails) and what the model does (M6, this module). The guide has one more piece before the final capstone: module 7, the data and feedback loop. And there's a direct bridge from here: the audit log your shell produced —each proposal, each approval, each block with its reason— isn't just a record to operate; it's the raw material of the data flywheel. Each decision the shell made is a datum about how the model behaves, and module 7 will teach you to close the loop: use → data → better system. The containment you built here generates, as a byproduct, exactly the data with which the system learns.
And the boundaries you respected throughout the module keep marking where to go next: to build the agent that proposes the actions —better prompt, tool use, function calling—, the AI Engineering ecosystem; to decide which the refund policy's rules are, Mercado's domain guides; for the shell's resilience when the model goes down, module 5 and the resilience-and-reliability-patterns-guide. Here you built the containment pattern; those guides build the pieces it contains.
Resources
- Anthropic, "Building Effective Agents" (2024) — anthropic.com/engineering/building-effective-agents. The reference guide for designing agents where the model proposes bounded and verified actions, instead of executing freely. It's the direct backing of this capstone's shell. In English.
- Claude documentation, tool use — docs.anthropic.com. The mechanism by which a model returns structured actions (tool calls with typed arguments) that your system decides whether or not to execute. The technical basis of the structured action. Without focusing on a specific model version. In English.
- Michael Nygard, "Documenting Architecture Decisions" (2011) — the ADR format we used in this capstone. Recording the context, the decision, the alternatives, and the consequences of a design decision is a standard architecture practice, and here we apply it to the decision to contain the model's actions. In English.
- Martin Fowler and Bharani Subramaniam, "Emerging Patterns in Building GenAI Apps" — martinfowler.com/articles/gen-ai-patterns. The complete map of containment patterns for GenAI apps, where this action shell is one piece among several. In English.
- Chip Huyen, AI Engineering (O'Reilly, 2024). For the complete view of the architecture of applications with models —the AI component surrounded by application logic, with its controls and its observability—. In English.