Module 6: The Deterministic Shell
Validating the proposal against the business rules
Overview
You already have the shell's first two gates. Lesson 3 verified that the proposal is a well-formed command (channel); lesson 4 verified that the action is the agent's (capability). This lesson reaches the third and central one: for the actions that did pass the previous two —a well-formed refund, within the agent's menu—, does the action meet the policy? This is the gate that caught the $5000 refund from the beginning of the guide, and it's the heart of the module, because it's where "the system disposes" becomes concrete: a set of deterministic business rules that the proposal must pass all of before executing.
You'll see the validation opened rule by rule. Mercado's refund policy has five rules: the order exists, wasn't already refunded, is within the return window, the amount doesn't exceed the order total, and the amount doesn't exceed the agent's limit. For each model proposal, the shell evaluates the five and shows which ones it passes and which one it fails, and executes only if it passes all of them. The result is a matrix that makes visible, proposal by proposal, exactly what contained it —or why it was approved—.
Connection with the module. This is the heart lesson: the validation of the proposed action against the business rules, which is what gives the guarantee of "the system disposes". The boundary with the domain guides is here more important than ever and must be drawn carefully: we do not teach what Mercado's refund policy should be —what maximum amount, what window, what exceptions— because that's a business decision, domain content. We teach the pattern: that there exists a deterministic layer that validates the action against the policy before executing it, whatever the policy is. And the boundary with module 4, which this lesson makes definitively clear: the M4 guardrail validated the content/format of the output; here we validate the action against business rules. An output can be impeccable JSON (passes M4) and an action that violates the policy (blocked by M6).
An analogy: the bank teller and the loan's checklist
Go back to the teller from lesson 1, the one who recommends but doesn't approve. When their loan recommendation reaches the bank's system, the system doesn't approve it "by eye" nor trust that the teller did their job well. It runs a checklist: does the applicant exist in our records? is their score above the minimum? is their current debt below the ceiling? is the requested amount within their profile's limit? do they not already have an active loan of the same type? Each item on the list is a hard condition, evaluated by code, and the loan is approved only if it passes all of them. One failing —the low score, the high debt— is enough for the application to be rejected, with a concrete reason.
Notice three things about that list. First, it's exhaustive and deterministic: it's not a general impression of "looks good", it's specific items with yes/no answers. Second, it's conjunctive: passing most isn't enough; you have to pass all, because each rule protects against a different risk and skipping one leaves that risk open. And third, the list lives in the bank, not in the teller: the teller can recommend all they want, but the list is run by the system, which is the one that knows the rules and gives the guarantee that no approval violates them.
That checklist is the business-rule validation. The model's proposal (the refund) is the teller's recommendation; the checklist is the refund policy; the system that runs it is the deterministic shell. The model can propose with full conviction, just as the teller can recommend with enthusiasm; the approval depends on passing each item on the list, not on how convinced whoever proposed it was. And —key for the boundary with the domain guides— which items are on the list (what minimum score, what debt ceiling) is a bank decision; what this lesson teaches is that the list exists, is run before approving, and must be passed entirely.
Worked example: the validation, rule by rule
Let's run the checklist. The refund policy has five rules, and the check_rules function evaluates each one for each proposal, returning a row with the result of the five plus the final verdict. We use three marks: ok (the rule passed), x (the rule failed), - (not applicable, couldn't be evaluated —for example, if the order doesn't exist, it makes no sense to evaluate its window—). The verdict is EXECUTE only if all five are ok. We run the same six proposals from lesson 1 to see the complete matrix.
# Module 6, Lesson 5: validate the proposal against the business rules.
# The heart of the pattern: before EXECUTING, the deterministic shell runs the
# LLM's proposal against EACH rule of the refund policy and only executes if
# ALL pass. No network, no API, no keys. Fixed data.
ORDERS = {
"A-1001": {"total": 50.00, "days_since_delivery": 3, "refunded": False},
"A-1002": {"total": 120.00, "days_since_delivery": 45, "refunded": False},
"A-1003": {"total": 30.00, "days_since_delivery": 5, "refunded": True},
"A-1005": {"total": 75.00, "days_since_delivery": 8, "refunded": False},
}
MAX_REFUND = 100.00
RETURN_WINDOW_DAYS = 30
def check_rules(proposal):
# Returns a row with the result of each rule and the final verdict.
# 'ok' = passed; 'x' = failed; '-' = not applicable (could not be evaluated).
oid, amount = proposal["order_id"], proposal["amount"]
row = {"exists": "-", "not_refunded": "-", "in_window": "-",
"amt<=total": "-", "amt<=limit": "-"}
if oid not in ORDERS:
row["exists"] = "x"
return row, False
row["exists"] = "ok"
order = ORDERS[oid]
row["not_refunded"] = "ok" if not order["refunded"] else "x"
row["in_window"] = "ok" if order["days_since_delivery"] <= RETURN_WINDOW_DAYS else "x"
row["amt<=total"] = "ok" if amount <= order["total"] else "x"
row["amt<=limit"] = "ok" if amount <= MAX_REFUND else "x"
passed = all(v == "ok" for v in row.values())
return row, passed
PROPOSALS = [
{"action": "refund", "order_id": "A-1001", "amount": 50.00},
{"action": "refund", "order_id": "A-1002", "amount": 120.00},
{"action": "refund", "order_id": "A-1003", "amount": 30.00},
{"action": "refund", "order_id": "A-9999", "amount": 40.00},
{"action": "refund", "order_id": "A-1005", "amount": 5000.00},
{"action": "refund", "order_id": "A-1005", "amount": 75.00},
]
cols = ["exists", "not_refunded", "in_window", "amt<=total", "amt<=limit"]
header = f"{'order':<8}{'amount':>9} " + "".join(f"{c:<14}" for c in cols) + "verdict"
print(header)
print("-" * len(header))
executed = blocked = 0
for p in PROPOSALS:
row, passed = check_rules(p)
verdict = "EXECUTE" if passed else "BLOCK"
executed += passed
blocked += (not passed)
cells = "".join(f"{row[c]:<14}" for c in cols)
print(f"{p['order_id']:<8}{p['amount']:>9.2f} {cells}{verdict}")
print()
print(f"Executed : {executed}/{len(PROPOSALS)}")
print(f"Blocked : {blocked}/{len(PROPOSALS)} (contained by at least one rule)")What to expect. When you run the file, the output is exactly this:
order amount exists not_refunded in_window amt<=total amt<=limit verdict
------------------------------------------------------------------------------------------------
A-1001 50.00 ok ok ok ok ok EXECUTE
A-1002 120.00 ok ok x ok x BLOCK
A-1003 30.00 ok x ok ok ok BLOCK
A-9999 40.00 x - - - - BLOCK
A-1005 5000.00 ok ok ok x x BLOCK
A-1005 75.00 ok ok ok ok ok EXECUTE
Executed : 2/6
Blocked : 4/6 (contained by at least one rule)
Read the matrix row by row, because each one tells a different story of how a proposal can violate the policy.
A-1001 ($50): all ok → EXECUTE. The order exists, wasn't refunded, is within the window (3 days ≤ 30), the amount doesn't exceed the total ($50 ≤ $50) nor the limit ($50 ≤ $100). Five ok, verdict EXECUTE. It's the ticket that passes the entire checklist: a legitimate refund the shell approves with confidence, because it meets every rule.
A-1002 ($120): fails in_window and amt<=limit → BLOCK. Here the matrix shows something valuable: a single proposal can violate several rules at once. The order was delivered 45 days ago (outside the 30-day window → in_window is x) and the $120 amount exceeds the agent's $100 limit (amt<=limit is x). Two independent reasons to block. The shell doesn't need to choose "the" reason: it's enough for some rule to fail for the verdict to be BLOCK. Seeing the two x is useful for diagnosis —it says everything that's wrong, not just the first thing—.
A-1003 ($30): fails not_refunded → BLOCK. The order exists, is within the window, the amount is reasonable... but it was already refunded (not_refunded is x). Without this rule, it would be a double refund: paying twice for the same order. It's a violation that has nothing to do with the amount or the window; it's purely of state. It shows why the list has to be exhaustive: each rule protects against a risk the others don't see.
A-9999 ($40): fails exists, the rest - → BLOCK. The order doesn't exist in the source of truth —a model hallucination, which invented an order_id—. Notice the marks: exists is x, and the other four are - (not applicable). This is deliberate and correct: if the order doesn't exist, it makes no sense to ask about its window or its total —there's no order whose total to look up—. The validation stops short when a fundamental precondition fails, instead of inventing answers for rules that can't be evaluated. Asking "does the amount exceed the total?" of a nonexistent order has no answer; marking - says it honestly.
A-1005 ($5000): fails amt<=total and amt<=limit → BLOCK. The order exists, wasn't refunded, is within the window... but the $5000 amount exceeds both the order total ($75) and the agent's limit ($100). Two x, again several rules failing together. This is the $5000 refund that has appeared throughout the guide: an absurd amount that a single model error would disburse, contained here by two independent policy rules.
A-1005 ($75): all ok → EXECUTE. Same order as the previous row, different amount. $75 doesn't exceed the total ($75 ≤ $75) nor the limit ($75 ≤ $100), and the other rules pass. Verdict EXECUTE. It shows that the shell doesn't block orders nor customers: it validates actions. The same order A-1005 produces a block with $5000 and an execution with $75, because what the policy evaluates is the concrete action, not the order in the abstract.
Two executed, four contained. The final result is the same as lesson 1, but now you know exactly why: the matrix shows, rule by rule, what contained each proposal. And the design lesson is the conjunctive nature of the validation: an action is executed only if it passes all the rules; it's enough for one to fail to block. Each rule is an independent wall, and the refund has to pass through all of them to touch the money.
Going deeper: the checklist as a guarantee
The example showed the matrix; it's worth understanding the properties that make this validation a guarantee and not a suggestion.
The validation is conjunctive: you pass all the rules or there's no execution. The verdict is all(v == "ok" for v in row.values()) —a conjunction—. This isn't an implementation detail: it's the essence of the guarantee. If the validation were "pass most" or "pass the most important", each rule that isn't required becomes an open risk. The policy protects only if each of its rules is mandatory, because each covers a different hole: exists covers the order hallucinations, not_refunded covers the double refunds, in_window covers the out-of-deadline refunds, amt<=total and amt<=limit cover the absurd amounts. Skipping one is leaving its hole open. That's why the list is passed entirely or isn't executed.
Stopping short avoids evaluating the unevaluable. When exists fails, the function returns immediately, marking the rest as -. This is correct for two reasons. The technical one: you can't look up order["total"] of an order that isn't in ORDERS —trying would throw an error—. And the conceptual one: asking "does the amount exceed the total?" of a nonexistent order has no meaningful answer; - (not applicable) is more honest than forcing an ok or an x. There's a natural order in the rules: first the preconditions (does the object I'm acting on exist?), then the rules that depend on it (its state?, its amount?). Stopping short respects that order.
The validation is a conjunctive CHECKLIST (pass ALL or block):
proposal: {refund, order_id, amount}
│
▼
┌──────────────┐ no → BLOCK (and doesn't evaluate the rest: '-')
│ exists? │──────────────────────────────────────────┐
└──────────────┘ │
│ yes │
▼ │
┌──────────────┐ no → BLOCK │
│ not refunded? │─────────────────────────┐ │
└──────────────┘ │ │
│ yes │ │
▼ ▼ ▼
┌──────────────┐ no → BLOCK any 'x' in the list
│ in window? │────────────► means: it does NOT execute
└──────────────┘
│ yes (amt<=total and amt<=limit still apply)
▼
all 'ok' ──► EXECUTE (the money moves only here)
The distinction with module 4, now definitive. Every proposal in this matrix is impeccable JSON —{action: "refund", order_id: "A-1005", amount: 5000.00} is perfectly well-formed—. A module 4 schema guardrail would approve them all: they're valid outputs, well-typed, with no forbidden content. And yet four of the six violate the policy. This is what M4 can't see and M6 can: M4 validates that the output is a valid output; M6 validates that the action is executable according to the business rules. A $5000 amount is a perfectly valid number in any schema; only the amt<=limit rule, which knows the policy, rejects it. The two gates go in series: the M4 guardrail first (is it a well-formed output?), the M6 validation after (is it an action that meets the policy?). Neither replaces the other.
Where this lesson ends and the domain begins. It's crucial not to cross the boundary. This lesson teaches that the checklist exists, is run before executing, and is passed entirely. What it does not teach —and what belongs to Mercado's domain guides— is which the rules should be: whether the limit is $100 or $200, whether the window is 30 or 60 days, whether there are exceptions for premium customers, how the days since delivery are calculated considering time zones, what happens with partial refunds. Those are business decisions, and changing them doesn't change the pattern: the pattern is that there be a deterministic layer that validates the action against the current policy, whatever it is. If tomorrow Mercado raises the limit to $200, you change MAX_REFUND and the pattern stays identical. The shell is the architecture; the concrete rules are the domain.
Common mistakes
Validating "most" of the rules or "the most important", not all. What happens: for simplicity or in a hurry, the validation requires the amount but doesn't check whether the order was already refunded, or validates the window but doesn't verify that the order exists. Each rule that isn't required is a hole: the day a proposal arrives that violates exactly that rule, it passes. Why it happens: each rule seems to cover a rare case, and it's underestimated. How to detect it: if your verdict isn't a conjunction of all the policy's rules, you have holes. How to fix it: the validation is conjunctive —you pass all or it blocks—. Each rule covers a risk the others don't see; none is optional.
Evaluating rules on an object that doesn't exist. What happens: the validation assumes the order exists and goes straight to checking its amount or its window, without first verifying it's in the database. When the model hallucinates an order_id, the code tries to read the data of a nonexistent order and either throws an error (and crashes) or —worse— uses a default value and validates against garbage. Why it happens: it's forgotten that the order_id comes from a component that can hallucinate. How to detect it: does your validation check the object's existence before reading its properties? If not, a hallucinated identifier breaks it. How to fix it: order the rules with the preconditions first (does it exist?) and stop short if they fail, marking the rest as not applicable, as in the example.
Putting the rules in the prompt instead of in the shell. What happens: instead of the checklist in code, the policy lives in the prompt —"refund only within 30 days, never more than $100, never an already-refunded order"— and the model is trusted to respect it. Most of the time it does, but the prompt is a suggestion to a probabilistic component: sooner or later it proposes $5000 or an already-refunded order anyway. Why it happens: putting the rule in the prompt is fast and seems to work in the tests. How to detect it: if your only barrier against an out-of-policy refund is an instruction in the prompt, you have a probability, not a guarantee. How to fix it: the policy lives in deterministic code —this lesson's checklist—, which holds 100% of the time. The prompt can ask the model to propose within policy; the guarantee is given by the if, not the prompt.
Confusing validating the content (M4) with validating the action (M6). What happens: the team put a schema guardrail that verifies the output is well-formed JSON with the correct fields, and believes that validates the refund. But {action: "refund", amount: 5000, order_id: "A-9999"} passes the guardrail —it's perfect JSON— and is still an action that refunds an absurd amount of a nonexistent order. Why it happens: "valid output" is confused with "valid action". How to detect it: ask yourself whether a perfectly well-formed output could violate the business policy; if the answer is yes, you're missing the M6 rule validation. How to fix it: after the schema guardrail (M4), run the business-rules checklist (M6). They're gates in series, not alternatives.
Exercises
Exercise 1 — Read the matrix. Without running the code, for a proposal {refund, order_id: "A-1002", amount: 60.00} (remember: A-1002 has total $120, 45 days since delivery, not refunded), predict the result of each of the five rules and the final verdict. Explain why.
See solution
- exists → ok: A-1002 is in
ORDERS. - not_refunded → ok: A-1002 has
refunded: False. - in_window → x: it was delivered 45 days ago, and 45 > 30 (
RETURN_WINDOW_DAYS). Fails. - amt<=total → ok: $60 ≤ $120 (the order total). Passes.
- amt<=limit → ok: $60 ≤ $100 (the agent's limit). Passes.
- Verdict → BLOCK. Four rules pass, but
in_windowfails, and the validation is conjunctive: onexis enough to block.
The lesson of this row: a refund can have a perfectly reasonable amount ($60, within the total and the limit) and still be blocked for a reason that has nothing to do with the amount —being outside the window—. That's why the list has to be exhaustive: the correct amount doesn't compensate for the expired window. Each rule protects against a different risk, and all are mandatory.
Exercise 2 — The boundary with the domain. Mercado's business team decides to change the policy: raise the agent's limit from $100 to $250 and shorten the window from 30 to 15 days. What changes in the shell's code and what does not change? Use the answer to explain the boundary between this guide and the domain guides.
See solution
What changes are two values: MAX_REFUND = 250.00 and RETURN_WINDOW_DAYS = 15. Nothing else. The structure of the validation —that it runs the five rules, that it's conjunctive, that it stops short if the order doesn't exist, that it executes only if all pass— stays identical. The pattern isn't touched; only the policy's parameters are adjusted.
This illustrates the boundary exactly. The domain guides decide which are the policy's values and rules: how much the limit is, how many days the window, whether there are exceptions for premium customers, how the days are counted. They're business decisions that change with the business. This guide (architecture) teaches the containment pattern: that there be a deterministic layer that validates the action against the current policy, before executing, passing it entirely. The pattern is stable even if the policy changes: raising the limit to $250 or lowering the window to 15 days doesn't change that there's a conjunctive checklist running before moving the money. That's why we say the shell is the architecture and the concrete rules are the domain: the first we teach here, the second is decided by the domain guides and the business.
Exercise 3 — A new rule. Mercado wants to add a rule: the same customer can't receive more than 3 refunds in 30 days (to curb abuse). Explain why this rule can't be validated with the order's data alone, what the shell would need to evaluate it, and which module gate it belongs to. Bonus: why is it dangerous to leave this rule in the prompt?
See solution
This rule can't be validated with the order's data alone because it depends on the customer's history, not on the order in question: you have to know how many refunds that customer received in the last 30 days. The shell would need access to an additional source of truth —a record of refunds per customer— to count the recent refunds and compare them against the limit of 3. It's another business rule, and it belongs to the rule-validation gate (this lesson, L5): it's evaluated on the proposed action, against the policy, before executing. It would be added to the checklist as a sixth rule (refunds_last_30d < 3), and since the whole list is conjunctive, a refund would only be executed if it also passes this one.
It's dangerous to leave it in the prompt because the model has no reliable way to know how many refunds the customer received —it doesn't keep that count, and even if you put it in the context, it's a probabilistic component that could count wrong or ignore the instruction—. An anti-abuse rule that depends on the model "remembering" and "respecting" a count is a rule without a guarantee: exactly the kind of control that an attacker who wants to abuse the system would look to skip. The count is kept by the code (a deterministic query to the refund record), and the if refunds_last_30d >= 3: block gives the guarantee. Again the module's principle: the rule that protects the money lives in the deterministic shell, not in the prompt.
Summary and next step
In this lesson you reached the heart of the module: validating the proposal against the business rules. For the actions that passed the channel (L3) and the capability (L4), the shell runs a checklist of deterministic rules —order exists, not refunded, within the window, amount ≤ total, amount ≤ limit— and executes only if it passes all of them. You saw it rule by rule in a matrix: two proposals passed the five and were executed; four failed at least one and were blocked, each with its reason visible. You learned that the validation is conjunctive (all or none, because each rule covers a different risk), that it stops short when a precondition fails (the nonexistent order has no window to evaluate), and that it's definitively distinguished from the M4 guardrail (an output can be impeccable JSON and an action that violates the policy). And you marked the boundary with the domain: here the pattern of the checklist; which the rules are is a business decision.
Before moving on you should be able to: explain why the validation is conjunctive; order the rules with the preconditions first and stop short; distinguish validating the content (M4) from validating the action (M6); and place the boundary between the pattern (this guide) and the concrete rules (domain).
Lesson 6 takes a step back and asks a broader design question: how much should the model decide in the first place? You already know how to contain what the model proposes; but the less surface the LLM has over actions that touch state, the less there is to contain. You'll see, measured, the difference between a fat core —where the model decides many things that touch money— and a thin core —where the model proposes one thing and the rest are deterministic rules—: moving decisions out of the core reduces the surface that can hallucinate, and that reduction is counted. The discipline of keeping the probabilistic core small, which gives the whole guide its name.
Resources
- Martin Fowler and Bharani Subramaniam, "Emerging Patterns in Building GenAI Apps" — martinfowler.com/articles/gen-ai-patterns. The pattern of validating a model's action against deterministic rules before executing it is a central piece of the containment map the article describes. In English.
- Anthropic, "Building Effective Agents" (2024) — anthropic.com/engineering/building-effective-agents. Its emphasis on putting deterministic controls around the actions an agent proposes, instead of trusting the model, is the basis of this lesson's rule validation. In English.
- Chip Huyen, AI Engineering (O'Reilly, 2024). Its treatment of validation and controls over a model's outputs with side effects covers why the policy lives in code and not in the prompt. In English.
- Claude documentation, tool use — docs.anthropic.com. When the model proposes a tool call, your code decides whether to execute it; the business rules you run before executing are this lesson's checklist. Without focusing on a specific model version. In English.