Module 6: The Deterministic Shell

Keep the probabilistic core small

Overview

Lessons 2 through 5 gave you the art of containing what the model proposes: propose instead of execute, a structured format, a bounded menu, a rule checklist. This lesson takes a step back and asks a prior, more design-oriented question: how much should the model decide, in the first place? Because there's a simple truth behind the whole module: the less surface the LLM has over the actions that touch state, the less there is to contain. Each decision you take away from the probabilistic core and give to a deterministic rule is a decision that can no longer hallucinate —and therefore one you no longer have to validate, monitor, or fear—.

This is the principle that gives the whole guide its name —the small probabilistic core inside the deterministic shell— taken to its practical consequence in the terrain of actions. The AI-native discipline isn't just to contain the model well: it's to give it as little as possible to contain. You'll see the difference measured between two designs of the same agent: a fat core, where the LLM decides four things that touch money, and a thin core, where the LLM proposes one and the rest are deterministic rules. Over 20 cases, the fat core has 80 probabilistic decisions and the thin core has 20 —a 75% reduction of the surface that can hallucinate—.

Connection with the module. This lesson is the module's design synthesis: after learning to contain (L2-L5), you learn to minimize what there is to contain. It picks up the core/shell metaphor from module 1 —where you saw it measured in contained garbage outputs— and applies it to the actions: how much decision power over the state lives in the core. The boundary with AI Engineering holds firm: we're not talking about making the core smarter (better prompt, better model) —that's AI Eng—; we're talking about making the core smaller, which is an architecture decision: which decisions are the model's and which are rules. The two things are distinct and both matter; this guide deals with the second.

An analogy: how much the teller decides versus how much the system decides

Go back one last time to the bank teller. Imagine two banks with two very different distributions of decisions. In the first bank, the teller decides almost everything: how much credit to give, what interest rate to apply, whether to waive a fee, whether to approve an exception to the policy. The system only records what the teller decided. This bank depends on the teller's judgment in four distinct decisions, each with money at stake; if the teller makes a mistake —or is tricked, or has a bad day— in any of the four, the bank loses.

In the second bank, the teller decides one single thing: whether or not to recommend the applicant. Everything else —the amount according to the profile, the rate according to the score, whether the fee applies, whether an exception is needed— is decided by the system, with fixed rules. The teller contributes the human judgment where it's really needed (reading the customer, evaluating the case), and the system contributes the certainty where the rules suffice (calculating the amount, the rate, the fee). This bank depends on the teller's judgment in one decision, not four; the other three are rules that don't make mistakes, don't get tired, and don't let themselves be tricked.

The two banks have equally capable tellers. The difference is how much decision surface rests on fallible judgment versus how much rests on certain rules. The second bank doesn't trust its teller less; it simply recognizes that human judgment is valuable but fallible, and reserves it for where it's really needed, leaving everything else in the hands of rules. An LLM is that teller. The fat core has it decide four things that touch money; the thin core lets it propose one and turns the other three into deterministic rules. Not because the model is bad, but because each decision you take away from it is one it can no longer hallucinate.

Worked example: the surface that can hallucinate

Let's measure the probabilistic surface. We model a Mercado refund case that involves four decisions that touch money or state: the amount of the refund, whether to issue extra store credit, whether to waive the shipping cost, and whether to escalate to a human. In the fat core, all four are decided by the LLM (all probabilistic). In the thin core, only the amount is the LLM's —it requires understanding the customer's language—; the other three are deterministic rules (issue credit if the total exceeds a certain threshold, waive shipping if the refund is approved, escalate if the amount exceeds the limit). We count, over 20 cases, how many probabilistic decisions —how many that can hallucinate— each design has.

# Module 6, Lesson 6: keep the probabilistic core small.
# Every decision that touches money/state and lives in the CORE can hallucinate.
# Every decision we move out to DETERMINISTIC code can no longer hallucinate.
# We compare a "fat core" (the LLM decides 4 things) against a "thin core"
# (the LLM proposes 1 and the rest are rules). No network, no API. Fixed data.

N_CASES = 20   # support cases to process

# The 4 decisions each Mercado refund case touches:
#   1) refund amount               (requires understanding language -> core)
#   2) issue extra store credit    (rule: if total > 100)
#   3) waive the shipping cost      (rule: if the refund is approved)
#   4) escalate to a human          (rule: if the amount exceeds the limit)
DECISIONS = ["refund_amount", "store_credit", "waive_shipping", "escalate"]

# --- Fat core: the 4 decisions are made by the LLM (all probabilistic). ---
fat_probabilistic = len(DECISIONS) * N_CASES
fat_deterministic = 0

# --- Thin core: only 'refund_amount' is the LLM's; the rest are rules. ---
CORE_DECISIONS = {"refund_amount"}                       # the only probabilistic one
RULE_DECISIONS = set(DECISIONS) - CORE_DECISIONS         # extracted to the shell
thin_probabilistic = len(CORE_DECISIONS) * N_CASES
thin_deterministic = len(RULE_DECISIONS) * N_CASES

print("=== Probabilistic surface: decisions that can hallucinate ===")
print(f"{'design':<12}{'dec/case':>9}{'probabil.':>11}{'determin.':>11}"
      f"{'surface(20 cases)':>22}")
print("-" * 65)
print(f"{'fat core':<12}{len(DECISIONS):>9}{len(DECISIONS):>11}{0:>11}"
      f"{fat_probabilistic:>22}")
print(f"{'thin core':<12}{len(DECISIONS):>9}{len(CORE_DECISIONS):>11}"
      f"{len(RULE_DECISIONS):>11}{thin_probabilistic:>22}")

print()
removed = fat_probabilistic - thin_probabilistic
pct = removed / fat_probabilistic * 100
print(f"Decisions moved out of the core (now deterministic) : {removed}")
print(f"Reduction of probabilistic surface                  : {pct:.0f}%")

print()
print("=== Which decision lives where, in the thin core ===")
for d in DECISIONS:
    where = "CORE (LLM proposes)" if d in CORE_DECISIONS else "SHELL (deterministic rule)"
    print(f"  {d:<16}-> {where}")

What to expect. When you run the file, the output is exactly this:

=== Probabilistic surface: decisions that can hallucinate ===
design       dec/case  probabil.  determin.     surface(20 cases)
-----------------------------------------------------------------
fat core            4          4          0                    80
thin core           4          1          3                    20

Decisions moved out of the core (now deterministic) : 60
Reduction of probabilistic surface                  : 75%

=== Which decision lives where, in the thin core ===
  refund_amount   -> CORE (LLM proposes)
  store_credit    -> SHELL (deterministic rule)
  waive_shipping  -> SHELL (deterministic rule)
  escalate        -> SHELL (deterministic rule)

Read the two designs, because the difference between them is the module's discipline.

The two designs process the same cases with the same four decisions. Notice the dec/case column: both have 4. The refund case didn't change —it's still the same four decisions to make (amount, credit, shipping, escalation)—. What changed is where each decision lives: in the fat core, all four are made by the LLM; in the thin core, one is made by the LLM and three are rules. The amount of work is the same; the split between the probabilistic and the deterministic is what differs.

The fat core has 80 probabilistic decisions; the thin core, 20. Over 20 cases, the fat core puts 4 × 20 = 80 decisions that touch money in the hands of a component that can hallucinate. The thin core puts 1 × 20 = 20. Those 80 and those 20 are the probabilistic surface: the amount of decisions over the state that depend on the model's fallible judgment, and that therefore have to be validated, monitored, and feared. Each of the fat core's 80 is an opportunity for the model to issue a store credit that wasn't warranted, waive a shipping cost by mistake, or escalate (or not escalate) when it shouldn't. The 60 decisions the thin core moved out of the core simply can't hallucinate: they're rules —if total > 100: issue_credit— that hold the same way always.

The 75% reduction is the discipline measured. Moving three of the four decisions out of the core reduced the probabilistic surface from 80 to 20 —75% less—. And notice what this reduction didn't do: it didn't improve the model, it didn't change its prompt, it didn't lower its error rate. It reduced how many decisions depend on the model. It's an architecture gain, not a core-quality gain. The fat core and the thin core could use exactly the same model, with the same hallucination rate; the thin core is safer because it exposes less surface to that rate, not because the rate is lower.

The final split shows the logic. The table below says which decision lives where in the thin core, and the logic is clear: refund_amount stays in the core because it requires understanding the customer's language —"I want my money for the order that arrived broken" has to be interpreted, and only an LLM does that well—. The other three go to the shell because they're mechanical rules over data we already have: issuing credit depends on the order total (a number), waiving shipping depends on whether the refund was approved (a boolean), escalating depends on whether the amount exceeds the limit (a comparison). None of the three needs to "understand" anything; they're calculations. And the AI-native golden rule is exactly that: what can be a rule, is a rule; the core keeps only what genuinely needs intelligence.

Going deeper: the discipline of shrinking the core

The example showed the number; it's worth understanding the discipline that produces it and why it matters so much.

The probabilistic surface is what you have to secure. Each decision in the core isn't only an opportunity to hallucinate: it's also containment work you have to do. For each decision you give the model, you have to think of all the ways it could go wrong, write its validation, test its edge cases, monitor its use in production. A fat core with 4 probabilistic decisions per case is 4 surfaces to secure; a thin core with 1 is one. Shrinking the core doesn't only reduce the hallucination risk: it reduces the shell's work. Each decision you move out of the core is one you don't have to contain, because it's no longer made by a fallible component —it's made by a rule that's correct by construction—.

The design question: does this decision need intelligence, or is it a rule in disguise? Facing each decision a case requires, the AI-native discipline asks: does it really need the model's judgment, or is it a calculation I can write as a rule? Many decisions we instinctively give the LLM are, on a closer look, rules: "issue store credit?" sounds like a judgment, but it's if order_total > 100. "Escalate to a human?" sounds like criteria, but it's if amount > limit or not verifiable. The temptation is to leave them to the model because "it's already there and it's smart"; the discipline is to move them out because they're deterministic. Only what genuinely needs to understand language, generate text, or reason about something ambiguous stays in the core. Everything else is shell.

   The same task, two core splits:

   FAT CORE (4 decisions to the LLM):       THIN CORE (1 to the LLM, 3 rules):

   ┌─────────────────────────────┐         ┌───────────────┐
   │           CORE (LLM)         │         │   CORE (LLM)  │
   │  refund_amount   ◄ probabil. │         │ refund_amount │ ◄ the only one that
   │  store_credit    ◄ probabil. │         └───────────────┘   needs to
   │  waive_shipping  ◄ probabil. │                 │           understand language
   │  escalate        ◄ probabil. │                 ▼
   └─────────────────────────────┘         ┌─────────────────────────────┐
                                           │       SHELL (rules)          │
   surface that hallucinates: 4/case       │  store_credit   = total>100  │
                                           │  waive_shipping = if approved│
                                           │  escalate      = amount>limit│
                                           └─────────────────────────────┘

                                           surface that hallucinates: 1/case

Shrinking the core isn't limiting the feature. The natural objection: "but if I take decisions away from the model, the feature does less". The opposite: the feature does exactly the same —the four decisions are still made (notice that dec/case is 4 in both)—; what changes is who makes them. The thin core doesn't issue fewer store credits nor escalate fewer cases; it issues and escalates them the same, but with rules instead of the model's judgment. And the rules usually make better decisions than the model in these mechanical tasks: an if total > 100 never gets it wrong about whether the total exceeds 100, while an LLM occasionally does. Shrinking the core improves the feature in the deterministic decisions and reserves the model for where its intelligence really adds value.

This is the same principle from module 1, now in actions. In module 1 you measured the core/shell metaphor with outputs: a core with one responsibility (propose a tag) inside a shell with several guarantees. Here you measure it with decisions that touch state: how many of them live in the core. It's the same discipline —small core, robust shell— applied to the most dangerous terrain, that of irreversible actions. And the mandate is identical: the temptation is always to put more into the core (it's smart, it's already there), and the discipline is always to move out of it everything that can be deterministic. The smaller the core, the more of the system is certain, and the easier to contain is the little that stays uncertain.

Common mistakes

Putting decisions into the core because "the model is already there and it's smart". What happens: since the LLM already processes the case, decisions keep getting added —"let it also decide the store credit, and while it's at it whether to waive shipping"—, even if each one is a deterministic rule. Each extra decision in the core is surface that can hallucinate and that has to be contained. Why it happens: it's more convenient to ask the model for one more thing than to write a rule, and the model "seems" capable of deciding it. How to detect it: for each decision your core makes, ask yourself if it's an if in disguise; if you can write it as a condition over data you already have, it's a rule, not a model decision. How to fix it: move out of the core every decision that can be a rule, and leave it only what needs to understand language or reason about the ambiguous. The example measures it: moving three of four decisions out lowered the probabilistic surface by 75%.

Confusing "making the core better" with "making the core smaller". What happens: to reduce the risk, the team invests in improving the model —better prompt, bigger model— but keeps the four decisions in the core. The error rate goes down, but the surface stays: there are still 80 probabilistic decisions over 20 cases, they just fail a bit less often. Why it happens: two distinct axes are confused —the core's quality and the core's size—. How to detect it: if your safety strategy is "improve the model" and not "reduce how much the model decides", you're moving one axis and not the other. How to fix it: do both, but recognize that they're distinct —improving the model (AI Eng) lowers the rate; shrinking the core (architecture) lowers the surface—. The small surface protects you even when the rate doesn't go to zero, which is always.

Leaving the escalation decision in the core. What happens: the model is asked to decide when to escalate to a human —"escalate if you're not sure"—, when the escalation is precisely the safety net that should be deterministic. The day the model should escalate but, confident, doesn't, the emergency exit is lost. Why it happens: "knowing when you don't know" sounds like something the model should judge. How to detect it: if the escalation decision depends on the model's judgment and not on a rule, your safety net has the same fallibility as what it protects you from. How to fix it: make the escalation a deterministic rule —escalate if the amount exceeds the limit, if the order can't be verified, if the proposal failed the validation—. The escalation is the shell catching what the core shouldn't decide alone; putting it in the core is asking the fox to guard the henhouse.

Exercises

Exercise 1 — Core or rule. Mercado's "describe your product" generator makes these five decisions. For each one, say whether it should live in the core (LLM) or in the shell (deterministic rule), and why: (a) writing the description text; (b) deciding whether the text exceeds 200 characters; (c) choosing the tone (formal/casual) according to the product's category; (d) deciding whether to publish or send to review; (e) translating the description to English.

See solution

(a) Writing the text → core. Generating natural language from attributes is exactly what an LLM does and a rule couldn't. It needs intelligence. It stays in the core.
(b) Does it exceed 200 characters? → shell (rule). It's len(text) > 200: a deterministic calculation over data you already have. It doesn't need judgment. It moves out of the core.
(c) Choosing the tone according to the category → shell (rule), or core as an input. If the tone is decided by the category (if category == "books": formal), it's a deterministic rule —the category→tone mapping is fixed—. The model applies the tone when writing (that's core), but deciding which one is a rule. Don't let the model choose the tone freely if there's a fixed mapping.
(d) Publish or review? → shell (rule). It's a state decision that should be deterministic: publish if it passed the content validation, send to review if not. Like the escalation, this is a safety net that shouldn't depend on the model's judgment. It moves out of the core.
(e) Translate to English → core. Translating natural language needs the model. It stays in the core (or in another AI component), but it's genuinely an intelligence task.

The pattern: (a) and (e) need intelligence (core); (b), (c), and (d) are rules or fixed mappings (shell). The discipline: leave in the core only what genuinely needs to understand or generate language, and turn everything else into rules —especially the state decisions like publishing or reviewing—.

Exercise 2 — The surface number. In the example, the fat core has 80 probabilistic decisions and the thin core 20, over 20 cases. Suppose the model hallucinates in 2% of its decisions. How many hallucinated decisions would you expect in each design? Use the result to argue why shrinking the core protects even with a good model.

See solution

With a hallucination rate of 2%:

Fat core: 80 probabilistic decisions × 2% = 1.6 hallucinated decisions expected (over 20 cases).
Thin core: 20 probabilistic decisions × 2% = 0.4 hallucinated decisions expected (over 20 cases).

The thin core expects four times fewer hallucinations than the fat core, with the same model and the same 2% rate. The difference doesn't come from a better model —it's identical— but from the thin core exposing less surface to that rate: 20 decisions instead of 80.

The argument: improving the model lowers the rate (say, from 2% to 1%), but the rate never reaches zero —an LLM always hallucinates some fraction—. Shrinking the core, on the other hand, lowers the surface directly, and multiplies its effect by that of any model improvement: a thin core with a model at 1% expects 20 × 1% = 0.2 hallucinations, ten times fewer than the original fat core. The two levers multiply, but the surface one is architectural (you control it, with certainty) and the rate one is statistical (it depends on the model, it's never perfect). That's why shrinking the core protects even —above all— when the model is already good: over a small base, even a low rate produces few absolute hallucinations.

Exercise 3 — Shrink this core. A team designed a support agent where the LLM decides, in a single call: (1) which order is the customer's, (2) the refund amount, (3) whether the customer is "premium" and deserves special treatment, (4) whether to apply the exception policy, and (5) the response message. Identify which decisions should move out of the core to deterministic rules, which stay, and what the design gains.

See solution

Decision-by-decision analysis:

(1) Which order is the customer's → mixed. Interpreting "the order that arrived broken" is the core's (it needs to understand language), but resolving that to a concrete order_id must be validated against the source of truth (does that order of this customer exist?). The model proposes which one it thinks it is; the shell verifies it exists and belongs to the customer.
(2) Refund amount → core. It depends on understanding the customer's claim ("I was overcharged for shipping"). It stays, but it passes through the rule validation (L5) before executing.
(3) Whether the customer is "premium" → shell (rule). It's a datum in the database: customer.tier == "premium". It's not a judgment; it's a query. It moves out of the core —letting the model guess the customer's tier is absurd and dangerous—.
(4) Whether to apply the exception policy → shell (rule). Exceptions have defined conditions (premium tier + within X days + amount under Y). It's a conjunctive rule, not a model judgment. It moves out of the core.
(5) The response message → core. Generating the friendly response in natural language is the model's. It stays (probably as the text field of a send_message).

What moves out: (3) and (4) —the premium tier and the application of exceptions— are data and rules, not judgments; they move to the shell. (1) splits: the interpretation is the core's, the resolution/verification is the shell's. What stays: (2) the amount and (5) the message, which genuinely need intelligence.

What the design gains: the probabilistic surface goes down from 5 decisions to ~2-3 (depending on how you count the mixed one). The most dangerous and verifiable decisions —the customer's tier, whether an exception applies— stop depending on the model guessing them well and become deterministic queries and rules, which don't make mistakes. The model keeps what only it can do (understand the claim, write the response), and everything verifiable becomes certain. Less surface to contain, more reliable state decisions, same feature.

Summary and next step

In this lesson you stepped back from containing to ask how much the model should decide in the first place, and you reached the discipline that gives the guide its name: keep the probabilistic core small. Each decision you move out of the core to a deterministic rule is one that can no longer hallucinate —and that you no longer have to contain—. You measured it: a fat core with four decisions per case exposes 80 probabilistic decisions over 20 cases; a thin core with one exposes 20 —75% less surface that can hallucinate—, without changing the model, only changing what it decides. You saw that the design question is "does this decision need intelligence or is it a rule in disguise?"; that shrinking the core doesn't limit the feature (the four decisions are still made, only who changes); that it's a distinct axis from "improving the model" (surface vs rate, and they multiply); and that the escalation decision, in particular, should be a rule and not a model judgment.

Before moving on you should be able to: distinguish the core's surface from the core's quality; identify decisions that are rules in disguise and move them out of the core; argue why shrinking the core protects even with a good model; and explain why the escalation belongs to the shell.

Lesson 7 assembles everything you built into a single flow: the proposal → execution pipeline. You already have the pieces —the model proposes (L2), in a structured format (L3), from a bounded menu (L4), validated against rules (L5), with a small core (L6)—; now you put them in order as gates in series —structure → capability → policy → execute— with an audit log that records what was proposed, what was approved, and what was blocked at each stage. You'll see the complete pipeline running over a batch, showing at which stage each blocked proposal stopped. The deterministic shell, whole and executed.

Resources

Anthropic, "Building Effective Agents" (2024) — anthropic.com/engineering/building-effective-agents. Its principle of keeping the AI component simple and bounded, and putting the deterministic work around it, is exactly this lesson's "small core" discipline applied to actions. In English.
Martin Fowler and Bharani Subramaniam, "Emerging Patterns in Building GenAI Apps" — martinfowler.com/articles/gen-ai-patterns. The discussion of how much to delegate to the model and how much to leave in deterministic code is the patterns version of the core/shell decision. In English.
Chip Huyen, AI Engineering (O'Reilly, 2024). Its treatment of application architecture —the model as a component surrounded by application logic— is the extended version of the core/shell metaphor. Here we apply it to the decisions that touch state. In English.
The architecture-decisions-and-tradeoffs-guide (this same ecosystem). The decision of what lives in the probabilistic core and what in the deterministic shell is an architectural tradeoff —intelligence vs certainty— of the kind that guide treats as an explicit design decision. In Spanish.

← PreviousValidating the proposal against the business rules Next →The proposal → execution pipeline