Module 1: What Changes When a Component Is Non-Deterministic
The probabilistic core and the deterministic shell
Overview
So far the module has given you loose pieces: a contract that verifies properties (lesson 2), a boundary that validates proposals (lesson 3), five properties that arrive all at once (lesson 4). This lesson joins them into a single idea —the one that gives the whole guide its name and orders everything that follows—: an AI-native system is a probabilistic core (the LLM, uncertain, variable, fallible) that lives inside a deterministic shell (normal code, certain, testable) that contains it. The core contributes the intelligence no deterministic function could give; the shell contributes the guarantee that this intelligence never touches money, state, or the user's trust without passing through hard rules. The design principle that follows is as simple as it is powerful: keep the core small and contained, and load into the shell everything that gives guarantees.
You're going to see this idea measured. A core that sometimes returns garbage (an invented category, an empty string) runs twice: once naked, with no shell, and the garbage reaches the user; another time wrapped in a deterministic shell that contains it, and not a single unsafe output escapes. The difference between 5 garbage outputs out of 12 and 0 out of 12 isn't a trick: it's the complete shape of an AI-native system in a minimal example.
Connection with the module. This is the metaphor-lesson, the conceptual center of the module. Lessons 2, 3, and 4 kept showing pieces of the shell without naming it; here we give it its name and its shape, and from here on everything is ordered under it. Lesson 6 measures how much shell each feature needs (thickness); lesson 7 lists the fields of the shell (the property sheet); and lesson 8 applies it to a real feature. The boundary with the rest of the guide is precise: the deterministic shell in depth is module 6 —what patterns compose it, how to keep the core minimal, how to compose validations—. Here we install the metaphor and measure it in its simplest form; we don't exhaust the shell. And the boundary with AI Engineering stays firm: we don't talk about how to make the core smarter (better prompt, better model, RAG) —that's AI Eng—; we talk about how to contain the core you have, whatever its quality.
An analogy: the nuclear reactor and its containment building
A nuclear reactor produces an amount of energy that no other compact source can match. It's also intrinsically dangerous: reactions that, uncontrolled, run away. Nobody builds a "naked" reactor, out in the open, trusting the reaction to behave. A containment building is built around it: concrete and steel walls several meters thick, redundant cooling systems, barriers that ensure that —whatever happens inside— nothing dangerous escapes to the outside. Nuclear engineering isn't about making the reaction "safe by itself" (it isn't); it's about containing it so its enormous energy is usable without putting the outside at risk.
Notice the division of responsibilities. The reactor (the core) does one thing: produce energy. It doesn't decide where that energy goes, doesn't manage safety, doesn't talk to the power grid. The containment and the surrounding systems (the shell) do everything else: contain, cool, monitor, convert, distribute, cut off if something goes out of range. And a golden rule: the core is kept as small and bounded as possible. The bigger and more spread out the reactive core, the harder to contain. Safety lives in keeping the core compact and the containment robust.
Here's the point: an LLM is the reactor; the deterministic shell is the containment building. The LLM produces something enormously valuable —language understanding, reasoning, generation— that no deterministic function can give. And it's intrinsically uncertain: variable, fallible, sometimes "runaway" (a hallucination, a response to an injection). AI-native design doesn't try to make the LLM "safe by itself" —impossible, just like a reactor—; it contains it. It keeps the probabilistic core small (so it only proposes, one responsibility) and puts around it a robust deterministic shell that validates, bounds, degrades, and guarantees that nothing dangerous escapes to the system. In Mercado, the category classifier is the reactor; the shell that only lets valid categories through is the containment; and no invented category ever reaches the catalog, however much the reactor produces it.
Worked example: the shell contains, measured
We're going to measure the containment. The core is a probabilistic_core: a stub that simulates an LLM suggesting a product's category. Deliberately, it sometimes suggests a valid category and sometimes returns garbage —a misspelled category, an invented one, an empty string—, because the point is to see what the shell does with the garbage. We run the same core twice: naked (no shell) and contained (with shell), with the same seed, and count how many unsafe outputs escape in each case.
# Lesson 5: the probabilistic core inside the deterministic shell.
# The guide's central idea: keep the core (the LLM) SMALL and CONTAINED;
# the deterministic shell validates, bounds, and gives a safe output no matter what.
import random
_RNG = random.Random(3)
# --- Probabilistic core: SIMULATES an LLM that suggests a category tag. ---
# Sometimes it suggests a valid category; sometimes it returns garbage (hallucination).
VALID_CATEGORIES = {"audio", "computing", "home", "fashion", "books"}
def probabilistic_core(product_title):
# The simulated LLM: a single responsibility, and UNtrusted output.
candidates = ["audio", "computing", "home", "fashion", "books",
"electrncs", "", "made-up-category-42"]
return _RNG.choice(candidates)
# --- Deterministic shell: contains the core's non-determinism. ---
def deterministic_shell(product_title):
suggestion = probabilistic_core(product_title) # the core PROPOSES
if suggestion in VALID_CATEGORIES: # the shell VALIDATES
return suggestion
return "unclassified" # deterministic fallback
N = 12
print("=== Without shell: the core's raw output reaches the user ===")
_RNG.seed(3)
leaked = 0
for i in range(N):
raw = probabilistic_core("BT Headphones")
is_valid = raw in VALID_CATEGORIES
if not is_valid:
leaked += 1
print(f" req {i:>2}: {raw!r:<24} {'ok' if is_valid else 'GARBAGE TO THE USER'}")
print(f"Invalid outputs that escaped: {leaked}/{N}")
print()
print("=== With shell: the core is contained, the output is ALWAYS valid ===")
_RNG.seed(3)
unsafe = 0
for i in range(N):
result = deterministic_shell("BT Headphones")
safe = result in VALID_CATEGORIES or result == "unclassified"
if not safe:
unsafe += 1
print(f" req {i:>2}: {result!r:<16} {'safe' if safe else 'UNSAFE'}")
print(f"Unsafe outputs that escaped: {unsafe}/{N}")
print()
print("=== Surface: responsibilities of the core vs the shell ===")
print(" core (probabilistic): 1 -> propose a tag")
print(" shell (deterministic): 3 -> validate, fall back, guarantee output")
print("Rule: keep the core small and contained; the shell carries the rest.")What to expect. When you run the file, the output is exactly this:
=== Without shell: the core's raw output reaches the user ===
req 0: 'fashion' ok
req 1: 'home' ok
req 2: 'electrncs' GARBAGE TO THE USER
req 3: 'made-up-category-42' GARBAGE TO THE USER
req 4: 'computing' ok
req 5: 'audio' ok
req 6: 'made-up-category-42' GARBAGE TO THE USER
req 7: 'books' ok
req 8: 'fashion' ok
req 9: 'fashion' ok
req 10: 'made-up-category-42' GARBAGE TO THE USER
req 11: 'made-up-category-42' GARBAGE TO THE USER
Invalid outputs that escaped: 5/12
=== With shell: the core is contained, the output is ALWAYS valid ===
req 0: 'fashion' safe
req 1: 'home' safe
req 2: 'unclassified' safe
req 3: 'unclassified' safe
req 4: 'computing' safe
req 5: 'audio' safe
req 6: 'unclassified' safe
req 7: 'books' safe
req 8: 'fashion' safe
req 9: 'fashion' safe
req 10: 'unclassified' safe
req 11: 'unclassified' safe
Unsafe outputs that escaped: 0/12
and ends with:
=== Surface: responsibilities of the core vs the shell ===
core (probabilistic): 1 -> propose a tag
shell (deterministic): 3 -> validate, fall back, guarantee output
Rule: keep the core small and contained; the shell carries the rest.
Compare the two runs, because the difference between them is the lesson.
Without a shell, the naked core produces what it produces, and what it produces reaches the user as is. In 12 requests, 5 garbage outputs escaped: twice the misspelled category 'electrncs', and four times the invented 'made-up-category-42' (one of them would have been an empty string on another seed). Each of those five is a mislabeled product in Mercado's catalog, visible to the customer, broken. The core did what a probabilistic model does; the absence of containment turned its fallibility into exposed errors.
With a shell, the core is identical —same seed, same garbage suggestions inside—, but now each suggestion passes through deterministic_shell before going out. Valid categories pass; garbage ones become 'unclassified', a deterministic, honest, safe fallback. In 12 requests, 0 unsafe outputs escaped. The core kept failing exactly as often; the difference is that its failure no longer reaches the user. That's containment: you didn't eliminate the core's fallibility (you can't), you contained it so it doesn't escape.
And the closing measures the design principle: the core has 1 responsibility (propose a tag), the shell has 3 (validate, fall back, guarantee an always-valid output). That asymmetry is deliberate and is the reactor's golden rule: the core small and bounded, the containment carrying all the weight of the guarantees. A well-designed AI-native system looks like this —a drop of uncertainty surrounded by an ocean of deterministic, certain, testable code—.
Going deeper: the shape of an AI-native system
It's worth drawing the complete shape, because it's the mold for everything that comes.
┌──────────────────────────────────────────────┐
│ DETERMINISTIC SHELL (certain) │
│ │
│ validate input · validate output · fallback │
│ budget · eval gate · guardrail │
│ │
│ ┌───────────────────────────┐ │
│ │ PROBABILISTIC CORE │ │
│ │ (the LLM: uncertain) │ │
│ │ one responsibility: │ │
│ │ PROPOSE │ │
│ └───────────────────────────┘ │
│ │
└──────────────────────────────────────────────┘
│
▼
state / money / user
(only what the shell approved)
The core is small on purpose. The temptation is to put more into the core —have the LLM not only propose the category but also decide whether to publish, compute the price, send the email—. Every extra responsibility you give the core is one more piece of system that inherits the uncertainty: more surface to contain, more ways to fail without a guarantee. The AI-native discipline is the opposite: take out of the core everything that can be deterministic. The LLM proposes a category; the shell decides what to do with it. The LLM proposes a refund; the shell validates the policy. The smaller the core, the more of the system is certain and testable, and the easier to contain is the little that stays uncertain.
The shell is where the whole guide lives. Look at the labels inside the shell in the diagram: validate input and output (guardrails, module 4), fallback (module 5), latency/cost budget (module 2), eval gate (module 3). It's no coincidence: each module of this guide is a part of the shell. Learning to build AI-native systems is, almost literally, learning to build the shell —because AI Engineering builds the core, and here we treat it as a box that proposes—. Lesson 7 makes this explicit with the property sheet, where each field is a portion of the shell and points to its module.
The shell guarantees; the core can't. This is the deep reason the design works. You can't guarantee anything about the core's output —it's probabilistic, it can hallucinate—. But you can guarantee things about the shell, because it's deterministic: "no invalid category reaches the catalog" is a guarantee the shell always meets, no matter what the core does, because if suggestion in VALID_CATEGORIES is a hard condition. The whole system gets strong guarantees despite having a component with no guarantees, and the trick is that the guarantees come from the containment, not from the core. That's why the shell is deterministic and not another LLM: only deterministic code can promise something with certainty.
Containing isn't wasting the core. It might seem that if the shell "fixes" so much, the core matters little. It's the opposite: the shell exists in order to be able to use such a powerful core. Without containment, you wouldn't dare put an LLM near money or the catalog —it would be too risky—. The shell is what makes it safe to leverage the core's intelligence, just as containment is what makes it safe to leverage the reactor's energy. They don't compete: the shell is the condition that enables the core.
Common mistakes
Putting too many responsibilities into the core. What happens: the team, excited about how capable the model is, tasks it with a whole chain —"let it classify the product, decide whether it's fit to publish, suggest the price, and write the description"—, all in a single LLM call. Each link in that chain inherits the model's uncertainty. Why it happens: it's more convenient to ask the core for everything than to separate the deterministic part. How to spot it: your core produces decisions that have effects (publish, set price), not just proposals the shell validates. How to fix it: take out of the core everything that can be deterministic and leave it a single proposal responsibility. Classification can be the LLM's; "is it fit to publish" is a deterministic rule over that classification; "the price" shouldn't be the LLM's at all. Small core, robust shell.
Building the shell with another LLM. What happens: to "contain" the model, someone puts a second model to review the first's outputs, believing that validates it. Why it happens: reviewing sounds like "understanding," and understanding is what an LLM does well. How to spot it: your containment layer is itself non-deterministic; you can't guarantee anything about it. How to fix it: the shell has to be deterministic, because only the deterministic gives guarantees. A second LLM doesn't contain the first; it just adds another core that in turn would need containing —an infinite tower of models reviewing each other with nobody promising anything—. The guarantee has to come, at some point, from a hard code condition. (A second model can help filter, but it isn't the containment; the containment is the final deterministic rule.)
Believing a better core eliminates the need for a shell. What happens: "when we use the big model and tune the prompt, it'll hallucinate so little that validating the output won't be necessary." Why it happens: "less fallible" is confused with "infallible," and the system's safety is bet on the core's quality. How to spot it: your justification for having no shell is the model's quality. How to fix it: remember the reactor —you don't contain a reactor less because the fuel is higher quality—. A better core hallucinates less often, but it still hallucinates, and at Mercado's scale "less often" is thousands of errors. The shell isn't proportional to the core's fallibility; it's the guarantee that makes the error impossible to escape, and you want that guarantee whatever model you have. Improving the core (AI Eng) and containing it (this guide) are two different jobs, and the second doesn't disappear with the first.
Exercises
Exercise 1 — Core or shell. For Mercado's "describe your product" generator, classify each responsibility as part of the probabilistic core (only the LLM can do it, it's an uncertain proposal) or of the deterministic shell (must be certain code with guarantees): (a) writing the description text from the attributes; (b) verifying that the description doesn't exceed 200 characters; (c) rejecting the description if it contains a forbidden claim; (d) deciding the product's sale price.
See solution
- (a) Write the description → core. It's exactly what an LLM does and a deterministic function couldn't: generate natural language from attributes. It's an uncertain proposal (it'll vary between calls). It's the only responsibility that belongs to the core.
- (b) Verify ≤ 200 characters → shell. It's a hard, deterministic condition (
len(text) <= 200), with a clear guarantee. Nothing probabilistic. It belongs to the containment. - (c) Reject forbidden claims → shell. Also deterministic: a list of forbidden terms and a membership check. It gives a guarantee ("no description with a forbidden claim gets published") that only certain code can promise.
- (d) Decide the price → neither core nor shell: the LLM probably shouldn't be involved. The price is a business decision with its own rules; it isn't a language-generation task. Putting it into the core would be the "too many responsibilities" mistake. If at most the LLM suggests a range, the final decision is deterministic (or human) in the shell; but the healthiest thing here is for the price simply not to go through the model.
The healthy shape: a core with a single responsibility (a), surrounded by a shell with several guarantees (b, c), and business responsibilities (d) that don't even enter the core.
Exercise 2 — Measure the containment. In the example, without a shell 5 of 12 garbage outputs escaped and with a shell 0 of 12. If the core improved (fewer hallucinations) and its garbage rate dropped from ~40% to ~10%, how many unsafe outputs would escape with a shell? And without a shell, at the scale of a million classifications a month? Use the numbers to argue why the shell is still necessary even if the core improves.
See solution
With a shell, 0 would escape, same as before. The shell guarantees that no invalid output passes, and that guarantee doesn't depend on the core's garbage rate: if suggestion in VALID_CATEGORIES blocks 40% or 10% with the same certainty. However much the core improves, the containment stays at 0 escapes.
Without a shell, at a 10% rate over a million classifications a month, on the order of 100,000 garbage outputs a month would escape —a hundred thousand mislabeled products, visible to customers—. The core having improved from 40% to 10% reduced the disaster (from ~400,000 to ~100,000), but a hundred thousand exposed errors a month are still unacceptable.
The argument: improving the core reduces the amount of garbage it produces, but doesn't bring it to zero (an LLM always has an error rate > 0), and at scale any positive rate is many errors in absolute terms. The shell, by contrast, brings the escapes to zero regardless of the core's rate. That's why the two things are complementary and not substitutes: AI Engineering lowers the core's garbage rate; this guide's shell guarantees that the garbage that's left doesn't escape. Betting safety only on improving the core is accepting "100,000 errors a month" as the floor; adding the shell brings it to zero.
Exercise 3 — The honest fallback. In the example, when the core suggests garbage, the shell returns 'unclassified' instead of inventing a category or letting the garbage through. Explain why 'unclassified' is a better fallback than (a) choosing a category at random, or (b) using the last valid category the core suggested, and what design principle it illustrates.
See solution
'unclassified' is better than both alternatives because it's honest about the uncertainty instead of disguising it:
- (a) Choosing a category at random would be inventing information. The product would end up, say, in "books" with no basis —just as bad as the core's garbage, but now disguised as valid data, impossible to distinguish from a real classification—. It turns a visible error ("I don't know") into an invisible one ("false data that looks true"), which is worse.
- (b) Reusing the last valid category drags data from another product into this one. A keyboard could inherit "fashion" because the previous product was a T-shirt. Again: invented information that looks legitimate.
'unclassified' tells the truth: "the core didn't give a usable answer, and the system acknowledges it." That allows correct downstream handling —show the product in an uncategorized section, queue it for manual classification, or ask the seller to choose—. The principle it illustrates: a fallback should degrade safely and honestly, not fabricate certainty that doesn't exist. An explicit, contained "I don't know" is better than an invented "yes" that leaks through as truth. Module 5 develops the fallback patterns; here the principle is that the fallback preserves the system's honesty about what it knows and what it doesn't.
Summary and next step
In this lesson you joined the module's loose pieces into the central idea of the whole guide: an AI-native system is a probabilistic core inside a deterministic shell. The core (the LLM) contributes the intelligence and is uncertain; the shell (normal code) contributes the guarantees and is certain. You measured it with the reactor and its containment: the same fallible core let 5 of 12 garbage outputs escape without a shell and 0 of 12 with a shell, without the core changing —the containment, not a perfect core, is what gives the guarantee—. And you saw the design principle: keep the core small (one responsibility: propose), load into the shell everything that guarantees, make it deterministic because only the deterministic promises with certainty, and use honest fallbacks that don't fabricate nonexistent certainty.
Before moving on you should be able to: draw the core/shell shape of an AI-native system; explain why the core is kept small on purpose; argue why the shell must be deterministic and not another LLM; and why a better core doesn't eliminate the need for a shell.
You already have the shape. Lesson 6 asks the natural question that follows: if every AI-native system is core plus shell, do all features need the same shell? No. You're going to see, executed, that each Mercado feature's tolerance to non-determinism —what you measured at a bird's-eye view in lesson 1— determines how thick its shell must be: semantic search calls for a thin one, the refunds agent for one that covers everything. From the tolerance are derived, calculated, each feature's mandatory containment mechanisms.
Resources
- Anthropic, "Building Effective Agents" (2024) — anthropic.com/engineering/building-effective-agents. The insistence on keeping the AI component simple and bounded, and on putting the deterministic work around it, is exactly this lesson's "small core, robust shell" principle. In English.
- Martin Fowler and Bharani Subramaniam, "Emerging Patterns in Building GenAI Apps" — martinfowler.com/articles/gen-ai-patterns. The guardrail and output-verification patterns are this lesson's containment walls, described as reusable patterns. In English.
- Chip Huyen, AI Engineering (O'Reilly, 2024). Its treatment of application architecture —the model as a component surrounded by application logic— is the extended version of the core/shell metaphor. We keep the containment; building the core is the boundary with AI Eng. In English.
- Michael Nygard, Release It! (2nd ed., Pragmatic Bookshelf, 2018). Although it's about resilience in distributed systems (not AI), the containment patterns —bulkhead, circuit breaker— are the same idea: contain a risky component so its failure doesn't propagate. The AI-native shell is a direct relative. It's applied in module 5. In English.