How much non-determinism can a feature tolerate?

Overview

You already have the shape of every AI-native system: a probabilistic core inside a deterministic shell (lesson 5). The question this lesson opens is the one that decides how much architecture each feature deserves: do all features need the same shell? The answer is no, and the difference isn't a matter of taste —it can be measured—. Each AI feature tolerates a different amount of non-determinism. Semantic search isn't much affected if one day it orders the results differently; the agent that executes refunds costs real money and trust with one error. From that tolerance is directly derived how thick the shell must be: which containment mechanisms are mandatory and which are optional.

This lesson turns the intuition you glimpsed in lesson 1 into a method. You're going to see, executed, the feature × mechanism matrix: for each Mercado feature, the computed tolerance and —derived from it— the containment mechanisms that become mandatory. Semantic search, tolerant, calls for a thin shell (almost just an eval and a data loop); the refunds agent, intolerant, calls for everything, including the complete deterministic shell because it touches money. The same core/shell pattern, sized to the measure of each feature's risk.

Connection with the module. Lesson 5 gave you the shell; this one tells you how much shell. It's the lesson that makes the metaphor applicable: without a criterion for sizing it, you either over-design tolerant features or —worse— under-design the dangerous ones. Lesson 7 takes this one's result —which mechanisms a feature needs— and formalizes it into the property sheet; lesson 8 applies it to a real feature. The boundary with the rest of the guide: here we decide which mechanisms each feature needs, not how each is built —the eval in depth is module 3, the guardrails module 4, the fallback module 5, the shell module 6—. And the boundary with AI Engineering holds: tolerance is measured by the impact of a feature's error on the system (money, state, trust), not by how "smart" the model is internally.

An analogy: levels of review in a company

In any healthy company, not all decisions go through the same level of review. The level of scrutiny is proportional to what's at stake.

A new employee can, without anyone approving it, answer a customer's email, write a draft, propose an idea in a meeting. If they're wrong, the cost is low and reversible: the email is corrected, the draft is discarded. Their work is tolerant to error, so the company gives them autonomy —reviewing each of their emails would be absurd and would make them useless—.

That same employee can't, on their own, sign a million-peso contract, transfer the company's money, or fire someone. Those decisions are intolerant to error: a mistake costs a lot, is hard to reverse, affects many people. That's why they go through several levels of approval —a manager, finance, sometimes legal, sometimes the board—. Not because the employee is dumb, but because what's at stake demands containment, no matter how good the employee is.

And there's a middle point: proposing a discount to a customer might require a supervisor's sign-off, but not the board's. The level of review rises in steps, to the measure of the risk.

Here's the point: the deterministic shell of an AI feature is its level of review, and it's sized the same way as in the company —by what's at stake, not by how good the "employee" (the model) is—. Semantic search is the new employee answering emails: tolerant, little review, thin shell. The refunds agent is the one who wants to sign a company check: intolerant, maximum review, a shell that covers everything. Giving both the same shell is either suffocating search with bureaucracy, or —the expensive mistake— letting the agent sign checks with no approval. This lesson computes, feature by feature, how much scrutiny each one deserves.

Worked example: from tolerance to the mandatory mechanisms

We're going to measure each Mercado feature's tolerance and derive from it which containment mechanisms are mandatory. Tolerance is the same as in lesson 1 —the inverse of the risk, 18 - (touches + error_cost + blast)—, and the derivation rule translates that risk into mechanisms: some are always mandatory (every LLM needs an eval gate and a data loop), and others activate depending on how much it touches money, how much the error hurts, and how much it propagates.

# Lesson 6: how much non-determinism each feature tolerates, and what it requires.
# Tolerance is NOT an opinion: we compute it, and from it we derive the
# MANDATORY containment mechanisms for each Mercado feature.

FEATURES = [
    # (name, touches_money_or_state, error_cost, blast_if_wrong)
    ("semantic_search",        1, 2, 2),
    ("recommendations",        1, 2, 3),
    ("describe_your_product",  1, 3, 2),
    ("support_agent_refunds",  5, 5, 5),
]


def nd_tolerance(touches, error_cost, blast):
    return 18 - (touches + error_cost + blast)  # 15 = very tolerant, 3 = none


def required_mechanisms(touches, error_cost, blast):
    tol = nd_tolerance(touches, error_cost, blast)
    # Lower tolerance -> more mandatory mechanisms.
    req = {"eval_gate": True, "feedback_loop": True}  # always, for every LLM
    req["guardrail"] = error_cost >= 3 or touches >= 3
    req["fallback"] = blast >= 3 or touches >= 3
    req["deterministic_shell"] = touches >= 3         # touches money/state
    return tol, req


MECHS = ["eval_gate", "guardrail", "fallback", "deterministic_shell",
         "feedback_loop"]

ranked = sorted(FEATURES, key=lambda f: nd_tolerance(f[1], f[2], f[3]),
                reverse=True)

header = f"{'feature':<24}{'tol':>4}  " + "".join(f"{m[:9]:>11}" for m in MECHS)
print(header)
print("-" * len(header))
for name, touches, error_cost, blast in ranked:
    tol, req = required_mechanisms(touches, error_cost, blast)
    cells = "".join(f"{('req' if req[m] else '-'):>11}" for m in MECHS)
    print(f"{name:<24}{tol:>4}  {cells}")

print()
print("Read the top row (semantic_search, tolerant): thin shell,")
print("only eval_gate + feedback. The bottom one (refunds, intolerant): EVERYTHING,")
print("including the deterministic shell because it touches money.")

What to expect. When you run the file, the output is exactly this:

feature                  tol    eval_gate  guardrail   fallback  determini  feedback_
-------------------------------------------------------------------------------------
semantic_search           13          req          -          -          -        req
recommendations           12          req          -        req          -        req
describe_your_product     12          req        req          -          -        req
support_agent_refunds      3          req        req        req        req        req

Read the top row (semantic_search, tolerant): thin shell,
only eval_gate + feedback. The bottom one (refunds, intolerant): EVERYTHING,
including the deterministic shell because it touches money.

Read the matrix from top to bottom, because it's a tolerance thermometer and, at the same time, a containment blueprint.

At the top, semantic_search with tolerance 13. Its row is almost all dashes: only eval_gate and feedback_loop marked as mandatory. Translation: its shell is thin. It needs an eval (to know whether a model or prompt change improves or worsens the search) and a data loop (clicks and purchases improve the system), but it doesn't need a heavy guardrail, nor a mandatory fallback, nor a deterministic action-validation shell —because it touches neither money nor state, and a slightly different result harms no one—. Containing it more would be the mistake of reviewing the new employee's every email.

At the bottom, support_agent_refunds with tolerance 3. Its row is all marked req: eval, guardrail, fallback, deterministic shell, and data loop. Translation: its shell is thick, it covers everything. It touches money (that's why the deterministic_shell is mandatory: the proposal is validated against policy before executing), an error hurts a lot (that's why the guardrail on input and output), and its blast radius is enormous (that's why the mandatory fallback for when the model fails). It's the employee who wants to sign a check: maximum review, no exception.

In the middle, recommendations and describe_your_product, both with tolerance 12 but with different shells, and this is the interesting part. The total tolerance isn't enough; what matters is where the risk comes from. Recommendations activate the fallback (its blast_if_wrong is 3: if the model goes down, many pages are left with no recommendations, so you need a degraded route —for example, "best sellers"—). "Describe your product," by contrast, activates the guardrail (its error_cost is 3: the description gets published, so false claims must be forbidden). Same tolerance, different mandatory mechanisms, because the risk enters through different axes. The lesson: the shell isn't sized with a single global number, but with the feature's risk profile.

Going deeper: the tolerance spectrum and how to read it

It's worth understanding the method well, because you're going to apply it to any AI feature, not just Mercado's four.

Tolerance measures impact, not the model's quality. A common mistake is to think "the refunds agent needs more shell because the model is worse there." False: it could use the best model in the world and would still need the thickest shell, because what determines it is what happens if it's wrong (it touches money), not how often it's wrong. Tolerance is a property of the feature in the system, not of the model. That's why it's computed with touches_money_or_state, error_cost, and blast_if_wrong —three impact axes— and none of them measures the core's quality. This connects directly with lesson 5: improving the core (AI Eng) doesn't change the tolerance; it only changes how much garbage it produces, not how much it hurts when it leaks.

The spectrum has two poles and a populated middle.

  tolerant  ◄─────────────────────────────────────────►  intolerant
  (thin shell)                                          (thick shell)

  semantic_search      recommendations                   support_agent_refunds
  describe_your_product
  ─────────────────    ─────────────────                 ─────────────────────
  eval + feedback      + fallback                         eval + guardrail +
                       + guardrail                        fallback + deterministic
                       (depending on axis)                shell + feedback

At the tolerant pole, the shell is almost just observation: you measure quality (eval) and learn from usage (feedback), but you leave the core fairly loose because its error does no harm. At the intolerant pole, the shell is a cage: every output is validated, contained, backed by a fallback, and nothing touches money without passing through deterministic rules. The middle is where most real features live, and there judgment matters: the same total tolerance can call for different mechanisms depending on which axis the risk enters through.

Why eval and feedback are always mandatory. Notice that the first two columns are marked for all features, even the most tolerant. It's deliberate: every AI component, however innocuous it seems, needs at minimum a way to measure its quality (the eval, to know whether a change improved it or broke it —module 3—) and a way to learn from usage (the feedback, the data loop —module 7—). Without an eval, you change the model blind; without feedback, the system never improves. They're the floor of the shell, not a luxury for the dangerous features. The other three mechanisms —guardrail, fallback, deterministic shell— are added according to the risk.

The symmetric mistake of sizing wrong. As with architectural decisions in general, being wrong has two directions and they're not equally serious. Over-sizing a tolerant feature (giving semantic search a deterministic action-validation shell when it validates no action) wastes effort and can make the feature slow or rigid —a real but bounded cost—. Under-sizing an intolerant feature (letting the refunds agent execute with no deterministic shell) risks money, trust, and security —a potentially catastrophic cost—. That's why, when in doubt, the cheaper mistake is to over-size the features that touch money or state, and under-size the ones that don't. This lesson's matrix is exactly the tool for not sizing by eye.

Common mistakes

Giving all AI features the same shell. What happens: the team defines an internal "AI framework" with a fixed shell —always a guardrail, always heavy validation, always review— and applies it the same to search as to the agent. Search ends up over-designed and slow; or, in the opposite version, they define a light shell "to go fast" and apply it to the refunds agent too, which ends up dangerously unprotected. Why it happens: a single shell is easier to standardize than one sized per feature. How to spot it: your process applies the same level of containment to features with very different risk profiles. How to fix it: size the shell by each feature's tolerance. Compute the impact (the three axes), derive the mandatory mechanisms, and adjust. Standardize the method (how to size), not the shell (the result).

Sizing by the model's quality instead of by the impact. What happens: "the model is great at classifying, it almost never gets it wrong, so the classification doesn't need a shell." The containment is sized by how good the model seems. Why it happens: it's intuitive to think a better model needs less protection. How to spot it: your justification for the shell's thickness mentions the model's quality, not the impact of an error. How to fix it: separate the two questions. "How often is it wrong?" (quality, which AI Eng lowers) is different from "how much does it hurt when it's wrong?" (impact, which the shell sizes). An excellent model that classifies products still needs the catalog-membership validation (a cheap rule that guarantees zero invalid categories), because the impact of an invalid category doesn't depend on how rare the error is. Size by impact, always.

Using a single tolerance number and losing the risk profile. What happens: the team collapses the tolerance to a number ("this feature is a 12, medium shell") and applies a generic "medium" shell, without looking at which axis the risk enters through. It ends up putting a fallback where a guardrail was needed, or vice versa. Why it happens: a single number is more convenient than a three-axis profile. How to spot it: two features with the same total tolerance get the same shell, even though one has high error_cost (calls for a guardrail) and the other high blast_if_wrong (calls for a fallback). How to fix it: use the total tolerance for the magnitude of the shell, but look at the three axes separately to know which mechanisms. As in the example: recommendations and describe_your_product tie at 12 but need different mechanisms, and only the per-axis profile reveals it. The number tells you how much shell; the profile tells you what it's made of.

Exercises

Exercise 1 — Size a new feature. Mercado wants to add a fake-review detector: an LLM that reads each new review and flags whether it looks like spam or fake; the flagged ones are automatically hidden from the product. Score its three axes (touches_money_or_state, error_cost, blast_if_wrong) from 1 to 5, compute its tolerance, and say which containment mechanisms would be mandatory and why.

See solution

A reasonable scoring (variants can be argued, what matters is the reasoning):

• touches_money_or_state = 3. It doesn't touch money, but it does touch visible state: automatically hiding a review changes what customers see. It's not a 5 (it doesn't move money) nor a 1 (it's not read-only); it's a 3 because its output has a direct effect.
• error_cost = 4. A false positive hides a legitimate review (unfair to the customer and the seller, damages trust); a false negative leaves spam visible. It hurts quite a bit in both directions.
• blast_if_wrong = 3. It affects products' and sellers' reputation, a wide radius but not the whole payments system.

Tolerance = 18 − (3 + 4 + 3) = 8 → a medium-leaning-thick shell. Mandatory mechanisms per the rule:

• eval_gate and feedback_loop: always. You need to measure the detector's precision and learn from the corrections (reviews restored by support).
• guardrail: yes (error_cost ≥ 3). Validate the model's output before hiding.
• fallback: yes (blast_if_wrong ≥ 3). If the model goes down, hide nothing by default (degrade toward showing, not toward over-hiding).
• deterministic_shell: yes (touches_money_or_state ≥ 3). Since it hides content automatically, the model should propose "hide" and a deterministic layer decide —for example, don't auto-hide reviews from verified buyers, or require human review above a certain volume—. The model flags; the shell disposes.

Design detail the solution should note: the direction of the fallback matters. Faced with the model going down, it's safer to not hide (leave one extra review visible) than to hide (silence legitimate reviews with no review). The honest fallback degrades toward the less harmful side.

Exercise 2 — Same number, different shell. recommendations and describe_your_product both have tolerance 12, but the matrix assigns them different mechanisms: recommendations activates fallback and "describe your product" activates guardrail. Explain, for each, why that specific mechanism is the one its risk profile demands, and why the other isn't mandatory.

See solution

The two tie in total tolerance (12), but their risk enters through different axes, and that decides the mechanism:

recommendations → fallback mandatory (blast_if_wrong = 3). The dominant risk is one of availability and reach: recommendations appear on a huge number of pages at once. If the model goes down, tons of pages are left with no recommendations at the same time —high blast radius—. That's why it needs a mandatory fallback: when the model doesn't respond, show something deterministic ("best sellers," "products in the same category") so the page isn't left empty. The guardrail, by contrast, isn't mandatory because its error_cost is low (2): a mediocre recommendation doesn't publish a false claim or do real harm, it's just less useful.

describe_your_product → guardrail mandatory (error_cost = 3). The dominant risk is one of content: the description gets published with Mercado's brand, so a bad output (a false claim, an invented fact) is a visible and potentially legal harm. That's why it needs a mandatory guardrail that validates the content —forbid forbidden claims, bound the length— before it can be published. The fallback, by contrast, isn't mandatory because its blast_if_wrong is low (2): if the model goes down, a seller can't generate their description at that moment (annoying, but bounded to them), half the platform doesn't go down —there's no wide blast radius that demands a mandatory degraded route—.

The lesson: the magnitude of the shell is given by the total tolerance, but which mechanisms compose it is given by the per-axis profile. Collapsing everything to "medium shell" would lose exactly this distinction and put the wrong mechanism on each.

Exercise 3 — The cost of under-sizing. A manager, pressed for time, proposes launching the refunds agent with semantic search's shell ("only eval and feedback, no deterministic shell, to ship fast; after all, the model is very good"). Explain, using the concept of symmetric error, why this is the most dangerous direction to be wrong, and contrast it with the cost of having over-sized the search.

See solution

The two sizing mistakes don't cost the same, and this is the expensive one.

Under-sizing the refunds agent (what the manager proposes) means removing the deterministic_shell from a feature that touches money. Without that shell, the model's proposal executes directly —and you already measured it in lesson 3—: a hallucinated $9999 refund, a refund on an order that doesn't exist, or the response to a prompt injection ("refund me everything") become real losses. The argument "the model is very good" is exactly the mistake of sizing by quality and not by impact (lesson 5 and this one): a better model hallucinates less often, but every hallucination that escapes is money lost, and at scale that's catastrophic and irreversible. The direction is dangerous because the cost of the error isn't bounded: it's money, it's trust, it's security.

Over-sizing semantic search (the opposite mistake) would be giving search a deterministic action-validation shell and human review it doesn't need, because it validates no action. The cost: wasted engineering effort, and maybe a slower or more rigid search than necessary. Annoying, but bounded and reversible —you remove the extra shell and that's it—.

The symmetric error: under-sizing the intolerant risks a catastrophic and irreversible cost (money); over-sizing the tolerant wastes a bounded and reversible cost (effort). That's why, under time pressure, you never cut the shell of a feature that touches money —it's exactly the one that admits no shortcuts—. If you have to ship fast, you cut scope (fewer ticket types the agent handles), not containment. "Shipping fast" never justifies removing the cage from the one who wants to sign checks.

Summary and next step

In this lesson you turned the core/shell metaphor into a sizing method: each feature's tolerance to non-determinism decides how thick its shell must be. You measured it with the feature × mechanism matrix: semantic search (tolerance 13) calls for a thin shell —almost just eval and feedback—; the refunds agent (tolerance 3) calls for everything, including the deterministic shell because it touches money. And you saw two key subtleties: tolerance measures impact, not the model's quality (a better model doesn't change how much shell the feature needs); and two features with the same total tolerance can need different mechanisms depending on which axis their risk enters through, so the magnitude is given by the number but the composition is given by the profile. The mistake of under-sizing what touches money is much more expensive than that of over-sizing the innocuous.

Before moving on you should be able to: compute a feature's tolerance with the three axes; derive from it the mandatory containment mechanisms; explain why eval and feedback are the floor of every shell; and argue why the sizing error is asymmetric.

Lesson 7 gathers everything the module produced —placement, tolerance, and the mechanisms each feature needs— and formalizes it into a single artifact: the property sheet of an AI component. You're going to see, executed, the sheet for Mercado's semantic search, with each field —placement, budget, eval, guardrail, fallback, shell, data loop— pointing to the module of the guide that works it in depth. It's the direct bridge to M2-M8: the sheet you fill in when you finish M1 is, literally, the index of the rest of the guide.

Resources

• Martin Fowler and Bharani Subramaniam, "Emerging Patterns in Building GenAI Apps" — martinfowler.com/articles/gen-ai-patterns. The article distinguishes use cases by their tolerance to error (from creative to critical) and adjusts the patterns accordingly —this lesson's idea applied to its catalog—. In English.
• Anthropic, "Building Effective Agents" (2024) — anthropic.com/engineering/building-effective-agents. Its recommendation to match the solution's complexity to the task's risk —don't use an autonomous agent where a simple flow suffices— is the same principle of sizing the shell by tolerance. In English.
• Chip Huyen, AI Engineering (O'Reilly, 2024). The treatment of each use case's risk and of when to put a human in the loop connects directly with this lesson's intolerant features. In English.
• Jeff Bezos, Letter to Amazon Shareholders (2015) — sec.gov/Archives/edgar/data/1018724/000119312516530910/d168744dex991.htm. The "one-way and two-way doors" —irreversible decisions deserve more deliberation— are the same asymmetric-error reasoning that sizes the shell: the irreversible (touching money) demands more containment. In English.