Module 3: The Eval as a Fitness Function

2. Why you can't assert a probabilistic component

Next →

Overview

By the end of this lesson you'll understand, by executing it, why the test with which you verify any classic code can't verify an AI component, and what replaces it. The classic test is the equality assert: you give an input to a function, compare its output against the value you expected, and if they're identical, it passes. With deterministic code it always works, because a normal function gives the same output for the same input, without exception. With an LLM it breaks on the first run, because the same input produces different wordings each time —you saw it in module 1—. Here you're going to see it fall: an assert output == expected that approves a deterministic function and throws AssertionError against a component that responded the same thing in other words. And you're going to see the change of shape that saves it: stop comparing by exact equality and switch to verifying a property —that the output contain the correct information—, aggregating the result into a score instead of a binary verdict per case.

This matters because the underlying mistake isn't a syntax detail: it's a wrong mental model that leads to two equally bad outcomes. The first is fragile tests: if you insist on ==, your suite breaks every time the model rephrases a correct answer, and you end up either deleting the tests as noisy, or forcing the model to repeat fixed phrases —killing the flexibility that justifies using an LLM—. The second, more common, is giving up: the team concludes "AI can't be tested" and leaves quality to the eye, with no gate. Both are born from the same root: treating a probabilistic component as if it were deterministic. Changing the shape of the test —from exact equality to property, and from binary per case to aggregate score— is what unlocks the whole module, because the score is the gate's raw material.

Connection with the module: this lesson installs the problem the next five solve. Here you see why the classic assert doesn't work and, for the first time, the word score appears. Lesson 3 formalizes that score with the anatomy of the eval-set —many cases, each with its criterion, run together—. Lesson 4 turns it into a gate with a threshold. Lessons 5 and 6 put it to work against regressions and in CI. Lesson 7 opens the types of success criterion —the contains we use here is only the simplest—. In one sentence: this lesson tells you why you can't use ==; the rest of the module gives you what you can.

The analogy: the multiple-choice exam versus the essay

Think of two ways to grade an exam. The first is a multiple-choice exam corrected by a scanner: each answer is a filled-in bubble, and the scanner compares the student's bubble against the key —A, B, C, or D—. It's an exact-equality comparison: either you marked the same letter as the key, or you didn't. Fast, objective, infallible... as long as the answers are identical, comparable bubbles. The second is an essay: you ask the student to explain in their own words why shipping takes 3 to 5 days. Each student writes something different —different wording, different order, different length— and none matches a "model answer" letter for letter. If you tried to grade the essay with the multiple-choice scanner, everyone would fail, because none marked exactly the key. The scanner isn't broken: it's using the wrong tool for the wrong question.

An LLM is an essay machine, not a bubble machine. Each response is free text that says the right thing in a different way each time. Grading it with assert output == expected is grading an essay with the multiple-choice scanner: it fails correct answers for not being identical. What a sensible teacher does when grading essays is use a rubric: they don't require the student to write the model answer word for word, but verify that it contains the key ideas —"did they mention the 3 to 5 days range?"—. That rubric is this lesson's property criterion, and the grade that comes out of applying it to many essays is the score. The whole module is, at bottom, learning to grade essays with a rubric instead of demanding bubbles.

Worked example: the assert that breaks and the criterion that saves it

We're going to put the two ways of testing side by side. First we ask the simulated support agent the same question three times —"how long does shipping take?"— and see that it responds correctly in three different ways. Then we try to verify it with an exact-equality assert (the multiple-choice scanner) and see it break. And finally we apply a property criterion (the rubric) and get a score.

Remember: the LLM is simulated with a deterministic stub —zero network, zero API, zero keys—. The non-determinism is simulated by making the same question produce a different wording on each call, in a fixed, reproducible cycle.

# Lesson 02 — why the exact assert can't test a probabilistic component
# Everything SIMULATED. Zero network, zero API, zero keys. Deterministic output.
import itertools

# --- The AI component, SIMULATED: same question, different wording each time ---
VARIANTS = [
    "Standard shipping takes 3 to 5 days.",
    "Your order usually arrives in 3 to 5 days.",
    "Your shipment should arrive within 3 to 5 days.",
]
_calls = itertools.count()

def support_agent_stub(question):
    # LLM STUB: the SAME question produces a different wording on each
    # call (fixed cycle, reproducible). That's how a real LLM is: non-deterministic.
    return VARIANTS[next(_calls) % len(VARIANTS)]

# Three calls with the SAME input.
question = "how long does shipping take"
outs = [support_agent_stub(question) for _ in range(3)]
print("Same input, three calls to the component:")
for i, o in enumerate(outs, 1):
    print(f"  call {i}: {o!r}")

# --- Attempt 1: the exact-equality assert (what you'd do with a normal function) ---
EXPECTED = "Standard shipping takes 3 to 5 days."
print("\n-- Attempt 1: exact-equality assert (what you'd do with a normal function) --")
try:
    for o in outs:
        assert o == EXPECTED
    print("all equal -> PASS")
except AssertionError:
    print("AssertionError: an output != the expected -> the test BREAKS")

# --- Attempt 2: the PROPERTY criterion (the rubric: contains the key info) ---
print("\n-- Attempt 2: property criterion (contains the key info) --")
crit = "3 to 5 days"
passed = sum(crit in o for o in outs)
print(f"cases meeting the criterion '{crit}': {passed}/{len(outs)}")
print(f"score = {passed/len(outs):.2f}")

What to expect. When you run it:

Same input, three calls to the component:
  call 1: 'Standard shipping takes 3 to 5 days.'
  call 2: 'Your order usually arrives in 3 to 5 days.'
  call 3: 'Your shipment should arrive within 3 to 5 days.'

-- Attempt 1: exact-equality assert (what you'd do with a normal function) --
AssertionError: an output != the expected -> the test BREAKS

-- Attempt 2: property criterion (contains the key info) --
cases meeting the criterion '3 to 5 days': 3/3
score = 1.00

Read the three calls first. The component responded the same all three times —shipping takes 3 to 5 days— but wrote it differently each time: "Standard shipping takes...", "Your order usually arrives...", "Your shipment should arrive...". All three are correct. A customer would be just as well served by any of the three. That's the nature of a probabilistic component: the information is stable, the form isn't.

Now Attempt 1, the assert output == EXPECTED. It pins the expected output to the first wording and compares the three against it. The first matches, but the second —"Your order usually arrives..."— isn't identical, and the assert throws AssertionError. The test breaks. And notice the perverse part: it didn't break because the component failed; it broke because the component responded well in other words. A test that fails correct answers isn't protecting you: it's lying to you. If you left this test in your suite, it would go red on every deploy without anything being wrong, and in two weeks someone would delete it as noisy —and with it, the component's only "test"—.

Attempt 2 is the rubric. Instead of demanding equality, it verifies a property: does the response contain the key phrase "3 to 5 days"? All three contain it, so all three pass: 3 out of 3, score = 1.00. Here are the module's two new ideas, together for the first time. The first: the criterion stopped being == (equality) and became a property (in, contains) —the rubric that grades the essay without demanding it be identical to the model answer—. The second, subtler but more important: the result stopped being a boolean per case ("did this one pass?") and became an aggregate score ("what fraction of the total passed?"). With three cases and a flexible criterion, the score was 1.00. When you have ten cases and some fail —lesson 3—, the score will be an intermediate number, and that number is what lesson 4's gate will compare against a threshold. The binary assert is no use for that; the score is.

Going deeper: from equality to property, and from case to aggregate

Why == is the wrong tool, in terms of contract. In module 1 you learned that an LLM's contract is probabilistic: you don't assert its exact output, you assert properties and invariants of its output. The assert output == expected is the code embodiment of the deterministic contract: it demands the exact output. Using it against an LLM is applying the wrong contract. It's not that the assert is wrong in general —it's the right tool for normalize_sku or for adding two numbers—; it's that it's wrong here, because the component doesn't promise equality, it promises correctness. The test has to verify what the component promises, not what you'd like it to promise.

Why the result has to be a score, not a boolean. An assert collapses all the information into one bit: passed or didn't pass. For a probabilistic component that loses too much. Imagine you run the agent against 100 questions: a component that answers 95 well and one that answers 60 well are very different, but an assert all(...) crushes both into "failed" as soon as a single case doesn't pass. The score preserves the difference —0.95 against 0.60— and that difference is exactly what you need to decide. The quality of a probabilistic component isn't binary (perfect or broken): it's a degree, and the score measures the degree. That's why the whole module operates on scores, not on asserts.

The score doesn't replace the per-case criterion: it aggregates it. Don't lose sight that the score is built from per-case checks. Each case does have a binary verdict —it met the criterion or not—; the score is the fraction of cases that met it. It's the difference between "did this student get this question right?" (binary, per case) and "what grade did the student get on the whole exam?" (aggregate, the score). The two levels coexist: the per-case detail tells you what failed (useful for debugging), and the score gives you the number you decide against (useful for the gate). Lesson 3 makes both levels explicit.

Where the boundary with AI Engineering is. You'll have noticed I chose the criterion "contains '3 to 5 days'" without justifying why that phrase and not another, nor why contains and not a finer criterion. That's deliberate: choosing the right criterion —which key phrase matters, whether contains suffices or you need something more semantic, how to keep a weak criterion from approving bad answers— is eval design, and it's AI Engineering. In this module the criterion comes given, and our job is to use it to produce a score and govern the deploy with it. This lesson only establishes the change of shape —from equality to property, from boolean to score—; which exact property to verify in each task is the other guide.

Common mistakes

Insisting on the exact assert and ending up with fragile tests (a mental-model mistake). What happens: the team tests the component with assert output == "model answer" and the suite goes red on every deploy because the model rephrases correct answers. To "fix it," they either delete the tests (and are left with no gate) or force the model with a rigid prompt to repeat the exact phrase (and lose the naturalness). Why it happens: == is every programmer's reflex and works with the other 99% of code. How to spot it: if your test for an AI component breaks when the model says the same thing in other words, you're using exact equality where a property goes. How to fix it: change == for a property check (contains, a schema, a judge —lesson 7—) and aggregate into a score.

Giving up: "AI can't be tested" (an omission mistake). What happens: after seeing the assert break, the team concludes that a probabilistic component is unverifiable by nature and leaves the quality to the judgment of whoever looks at the responses before a release. Why it happens: it's true that the classic test doesn't apply, and without knowing the replacement (the score) it seems there's no alternative. How to spot it: if the phrase "AI isn't tested, it's reviewed by eye" circulates in your team, you fell into this mistake. How to fix it: AI can be tested, just with a test of a different shape —an eval-set that produces a score (lesson 3)—; the exact assert doesn't work, but the score does, and it's as automatable as a test.

Collapsing the result into a boolean and losing the degree (an over-simplification mistake). What happens: the team makes the switch to a property criterion but wraps everything in an assert all(cases_pass), which collapses the result back into passed/didn't-pass. With that, a component that answers 95% well and one that answers 60% well look the same —"failed"— because both had at least one case fail. Why it happens: all() feels natural after a for, and returns the boolean you're used to. How to spot it: if your test for an AI component answers only "yes or no" and not a number between 0 and 1, you lost the degree. How to fix it: measure the fraction of cases that pass (the score), not whether all pass; the degree is the information the gate needs.

Exercises

Exercise 1 — Predict the verdict. An AI component responds these three ways to "can I pay in installments?": (1) "Yes, you can pay in installments with no interest by card.", (2) "Sure, we accept installment payments with a credit card.", (3) "Yes, we offer installments.". For each, give the verdict under two tests: (a) assert output == "Yes, you can pay in installments with no interest by card." and (b) the property criterion "installments" in output.lower(). Then give the score under each test.

See solution

Under (a), exact equality against the first wording:

  • (1) matches → PASS
  • (2) "Sure, we accept..." ≠ the expected → FAIL
  • (3) "Yes, we offer installments." ≠ the expected → FAIL
  • Exact score = 1/3 = 0.33. It failed two correct answers just for being worded differently.

Under (b), property criterion "installments" in output.lower():

  • (1) contains "installments" → PASS
  • (2) contains "installments" → PASS
  • (3) contains "installments" → PASS
  • Property score = 3/3 = 1.00. All three communicate the right thing, all three pass.

The moral: the same real quality (three correct answers) gives a score of 0.33 with exact equality and 1.00 with the property criterion. The 0.33 doesn't measure the component's quality, it measures how much its wording resembles an arbitrary phrase you chose. The property criterion measures what matters: whether the response contains the correct information.

Exercise 2 — The too-loose criterion. A colleague, after learning that == is too strict, goes to the other extreme and proposes the criterion len(output) > 0 (the response isn't empty) for the support agent. Explain why this criterion is as useless as ==, though for the opposite reason, and what it guarantees (nothing) about quality.

See solution

== errs on the strict side: it fails correct answers. len(output) > 0 errs on the loose side: it approves incorrect answers. Any non-empty text passes this criterion —including "Sorry, I don't know", "banana", or a complete hallucination—. A component that responded pure garbage, as long as it didn't return an empty string, would get a score of 1.00 with this criterion. That makes it useless as a gate: a gate that can never fail protects against nothing (it's the "useless fitness function" you'll see in lesson 4).

The point: a success criterion has to be neither so strict that it fails the correct, nor so loose that it approves the incorrect. It must capture what makes a response good —for the support agent, that it contain the correct information that answers the question—. Finding that just-right criterion is a craft (and it's AI Engineering); but recognizing the two extremes that don't work —exact equality and "non-empty"— is already part of knowing how to use the eval as a gate.

Exercise 3 — Normal function or probabilistic component? For each, say whether you'd test it with assert output == expected (exact equality) or with a property criterion over an eval-set, and why. (a) A function apply_discount(price, pct) that computes a discounted price. (b) The support agent that answers customer questions. (c) A function normalize_email(s) that lowercases an email and strips spaces. (d) The semantic search that reorders products by relevance to a query.

See solution
  • (a) apply_discount → exact equality. It's a deterministic, numeric function: 100 with 10% always gives 90.0. assert apply_discount(100, 10) == 90.0 is the right tool. The output is unique and exact.
  • (b) The support agent → property criterion over an eval-set. It's a probabilistic component: it responds correctly with different wordings. == would break; the criterion is that the response contain the correct information, aggregated into a score.
  • (c) normalize_email → exact equality. Also deterministic: " Ana@X.COM " always gives "ana@x.com". assert output == "ana@x.com" is correct.
  • (d) The semantic search → property criterion over an eval-set. It's probabilistic: for a query, the exact order of the products can vary, but the property that matters is that the relevant products come out on top. It's evaluated with a criterion over an eval-set (queries with the products that should appear), not with equality of the exact order.

The rule that separates: if the component is deterministic —same input, same exact output— (a, c), use exact equality; if it's probabilistic —same input, output that varies in form but must meet a property— (b, d), use a property criterion over an eval-set and measure a score.

Summary and next step

In this lesson you saw, by executing it, why the classic test can't verify an AI component. The assert output == expected —the multiple-choice scanner— broke against a component that responded the same thing in other words, not because the component failed but because it responded well in a different way. And you saw the change of shape that saves it, with its two parts: stop comparing by exact equality and verify a property (the rubric that grades the essay without demanding it be identical), and stop giving a boolean per case to give an aggregate score (the fraction of cases that meet the criterion). The property criterion gave 3 out of 3 —score 1.00— where == failed correct answers. Those two ideas —property instead of equality, score instead of boolean— are the raw material of everything that follows.

Before moving on you should be able to: explain why the equality assert is the wrong tool for a probabilistic component (it verifies the deterministic contract, which the LLM doesn't promise); distinguish a too-strict criterion (==) from a too-loose one (len > 0); and separate "did this case pass?" (binary, per case) from "what fraction passed?" (score, aggregate).

What follows is formalizing the score. Here you saw it with three runs of a single question; in lesson 3 you're going to build the complete tool —the eval-set—: many different cases, each with its success criterion, run together against the component to produce a single score. You're going to see Mercado's support agent eval-set executed case by case, with its detail and its final score (9 out of 10, 0.90). It's the step from "I know I need a score instead of an assert" to "I know exactly how that score is built, case by case."

Resources

← Previous1. Module introduction: the eval as a fitness functionNext →3. The eval-set and the score