5. The contract catches module 2's divergence

Description

This is the payoff of the whole module. In module 2 you saw a concrete and painful divergence: the FakeBookingRepository returned None when asking for a booking that doesn't exist, while the real SqliteBookingRepository raised. The bug passed the unit test —which only talked to the fake— and blew up in production —where the real one is on duty—. You turned the problem over without a prescription. Lessons 1 to 4 built the prescription: what a contract is, who defines it, how it's run against several implementations. Now we apply it to the exact case that motivated it and see, with real output, how the contract battery catches that divergence in red before it leaves your machine.

The experiment is clean: we take the same four-clause battery and change one provider. Instead of the correct FakeBookingRepository, we put in the BuggyFakeBookingRepository —the one from module 2, the one that returns None on get of a missing id— alongside the real SqliteBookingRepository. We run. And we observe something precise: seven of the eight cases pass, and one fails —test_get_of_a_missing_id_raises[buggy-fake]— with a message that admits no interpretation: DID NOT RAISE KeyError. The id in brackets points at the culprit with its finger: it's not the real one ([sqlite] passes that clause), it's the fake. The contract turned an invisible divergence —which in module 2 was only discovered in production— into a red with a name, a line, and a cause. That's the whole discipline working.

Connection to the module: this lesson closes the arc opened by lesson 1 ("from the problem to the cure") by demonstrating the cure on the original problem. Lessons 2 and 3 gave the concept; lesson 4 the mechanism; here it's collected. After this, the module opens to the nuances: lesson 6 distinguishes two ways of writing a clause (state vs interaction) and lesson 7 shows how the industry automates this between services (Pact). But the module's heart beats here: the fake that lied, caught by the contract in red.

Analogy: the reference weight against the stall's scale

At a market, each stall has its scale. A customer suspects a stall's scale is rigged —it reads less than what's there—, but can't prove it by looking at it: from the outside it looks identical to the others. So the weights-and-measures office brings a certified reference weight: an exact kilo, the same for every stall. They put it on the suspect scale and on an honest stall's, and compare. The honest one reads "1.000 kg"; the rigged one reads "0.920 kg". The same weight, two readings: the reference didn't change, the instrument did. Now the suspicion is a measurable fact, and the rigged stall is exposed —not by its appearance, but by failing the standard test—.

The contract battery is that reference weight. The same test —"get of a missing id must raise"— is applied to both providers. The real one passes it (it reads correctly); the buggy fake fails it (it reads None where it should have raised). The test didn't change between one and the other; the instrument changed, and that's why the divergence is exposed. In module 2, without a reference weight, the rigged scale went unnoticed until a customer complained —the equivalent of the bug in production—. With the contract, the rigged one is detected in the inspection, before opening the stall. The underlying lesson is the same: to know whether an instrument lies, don't look at it; measure it against a shared standard.

Reminder: module 2's divergence

Let's put the two providers face to face, in the single clause where they differ. The correct fake:

# reservo/doubles.py — the fake that COMPLIES
class FakeBookingRepository:
    def __init__(self):
        self._store = {}

    def get(self, booking_id):
        return self._store[booking_id]   # KeyError if it doesn't exist  <-- RAISES

The buggy fake from module 2:

# reservo/doubles.py — the fake that LIED
class BuggyFakeBookingRepository:
    def __init__(self):
        self._store = {}

    def get(self, booking_id):
        return self._store.get(booking_id)   # returns None if it doesn't exist  <-- DOES NOT RAISE

The difference is a single word: self._store[booking_id] (indexing, which raises KeyError if the key is missing) versus self._store.get(booking_id) (the dict's .get method, which returns None if it's missing). A tiny change, of one letter and a dot, with an enormous consequence: it breaks clause 2 of the contract. And the real SqliteBookingRepository, recall, raises KeyError explicitly when the row doesn't exist:

# reservo/sqlite_repo.py — the real one (get excerpt)
    def get(self, booking_id):
        row = self._conn.execute(..., (booking_id,)).fetchone()
        if row is None:
            raise KeyError(booking_id)       # <-- RAISES, as the contract demands
        return Booking(...)

So the buggy fake and the real one differ exactly on clause 2. In module 2, that difference hid because each was tested (or not) on its own. Now we'll put them under the same reference weight.

Worked example: the battery catches the fake that lies

We run the same four-clause battery, but this time with the BuggyFakeBookingRepository instead of the correct one. Notice that the only thing that changes from lesson 4 is the line in the fixture that builds the "fake" provider:

# tests/test_contract_catches_divergence.py
import sqlite3
from datetime import datetime

import pytest

from reservo.doubles import BuggyFakeBookingRepository
from reservo.models import Booking
from reservo.sqlite_repo import SqliteBookingRepository

START = datetime(2026, 3, 10, 9)
END = datetime(2026, 3, 10, 12)


def a_booking(id="bk-1", room_id="focus", status="confirmed", price_cents=6000):
    return Booking(id=id, room_id=room_id, member_id="m-ana",
                   start=START, end=END, status=status, price_cents=price_cents)


# The SAME battery, now against the DIVERGENT fake from module 2 and the real one.
@pytest.fixture(params=["buggy-fake", "sqlite"])
def repo(request):
    if request.param == "buggy-fake":
        return BuggyFakeBookingRepository()
    return SqliteBookingRepository(sqlite3.connect(":memory:"))


def test_save_then_get_returns_the_same_booking(repo):
    booking = a_booking()
    repo.save(booking)
    assert repo.get("bk-1") == booking


def test_get_of_a_missing_id_raises(repo):
    with pytest.raises(KeyError):
        repo.get("does-not-exist")


def test_saving_the_same_id_twice_updates_not_duplicates(repo):
    repo.save(a_booking(status="confirmed"))
    repo.save(a_booking(status="cancelled"))
    assert repo.get("bk-1").status == "cancelled"
    assert len(repo.find_by_room("focus")) == 1


def test_find_by_room_returns_only_that_rooms_bookings(repo):
    repo.save(a_booking(id="bk-1", room_id="focus"))
    repo.save(a_booking(id="bk-2", room_id="studio"))
    found = repo.find_by_room("focus")
    assert [b.id for b in found] == ["bk-1"]

The four clauses are identical to lesson 4's —the contract didn't change, it's a fixed text—. The only difference is that the fixture delivers the buggy fake. Let's run and read with a magnifying glass.

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

python3 -m pytest tests/test_contract_catches_divergence.py -v

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 8 items

tests/test_contract_catches_divergence.py::test_save_then_get_returns_the_same_booking[buggy-fake] PASSED [ 12%]
tests/test_contract_catches_divergence.py::test_save_then_get_returns_the_same_booking[sqlite] PASSED [ 25%]
tests/test_contract_catches_divergence.py::test_get_of_a_missing_id_raises[buggy-fake] FAILED [ 37%]
tests/test_contract_catches_divergence.py::test_get_of_a_missing_id_raises[sqlite] PASSED [ 50%]
tests/test_contract_catches_divergence.py::test_saving_the_same_id_twice_updates_not_duplicates[buggy-fake] PASSED [ 62%]
tests/test_contract_catches_divergence.py::test_saving_the_same_id_twice_updates_not_duplicates[sqlite] PASSED [ 75%]
tests/test_contract_catches_divergence.py::test_find_by_room_returns_only_that_rooms_bookings[buggy-fake] PASSED [ 87%]
tests/test_contract_catches_divergence.py::test_find_by_room_returns_only_that_rooms_bookings[sqlite] PASSED [100%]

=================================== FAILURES ===================================
_________________ test_get_of_a_missing_id_raises[buggy-fake] __________________

repo = <reservo.doubles.BuggyFakeBookingRepository object at 0x...>

    def test_get_of_a_missing_id_raises(repo):
>       with pytest.raises(KeyError):
E       Failed: DID NOT RAISE KeyError

tests/test_contract_catches_divergence.py:34: Failed
=========================== short test summary info ============================
FAILED tests/test_contract_catches_divergence.py::test_get_of_a_missing_id_raises[buggy-fake] - Failed: DID NOT RAISE KeyError
========================= 1 failed, 7 passed in 0.04s ==========================

Read the result calmly, because every detail matters:

• Seven pass, one fails. The buggy fake fulfills three of the four clauses —it saves and reads correctly, updates without duplicating, filters by room—; it only violates clause 2. The contract doesn't condemn it wholesale: it points exactly where it diverges. That precision is gold for debugging.
• The only red is [buggy-fake], not [sqlite]. The same test, test_get_of_a_missing_id_raises, passes for [sqlite] (50% percentile) and fails for [buggy-fake] (37% percentile). The id in brackets is the pointing finger: it tells you that the one that fails to comply is the fake, not the real one. If [sqlite] had failed instead, the reading would be the opposite —the real one is broken—. The id turns "something failed" into "this provider failed this clause".
• The message is unequivocal: DID NOT RAISE KeyError. pytest.raises(KeyError) expected repo.get("does-not-exist") to raise; the buggy fake returned None silently, so the block ended without an exception and pytest reports it as a failure with those exact words. There's no need to guess the cause: the fake didn't raise when the contract demanded it.

This is what we didn't have in module 2. There, this same divergence only manifested when BookingService.cancel received a None and blew up with a cryptic AttributeError, in production, far from the cause. Here it manifests as a clear red, on your machine, at line 34, with the guilty provider labeled. The contract moved the discovery of the bug from "an hour after the deploy, in a production log" to "the second you ran the battery". That shift —from late and expensive to early and cheap— is the whole module's reason for being.

How the red is closed (and how NOT to)

You have the red. Now what? The correct answer depends on which behavior is the correct one, and that decision is dictated by the consumer (lesson 3). BookingService.cancel needs get of a missing id to raise, so as not to carry on with a None and blow up later. So the correct behavior is "raise", the real one already fulfills it, and the one that's wrong is the fake. The fix is aligning the fake to the contract:

# The correct fix: the fake fulfills clause 2.
    def get(self, booking_id):
        return self._store[booking_id]   # indexing: raises KeyError if missing

You change .get(booking_id) for [booking_id], and the battery goes back to eight greens. The divergence was closed by making the incorrect provider fulfill the correct contract.

Now, the wrong fix, worth naming so as not to fall into it: making the real one return None "so it matches the fake" and the red disappears. That removes the symptom and bakes in the bug: you'd leave the SqliteBookingRepository returning None in a case where the consumer needs an exception, and cancel would blow up in production again —now without any test to warn, because you "aligned" the battery to the broken behavior—. The rule is firm: the contract describes the correct behavior (the one the consumer needs), and you fix the provider that doesn't fulfill it; you never degrade the contract to silence a red. A red isn't an enemy to silence; it's the reference weight telling you which instrument is rigged.

Why this couldn't be caught with unit tests alone

It's worth closing the circle with module 1. Why didn't a thousand unit tests of the fake catch this? Because they all shared the fake's premise. A unit test of cancel with the BuggyFakeBookingRepository that tested the happy case (canceling a booking that exists) would pass green —the booking exists, get returns it, everything flows—. The missing-id case, if someone tested it against the fake, would "confirm" that it returns None —because that's what the fake does—, without suspecting that the real one does something else. The unit test can't catch a bug that lives in the difference between the fake and the real one, because it only looks at one of the two.

The contract breaks that enclosure with an idea that's already yours: it doesn't test a provider, it tests the clause against all of them. By running test_get_of_a_missing_id_raises against the buggy fake and the real one at once, it forces both to match the expected behavior —and the one that doesn't match lights up in red—. It's the only way to see a divergence: look at both sides under the same test. The unit test looks at one side; the contract looks at all with the same yardstick. That's why the contract catches what the unit test, by design, can't.

Common mistakes

Silencing the red by degrading the contract. What happens: [buggy-fake] FAILED appears and someone "fixes" the test —deletes it, marks it xfail, or changes the clause to accept None—. Why it happens: a red is annoying and the quick exit is to silence it. How to detect it: if your fix makes the battery tolerate the behavior the consumer doesn't want, you degraded the contract. How to fix it: the red points to a provider that doesn't fulfill a real need; you fix the provider, not the test. Degrading the contract is turning off the alarm and leaving the fire.

Fixing the wrong provider. What happens: [buggy-fake] fails and someone changes the SqliteBookingRepository so it also returns None, "so both match". Why it happens: the goal becomes making the column all green without thinking which behavior is correct. How to detect it: if your fix makes the real provider stop fulfilling what the consumer needs, you fixed the one that was right. How to fix it: first decide what the correct behavior is according to the consumer (here, "raise"); align the provider that deviates from it (here, the fake). The real one was already right; touching it introduces the bug the contract just caught.

Ignoring the [...] id and debugging blind. What happens: "1 failed" is seen and code is reviewed at random without looking at which case failed. Why it happens: the rush makes you skip the FAILED line. How to detect it: if you don't know against which provider the clause failed, you're missing the datum the output already gave you. How to fix it: read the id. [buggy-fake] takes you straight to the fake; [sqlite] would take you to the real one. The id is the map to the culprit; starting without reading it is debugging blindfolded.

Exercises

Exercise 1 — Read the column. In the example output, test_get_of_a_missing_id_raises appears twice: [buggy-fake] FAILED and [sqlite] PASSED. A colleague concludes "SQLite's get is broken". Are they right? What does that pair of results really say?

Exercise 2 — Predict the red. Imagine a third provider, SloppyFakeBookingRepository, whose save saves correctly but whose find_by_room returns all the bookings, ignoring the room_id. You put it in the battery alongside the correct fake and SQLite (params=["good-fake", "sloppy-fake", "sqlite"]). Without running anything, say how many cases there will be, which will fail, and with what id.

Exercise 3 — The bug the contract didn't cover. The BuggyFakeBookingRepository also had another quirk the current contract does not catch: its find_by_room returns the bookings but mutating a shared copy, so that if the consumer modifies a returned booking, the fake's internal store gets corrupted. The four-clause battery passes anyway. Explain why it's not caught and what would need to be done.

# Clause 5: mutating a returned booking doesn't alter what's stored.
def test_returned_bookings_are_independent_from_storage(repo):
    repo.save(a_booking(status="confirmed"))
    got = repo.get("bk-1")
    got.status = "tampered"              # the consumer mutates its copy
    assert repo.get("bk-1").status == "confirmed"   # the store didn't change

Summary and next step

In this lesson you collected on the module's promise: you saw the contract catch module 2's divergence in red. You put the BuggyFakeBookingRepository —the one that returns None instead of raising— into the same battery as the real one, ran it, and got a surgical diagnosis: seven greens, one red, test_get_of_a_missing_id_raises[buggy-fake] — DID NOT RAISE KeyError. The id in brackets pointed at the culprit (the fake, not the real one); the message named the cause (it didn't raise when it should have); and it all happened on your machine, at line 34, not in a production log an hour late. With the reference weight you fixed the image: to know whether an instrument lies, measure it against a shared standard. And you learned to close the red well —align the non-complying provider with the behavior the consumer needs— and not to close it badly —degrade the contract or fix the healthy provider—.

Before moving on you should be able to: read the [fake]/[sqlite] column to know who doesn't comply and who is the correct reference; explain why no volume of unit tests caught this divergence and why the contract did; and decide the correct fix for a red according to the behavior the consumer needs, without degrading the contract or touching the healthy provider.

Up to now, all our clauses asserted about the result —what get returns, what find_by_room contains—. But not all contracts are written that way. In lesson 6 you'll see a second form: interaction contracts, which assert about the call —who was called, with what arguments, how many times— instead of about the result. The repository calls for state contracts; the PaymentGateway usually calls for interaction ones. Knowing which to use for each collaborator is the next refinement.

Resources

• pytest documentation — pytest.raises and assertions about exceptions — the exact mechanic of the clause that catches the divergence; in particular, what the DID NOT RAISE message means when the block doesn't raise the expected exception.
• pytest documentation — How to understand failure reports — to read the FAILURES block, the [buggy-fake] id, and the short test summary info as you did in the worked example.
• sqlite3 — DB-API for SQLite (Python documentation) — the reference for the real provider whose get raises KeyError on the missing id, and which serves as the correct reference against which the divergent fake is measured.
• test-doubles-and-test-data-guide — the sister guide where the FakeBookingRepository was born; useful to remember that a fake is code you write, and that's why it can deviate from the contract with a simple .get() instead of [].