Module 4: Verifying The Contract From Both Sides

4. The same battery against the fake and the real one

Next →

Description

You have the two chairs: the consumer test (lesson 2) against the fake, the provider test (lesson 3) against SQLite. You might leave here thinking they're two separate things —one battery for the fake, another for the real one—. They're not. They're a single contract battery, run against the two providers. And that "two" isn't a detail of convenience: it's the whole guarantee. This lesson's thesis is as short as it is powerful: if the fake and the real one pass exactly the same suite, the fake can't be lying about what the contract covers.

Go back to module 1's problem, the guide's reason for being: a green unit test with a fake can hide a broken integration, because the fake diverges from the real one right at the seam, and no one notices. The question left open was: "how do I guarantee that my fake behaves like the real one there?". This lesson gives the mechanical answer. You don't eyeball the fake, or ban fakes, or test everything against the real thing. You write the behavior once —the contract— and run it against both. If they diverge on any clause, the battery screams it: one side green, the other red, on the exact clause. As long as both are green in the same suite, you have proof —not a hope— that the fake doesn't lie.

Connection to the module: lessons 2 and 3 gave you the sides separately so you understood each point of view; this one reunites them and shows that the separation was conceptual, not physical. One battery, two providers. It's also the module's pivot: understanding why running it against both is the guarantee prepares you for the two lessons that follow, where that guarantee comes to life —lesson 5 catches a provider that breaks the contract, lesson 6 catches a consumer that assumes too much—. Here you see the mechanism at rest, green; in the next ones you see it firing, red.

Analogy: the same taste test to two cooks

Imagine a restaurant chain with a signature recipe —say, its house sauce— that must taste identical at every branch. The chain doesn't trust each cook to "cook similarly"; it writes a standardized taste test: a sheet with objective measurements —this much acidity, this much salt, this color, this texture when coating the spoon—. That sheet is the sauce's contract.

Now, the important part: the taste test is only useful if it's applied to both sauces at once, with the same sheet. If the evaluator tested cook A's sauce with one sheet and cook B's with a different one, it would prove nothing about whether they taste the same. The guarantee that the two sauces are interchangeable comes from both passing the same test. If one has more acidity than the sheet allows, the test fails it —and you know instantly which branch deviated and by how much—. As long as both pass the same sheet, a diner can't tell which branch they're at: that's the promise.

The FakeBookingRepository and the SqliteBookingRepository are the two cooks; the contract battery is the taste-test sheet; and running it against both with the same battery is what guarantees that the two "sauces" —save, read, raise, filter— taste the same. A fake that returns a str where the real one returns a datetime is a sauce with the acidity changed: the same sheet, applied to both, fails it immediately. Interchangeability isn't assumed; it's proven, and it's proven by running one sheet against two cooks.

Worked example: one battery, two providers, eight verdicts

Here's the complete mechanism. The piece that makes "one battery, two providers" is the parametrized fixture: a fixture called repo that declares params=["fake", "sqlite"]. Pytest, seeing that fixture, runs each test that asks for it once per params value. Four tests × two values = eight executions. The trick is that the four test_... don't know —or care— which provider they got: they ask for repo and work against it. The same clause, two implementations.

# tests/test_repository_contract.py — one battery, two providers
import sqlite3
from datetime import datetime

import pytest

from reservo.doubles import FakeBookingRepository
from reservo.models import Booking
from reservo.sqlite_repo import SqliteBookingRepository

START = datetime(2026, 3, 10, 9)
END = datetime(2026, 3, 10, 12)


def a_booking(id="bk-1", room_id="focus", status="confirmed", price_cents=6000):
    return Booking(id=id, room_id=room_id, member_id="m-ana",
                   start=START, end=END, status=status, price_cents=price_cents)


# THIS is the key piece: one fixture, two providers.
# pytest runs each test that asks for `repo` once per params value.
@pytest.fixture(params=["fake", "sqlite"])
def repo(request):
    if request.param == "fake":
        return FakeBookingRepository()
    return SqliteBookingRepository(sqlite3.connect(":memory:"))


def test_save_then_get_returns_the_same_booking(repo):
    repo.save(a_booking())
    got = repo.get("bk-1")
    assert got.id == "bk-1"
    assert got.room_id == "focus"
    assert got.start == START            # the field where fake and real COULD diverge
    assert got.price_cents == 6000
    assert got.status == "confirmed"


def test_get_of_a_missing_id_raises(repo):
    with pytest.raises(KeyError):
        repo.get("does-not-exist")


def test_saving_the_same_id_twice_updates_not_duplicates(repo):
    repo.save(a_booking(status="confirmed"))
    repo.save(a_booking(status="cancelled"))
    got = repo.get("bk-1")
    assert got.status == "cancelled"
    assert len(repo.find_by_room("focus")) == 1


def test_find_by_room_returns_only_that_rooms_bookings(repo):
    repo.save(a_booking(id="bk-1", room_id="focus"))
    repo.save(a_booking(id="bk-2", room_id="studio"))
    ids = {b.id for b in repo.find_by_room("focus")}
    assert ids == {"bk-1"}

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

python3 -m pytest tests/test_repository_contract.py -v
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 8 items

tests/test_repository_contract.py::test_save_then_get_returns_the_same_booking[fake] PASSED [ 12%]
tests/test_repository_contract.py::test_save_then_get_returns_the_same_booking[sqlite] PASSED [ 25%]
tests/test_repository_contract.py::test_get_of_a_missing_id_raises[fake] PASSED [ 37%]
tests/test_repository_contract.py::test_get_of_a_missing_id_raises[sqlite] PASSED [ 50%]
tests/test_repository_contract.py::test_saving_the_same_id_twice_updates_not_duplicates[fake] PASSED [ 62%]
tests/test_repository_contract.py::test_saving_the_same_id_twice_updates_not_duplicates[sqlite] PASSED [ 75%]
tests/test_repository_contract.py::test_find_by_room_returns_only_that_rooms_bookings[fake] PASSED [ 87%]
tests/test_repository_contract.py::test_find_by_room_returns_only_that_rooms_bookings[sqlite] PASSED [100%]

============================== 8 passed in 0.01s ===============================

Read it as four pairs. Each clause appears twice, [fake] right above [sqlite], and both pass. That pairing is the guarantee made visible: the line ...returns_the_same_booking[fake] PASSED and the line ...returns_the_same_booking[sqlite] PASSED, together, prove that the fake and the real one agree on the save-and-read clause —including the got.start == START, the datetime field where in module 1 they diverged—. It's not that the fake passes its own version and the real one its own; it's that both pass the same line of assertion code, with the same provider plugged into the same place. Eight greens = four promises, each honored by both sides.

Why "against both" is the guarantee —and "against one" is nothing

Let's pause on the heart of the lesson, because it's subtle and it's everything. Imagine you ran the battery against a single provider. What would it prove?

  • Only against the fake: it would prove that the fake fulfills the contract. But the fake is your assumption about the real thing; that your assumption is consistent with itself says nothing about whether it matches SQLite. It's like verifying that your scale model meets your own blueprints: true, and empty with respect to the building.
  • Only against the real one: it would prove that SQLite fulfills the contract. Useful, but it doesn't close module 1's gap, because the gap wasn't "does SQLite fulfill?" but "does my fake behave like SQLite?". Verifying only the real one leaves the fake unaudited —and the fake is the one your hundreds of fast unit tests use—.

The guarantee comes only from running the same battery against both. It's a transitivity argument: if the fake passes contract C, and the real one passes the same contract C, then the fake and the real one agree on everything C asserts. Therefore, any unit test that uses the fake and relies only on C is relying on something the real one also fulfills. The fake stops being "an assumption I hope is faithful" and becomes "an implementation demonstrably equivalent to the real one within the contract". That's the sentence that closes the gap: the fake can't lie about what the contract covers, because the same contract the fake passes is passed by the real one.

Note the final nuance: "about what the contract covers". The guarantee is as broad as the contract. If a clause isn't in the battery —say, the order of find_by_room—, the fake and the real one could diverge there without anyone noticing, because the taste-test sheet doesn't measure that taste. That's why the contract must cover every promise a real consumer depends on; a promise outside the contract is a promise without a guarantee. (That crack —an unwritten promise— is exactly what lesson 6 exploits from the consumer side.)

The anatomy of the parametrization, in detail

It's worth understanding the pytest mechanism that makes all this possible, because you're going to use it nonstop. Three pieces:

  1. The fixture with params. @pytest.fixture(params=["fake", "sqlite"]) tells pytest: "this fixture has two versions". Pytest runs it once per value, and in each run request.param is the value of the moment ("fake", then "sqlite"). The fixture's body uses that value to build the appropriate provider. The fixture is the only part of the file that knows there are two implementations.

  2. Injection by name. Each test declares a parameter called repo. Pytest sees that a fixture called repo exists and passes it —"dependency injection" in pytest's sense—. The test receives one already-built repository and works against it, blind to which one it got. That's why the four tests are identical for both providers: there isn't a single implementation if in them.

  3. The [fake]/[sqlite] ids. When a fixture is parametrized, pytest adds the param value to each test's id, in brackets. That's why you see test_get_of_a_missing_id_raises[fake] and ...[sqlite]. That id isn't cosmetic: it's your diagnosis map. When a test fails, the bracket will tell you which provider failed the clause —the whole mechanism of lessons 5 and 6 depends on reading that bracket—.

The beauty of the arrangement is that adding a promise or a provider is local. A new promise = one more test_... (it runs against both automatically). A new provider = one more value in params (all the clauses verify it automatically). The contract grows without you having to duplicate anything, because the battery and the providers are decoupled by the fixture.

Common mistakes

Writing two batteries, one per provider. What happens: someone copies the four tests, makes a copy "for the fake" and another "for SQLite", and maintains them separately. Why it happens: it seems more explicit to have a file per provider. How to detect it: if you have two files with the same assertions and different providers, and you have to edit both when a clause changes, you're duplicating. How to fix it: a single battery, one parametrized fixture. Duplication isn't just more work: it breaks the guarantee, because nothing forces the two copies to assert exactly the same thing, and as soon as they diverge (an extra assert in one), they stop proving that the providers match. The guarantee comes from it being the same assertion for both.

Habitually running only one side. What happens: out of habit, the team runs -k fake (fast) day to day and "every so often" the complete battery. Why it happens: the fake side is faster and gives a comfortable green. How to detect it: if the [sqlite] side doesn't run on every relevant change, a provider breaking change can live unseen until someone runs the whole battery. How to fix it: the value is in running both sides together, which is what paired the promises. It's fine to use -k to inspect one side (lesson 3), but the verdict that protects the deploy is the complete battery, with its eight.

Believing the guarantee covers what the contract doesn't say. What happens: the eight greens give the feeling that "the fake and the real one are identical", and someone relies on a behavior the battery doesn't verify (the order of find_by_room, that get returns the same object and not a copy). Why it happens: "eight greens" feels like total equivalence. How to detect it: ask yourself whether the thing you're relying on has an assert in the battery. If it doesn't, it's not guaranteed. How to fix it: the guarantee is exactly as broad as the written contract. To extend it, add the clause (one more test_...); to avoid the trap, don't assume equivalence beyond what the battery asserts. Lesson 6 is the living case of this error.

Exercises

Exercise 1 — Transitivity, in your words. Explain, without using the word "contract" more than once, why running the same battery against the fake and the real one guarantees that a unit test using the fake doesn't hide a divergence with the real one —as long as the unit test relies only on what the battery verifies—.

See solution

It's a chaining of three facts. First: the battery verifies a set of behaviors B (save-and-read returns the same, missing id raises, etc.). Second: the fake passes the battery, so the fake fulfills B. Third: the real one passes the same battery, so the real one fulfills B. From there it follows that, in everything B describes, the fake and the real one do the same thing —not "similar", the same, because they passed the same assertions—. Now take a unit test that uses the fake and relies only on behaviors of B: since the real one fulfills B just like the fake, that unit test would still be true if you plugged in the real one. Therefore there can't be a hidden divergence within B: the fake doesn't lie about B. The condition "as long as it relies only on what the battery verifies" is essential, because outside B (an unmeasured order, for example) the guarantee doesn't reach, and there they could indeed diverge.

Exercise 2 — Predict the output after breaking a clause. Without running anything: if someone broke FakeBookingRepository.get so it returned a copy of the booking instead of the saved object, but keeping all the fields the same, would any of the eight greens change? And if a new clause additionally verified repo.get("bk-1") is repo.get("bk-1") (identity, not equality)?

See solution

With the current four clauses: no green changes. The battery's assertions compare by equality (got.id == "bk-1", got.start == START, got.status == "cancelled"): they look at the fields' values, not the object's identity. A copy with the same fields is equal (==) to the original, so all the clauses still pass. The eight greens hold. And this is correct: the contract doesn't promise that get returns the very same object, only that it returns a booking with the correct fields.

With the new identity clause (is): the [fake] side of that clause would pass only if the fake returns the same object, and the [sqlite] side would always fail. SQLite can never return the same Python object: it reconstructs a new Booking from the columns on each get, so repo.get("bk-1") is repo.get("bk-1") is False for the real one. If you added that clause, [sqlite] would go red. Which reveals an important lesson: don't put in the contract a promise the real one can't fulfill. Object identity is a detail of the fake (which stores references in a dict), not a promise a repository should honor. A contract should assert only what every reasonable implementation can fulfill; demanding identity would break the real one and add nothing a sensible consumer needs.

Exercise 3 — Extend the guarantee. A new consumer depends on find_by_room returning the bookings sorted by start ascending. Today the contract doesn't promise it, so the guarantee doesn't cover it. Write the clause that would add it to the battery, and explain what the fake and the real one would have to do to pass it —and what would happen if one of the two doesn't—.

See solution

The new clause:

def test_find_by_room_returns_bookings_sorted_by_start(repo):
    repo.save(a_booking(id="bk-late", room_id="focus", start=datetime(2026, 3, 10, 11)))
    repo.save(a_booking(id="bk-early", room_id="focus", start=datetime(2026, 3, 10, 9)))
    starts = [b.start for b in repo.find_by_room("focus")]
    assert starts == sorted(starts)      # promises ascending order by start

(It requires a_booking to accept a variable start; it's a minor tweak to the helper.)

What each provider would have to do to pass it:

  • The real one: add ORDER BY start to find_by_room's query (SELECT ... WHERE room_id = ? ORDER BY start). Without that SQL clause, SQLite returns the rows in an unguaranteed order, and the assertion could fail.
  • The fake: sort the list before returning it (sorted(..., key=lambda b: b.start)), because the dict preserves insertion order, not start order.

What would happen if one doesn't do it: its side of the clause would go red, and you'd know exactly which. If you add the clause but only fix the real one, test_find_by_room_returns_bookings_sorted_by_start[fake] fails; if you only fix the fake, [sqlite] fails. That red is the guarantee working: the moment you write the promise as a clause, the battery demands that both fulfill it, and gives away the one that doesn't. That's how the guarantee is extended: the promise that before lived as a tacit assumption of the consumer (and was a time bomb, lesson 6) becomes a clause verified against both providers.

Summary and next step

In this lesson you reunited the two chairs and saw that they weren't two batteries, but one, run against the two providers. The parametrized fixture (params=["fake", "sqlite"]) makes each clause run twice, and the eight greens read as four pairs —[fake] and [sqlite] side by side—, each pair proving that the two providers agree on that promise. With the taste test to two cooks you understood that interchangeability isn't assumed, it's proven, and it's proven by applying one sheet to both sauces. And you arrived at the sentence that closes module 1's gap: the fake can't lie about what the contract covers, because the same contract the fake passes is passed by the real one —a guarantee by transitivity, as broad as the written contract, and not an ounce more—.

Before moving on you should be able to: explain why the guarantee comes from running the battery against both and not against one; read the output of eight as four pairs; describe the three pieces of the parametrization (fixture with params, injection by name, bracketed ids); and recognize that the guarantee doesn't cover what the contract doesn't assert.

Up to here you saw the mechanism at rest: green, calm, honored by both sides. What comes next is seeing it fire. Lesson 5 introduces a change in the provider —the get that returns None instead of raising— and runs the battery: the [sqlite] side goes red on the exact clause, before deploying, while the [fake] stays green. It's the guide's golden payoff: catching an incompatible change before it reaches production.

Resources

← Previous3. The provider sideNext →5. Catching a provider breaking change