Module 3: Contract Testing Consumer And Provider

4. The parametrized battery against the fake and SQLite

Description

You have the contract's what (behavior, lesson 2) and the who (the consumer, lesson 3). This lesson delivers the how: the concrete mechanism, in pytest, that makes a single battery of tests run against the FakeBookingRepository and against the SqliteBookingRepository without duplicating a line. That mechanism is the parametrized fixture, and it's so important that it deserves to be taken apart down to the last screw. You've seen it go by three lessons in a row —the line @pytest.fixture(params=["fake", "sqlite"])—; here you stop seeing it as magic and understand exactly what pytest does when it finds it.

The underlying idea is simple and powerful. A normal test receives a fixed value; a test that receives a parametrized fixture runs once per parameter of that fixture, with the corresponding value injected. If the fixture has two parameters —"fake" and "sqlite"— each test that uses it runs twice. Four clauses times two implementations give eight cases, and pytest labels them with the parameter in brackets: [fake] and [sqlite]. That label isn't decoration: it's your way of reading, at a glance, against which implementation each clause passed or failed. And that multiplication —one battery, N providers— is exactly what turns "I hope the fake and the real one match" into "they match, or there's a red with a first and last name".

Connection to the module: this lesson is the module's mechanical hinge. Lessons 1 to 3 established the concept; from lesson 5 on they collect on it. But to collect on it —to catch module 2's divergence in lesson 5, to distinguish state from interaction in lesson 6— you need to understand how a battery is run against several providers, because that's the instrument we'll use over and over. Here you master it: the fixture with params, the request object, the [fake]/[sqlite] ids, and why "one battery, two providers" is the technical guarantee that the fake can't lie about what the contract covers.

Analogy: one inspection line, many cars

Think of the vehicle inspection line. There's one test protocol —brakes, lights, emissions, steering play— and through that same line pass cars of all makes: a sedan, a truck, a sports car. The inspector doesn't rewrite the protocol for each car; they run the same protocol, and each car passes or fails it depending on whether it complies. At the end, the report says for each car which tests it passed: "sedan: brakes OK; truck: brakes OK; sports car: emissions FAILED". One protocol, many vehicles, a verdict for each.

The parametrized fixture is that inspection line, and the implementations are the cars. The protocol —the four clauses of the contract— is written once. The "cars" —the fake and the real one— are declared in params. Pytest runs each one through all the tests and gives you a report with each one's name in brackets: [fake] passed all four, [sqlite] passed all four. If one day you put in a car that doesn't comply —the buggy fake—, the report will say so exactly as the inspection line gives away the sports car with high emissions: [buggy-fake] FAILED on the test that fails. One protocol, many implementations, a verdict for each. That's a parametrized battery.

Anatomy of the parametrized fixture

Let's look at the central piece line by line. It's short, and each part does a job:

import pytest

from reservo.doubles import FakeBookingRepository
from reservo.sqlite_repo import SqliteBookingRepository


@pytest.fixture(params=["fake", "sqlite"])   # (1) two parameters: two "cars"
def repo(request):                            # (2) receives 'request'
    if request.param == "fake":               # (3) request.param is this run's parameter
        return FakeBookingRepository()        # (4a) in-memory provider
    return SqliteBookingRepository(sqlite3.connect(":memory:"))  # (4b) real provider
  1. @pytest.fixture(params=["fake", "sqlite"]) — the decorator turns repo into a parametrized fixture. The params list declares the values; there will be one run of each test per value. Here, two values: "fake" and "sqlite".
  2. def repo(request): — the fixture receives the special request object, which pytest injects. It's the thread that connects the fixture with the current run: through it we know which of the parameters is up at this moment.
  3. request.param — in the [fake] run, request.param is "fake"; in the [sqlite] run, it's "sqlite". It's how the fixture decides what to build in each pass.
  4. The return — depending on the parameter, the fixture builds and returns the corresponding provider: a FakeBookingRepository in memory, or a SqliteBookingRepository with an in-memory SQLite database (:memory:, an ephemeral database that lives only for the duration of the test —clean and extremely fast—).

Any test that declares repo as an argument will receive, without knowing it, first the fake and then the real one. The test doesn't change; pytest runs it twice, each with a different provider in the repo parameter. That's the whole mechanic: the test is written once and run N times, one per fixture parameter.

How pytest expands a battery

To see the multiplication with your own eyes, ask pytest to only collect the tests, without running them. With the four-clause battery and the two-parameter fixture:

python3 -m pytest tests/test_repository_contract.py --collect-only -q
tests/test_repository_contract.py::test_save_then_get_returns_the_same_booking[fake]
tests/test_repository_contract.py::test_save_then_get_returns_the_same_booking[sqlite]
tests/test_repository_contract.py::test_get_of_a_missing_id_raises[fake]
tests/test_repository_contract.py::test_get_of_a_missing_id_raises[sqlite]
tests/test_repository_contract.py::test_saving_the_same_id_twice_updates_not_duplicates[fake]
tests/test_repository_contract.py::test_saving_the_same_id_twice_updates_not_duplicates[sqlite]
tests/test_repository_contract.py::test_find_by_room_returns_only_that_rooms_bookings[fake]
tests/test_repository_contract.py::test_find_by_room_returns_only_that_rooms_bookings[sqlite]

8 tests collected in 0.01s

There's the expansion, without having run anything yet. You wrote four test functions; pytest collected eight cases. Each function appears twice, one with [fake] and one with [sqlite]. The text in brackets is the parameter's id: pytest takes it from the strings you put in params ("fake", "sqlite"), which is why they came out readable. That id is your map: it tells you, for each case, which implementation is being tested. When something fails, the id will tell you against which provider it failed —and that, as you'll see in lesson 5, is half the solution—.

Worked example: the complete battery, eight greens

Now let's actually run the BookingRepository's contract battery —the four clauses from the previous lessons— with the parametrized fixture:

# tests/test_repository_contract.py
import sqlite3
from datetime import datetime

import pytest

from reservo.doubles import FakeBookingRepository
from reservo.models import Booking
from reservo.sqlite_repo import SqliteBookingRepository

START = datetime(2026, 3, 10, 9)
END = datetime(2026, 3, 10, 12)      # Focus 3 h


def a_booking(id="bk-1", room_id="focus", status="confirmed", price_cents=6000):
    return Booking(id=id, room_id=room_id, member_id="m-ana",
                   start=START, end=END, status=status, price_cents=price_cents)


@pytest.fixture(params=["fake", "sqlite"])
def repo(request):
    if request.param == "fake":
        return FakeBookingRepository()
    return SqliteBookingRepository(sqlite3.connect(":memory:"))


def test_save_then_get_returns_the_same_booking(repo):
    booking = a_booking()
    repo.save(booking)
    assert repo.get("bk-1") == booking


def test_get_of_a_missing_id_raises(repo):
    with pytest.raises(KeyError):
        repo.get("does-not-exist")


def test_saving_the_same_id_twice_updates_not_duplicates(repo):
    repo.save(a_booking(status="confirmed"))
    repo.save(a_booking(status="cancelled"))     # same id "bk-1"
    assert repo.get("bk-1").status == "cancelled"
    assert len(repo.find_by_room("focus")) == 1


def test_find_by_room_returns_only_that_rooms_bookings(repo):
    repo.save(a_booking(id="bk-1", room_id="focus"))
    repo.save(a_booking(id="bk-2", room_id="studio"))
    found = repo.find_by_room("focus")
    assert [b.id for b in found] == ["bk-1"]

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

python3 -m pytest tests/test_repository_contract.py -v
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 8 items

tests/test_repository_contract.py::test_save_then_get_returns_the_same_booking[fake] PASSED [ 12%]
tests/test_repository_contract.py::test_save_then_get_returns_the_same_booking[sqlite] PASSED [ 25%]
tests/test_repository_contract.py::test_get_of_a_missing_id_raises[fake] PASSED [ 37%]
tests/test_repository_contract.py::test_get_of_a_missing_id_raises[sqlite] PASSED [ 50%]
tests/test_repository_contract.py::test_saving_the_same_id_twice_updates_not_duplicates[fake] PASSED [ 62%]
tests/test_repository_contract.py::test_saving_the_same_id_twice_updates_not_duplicates[sqlite] PASSED [ 75%]
tests/test_repository_contract.py::test_find_by_room_returns_only_that_rooms_bookings[fake] PASSED [ 87%]
tests/test_repository_contract.py::test_find_by_room_returns_only_that_rooms_bookings[sqlite] PASSED [100%]

============================== 8 passed in 0.03s ==============================

Eight greens, four clauses times two providers. Read the output like an inspection report: each clause passed both [fake] and [sqlite]. That's the complete certification of the contract: the fake and the real one behave the same across the four clauses, verified, not assumed. And I want you to notice something that's easy to overlook: you didn't write eight tests, you wrote four. The parametrized fixture duplicated each one. If tomorrow you add a third provider —say a file-based repository— it's enough to add "file" to params; the four clauses will run against it too, and you'll have twelve cases without touching a single line of test. That's the economy of the approach: the effort of writing the contract is fixed; verifying one more implementation is a word in a list.

Why "one battery, two providers" is the guarantee

Let's pause on the strong claim of the whole module: the parametrized battery is what guarantees the fake can't lie. The key word is one. When the fake and the real one are tested with the same battery —the same test text, run twice—, it's impossible for different things to be demanded of them. There aren't two specs that can drift: there's a single one. If a clause changes, it changes for both at once, because it's the same function. If a provider stops fulfilling a clause, its run goes red while the other's stays green, and the id in brackets tells you which failed. The only way for the fake to "pass as good" is for it to really behave like the real one across the four clauses —and that's exactly what we wanted to guarantee—.

Compare it with the alternative that seems equivalent and isn't: two twin test files, test_fake_repo.py and test_sqlite_repo.py. Even if today they assert the same thing, they're two independent texts. Tomorrow someone adjusts an assertion in one and forgets the other, and without any red jumping out you've gone back to module 2 —the fake and the real one tested against different expectations, free to diverge—. Parametrization eliminates that possibility at the root: there's no "the other file" to forget, because there's one. The guarantee doesn't come from the discipline of keeping two things in sync; it comes from there being a single thing. That's the technical argument for why this module insists so much on the parametrized battery over duplicated suites.

A detail: fixture with params versus @pytest.mark.parametrize

Pytest has two ways to parametrize, and it's worth knowing when each goes. We use a fixture with params because what varies is a resource —the repository— that several tests share and that sometimes needs to be built carefully (open a connection, create the schema). The fixture centralizes that construction: it's written once and all the tests that ask for repo receive it ready-made.

The other way, @pytest.mark.parametrize, goes directly on a test and is better when what varies is input data of that particular test —a list of expected prices, several input/output pairs—. You could parametrize the battery with @pytest.mark.parametrize over each function, passing the repository class, but you'd have to repeat the list in each test and rebuild the resource in each one. The fixture with params is the right tool when the axis of variation is "which implementation of the collaborator?", which is exactly our case. Practical rule: data that varies per test → parametrize; a resource/collaborator that several tests share → fixture with params.

Common mistakes

Putting params in the test's parametrize and ending up with duplicated specs. What happens: to vary the repository, someone copies @pytest.mark.parametrize("repo_cls", [FakeBookingRepository, SqliteBookingRepository]) over each of the four tests. Why it happens: parametrize is the first thing you learn. How to detect it: the same list of implementations repeated in four decorators is four places to forget to add the next provider. How to fix it: centralize the "which implementation" axis in a fixture with params; the tests just ask for repo and don't know how many providers there are. Adding one is a word in the fixture's list, not a change in four decorators.

Not reading the id in brackets on failure. What happens: a test goes red, the function name is read and the [fake]/[sqlite] at the end is ignored. Why it happens: the bracket seems like noise. How to detect it: if you ask yourself "but against which implementation did it fail?", the answer was in the id you didn't read. How to fix it: the id is half the diagnosis. test_get_of_a_missing_id_raises[sqlite] FAILED and ...[fake] FAILED tell opposite stories: the first says "the real one is broken", the second "the fake diverges". Always read it; in lesson 5 it's the key to knowing who's lying.

Building an expensive resource without considering the fixture's scope. What happens: the fixture opens a new connection and creates the schema on every run, and with many tests that starts to weigh. Why it happens: the default fixture is function scope —rebuilt per test—, which is right for isolating but not always the fastest. How to detect it: if the integration suite becomes slow, look at how many times the resource is built. How to fix it: for SQLite :memory: the cost is tiny and function scope is ideal (each test starts with a clean database, no contamination). For truly expensive resources, there are larger scopes and isolation strategies —but that's module 7's topic, Test data and isolation in integration, don't get ahead of it here—.

Exercises

Exercise 1 — Count the cases. You have a contract battery with 5 clauses (five test functions) and a repo fixture with params=["fake", "sqlite", "file"]. Without running anything, say how many cases pytest will collect and what one clause's ids will look like.

See solution

Pytest will collect 15 cases: 5 clauses × 3 parameters. Each test function expands once per params value.

For any clause —say test_get_of_a_missing_id_raises— the ids would be three:

test_get_of_a_missing_id_raises[fake]
test_get_of_a_missing_id_raises[sqlite]
test_get_of_a_missing_id_raises[file]

The math is simple and worth having clear: cases = clauses × implementations. It's also the reason adding an implementation is so cheap: adding "file" to params doesn't add a test function, but multiplies the coverage by the number of clauses you already have. Five well-written clauses become, for free, five more tests against each new provider.

Exercise 2 — Choose the tool. For each situation, say whether a fixture with params or a @pytest.mark.parametrize over the test is appropriate: (a) verifying the repository's contract against the fake, SQLite, and a future file-based repository; (b) verifying that price_cents gives 6000, 3000, and 0 for 3, 1.5, and 0 discount hours respectively; (c) verifying the PaymentGateway's contract against a stub and a fake.

See solution
  • (a) Fixture with params. The axis of variation is "which repository implementation?" —a collaborator the four clauses share and that needs to be built (open connection, create schema)—. The fixture centralizes that construction and each clause just asks for repo. Adding the file-based repository is a word in params.
  • (b) @pytest.mark.parametrize. Here what varies is input/output data of the same pure-logic test: (hours, expected) pairs. There's no resource to build or collaborator to swap. @pytest.mark.parametrize("hours, expected", [(3, 6000), (1.5, 3000), (0, 0)]) is the natural way.
  • (c) Fixture with params. Same as (a): the axis is "which gateway implementation?" —stub or fake—, a collaborator the gateway contract's clauses share. Fixture with params=["stub", "fake"].

The pattern: if you swap implementations of a collaborator that several clauses share, fixture with params; if you vary data of a single test, parametrize.

Exercise 3 — Add a provider. You have the BookingRepository battery with params=["fake", "sqlite"], eight greens. A colleague wrote a SqliteBookingRepository that uses a file on disk instead of :memory:, and wants to verify it with the same contract. Describe the minimal change to include it and how many cases there will be afterward.

See solution

The minimal change lives only in the fixture; none of the four test functions is touched. A third parameter and the branch that builds the file-based repository are added:

@pytest.fixture(params=["fake", "sqlite-memory", "sqlite-file"])
def repo(request, tmp_path):
    if request.param == "fake":
        return FakeBookingRepository()
    if request.param == "sqlite-memory":
        return SqliteBookingRepository(sqlite3.connect(":memory:"))
    db_file = tmp_path / "reservo.db"                       # the test's temp file
    return SqliteBookingRepository(sqlite3.connect(db_file))

(pytest's tmp_path fixture gives a unique temporary directory per test, so the file is created and destroyed clean on each run —the fine-grained isolation of real resources is module 7's topic—.)

After the change there will be 12 cases: 4 clauses × 3 parameters. The same four clauses now certify three implementations, and in the output you'll see [fake], [sqlite-memory], and [sqlite-file] for each. This is the economy of the parametrized contract in one sentence: the work of writing the spec is already done; adding an implementation is putting it in the list and letting the battery certify it.

Summary and next step

In this lesson you mastered the mechanism that makes the shared contract possible: pytest's parametrized fixture. You took apart the line @pytest.fixture(params=["fake", "sqlite"]) piece by piece —the params, the request object, request.param, the return that builds each provider— and saw, with --collect-only, how pytest expands four test functions into eight cases, each labeled with its [fake]/[sqlite] id. You ran the complete battery: eight greens, the certification that the fake and the real one fulfill the four clauses. And you understood why "one battery, two providers" —one, not two twin suites— is the technical guarantee that the fake can't lie: there aren't two specs that can diverge, there's a single one run N times. With the vehicle inspection line you fixed the image: one protocol, many vehicles, a verdict for each.

Before moving on you should be able to: explain what request.param does in each run; predict how many cases a battery collects (clauses × implementations) and what its ids look like; choose between a fixture with params and @pytest.mark.parametrize depending on whether a collaborator or some data varies; and add an implementation to the contract by touching only the fixture.

You have the instrument sharpened. Lesson 5 uses it for what the whole module promised: we put the BuggyFakeBookingRepository from module 2 —the one that returns None instead of raising— into the battery and see it caught in red, while the real one stays green. There the id in brackets stops being a detail and becomes the finger that points at the culprit.

Resources