Module 7: Test Data And Isolation In Integration

8. Mini-project: isolate a Reservo integration suite

Description

The moment has come to bring the seven lessons together in a single deliverable, over a complete and realistic case. You're going to receive a Reservo integration suite that is broken by shared state —the sin this whole module fought— and your job is to turn it into an isolated and repeatable suite: green in any order and however many times you run it. It's not a toy exercise: it's exactly the work you'll do when you inherit a flaky integration suite from a team, or when yours starts failing intermittently as it grows. The mini-project has the shape of an investigation with a fix: first you reproduce the problem and diagnose it with the signature you already know —the verdict changes with the order—, then you choose the right isolation technique and justify it, and finally you demonstrate the cure worked by running the suite in various orders and various times. The deliverable is the two versions —the contaminated one in red, the isolated one in green— plus the justification of which technique you chose and why.

What makes this close valuable is that the broken suite doesn't fail in just any way: it fails differently depending on the order, which is the most confusing manifestation of shared state and the one that makes you lose the most hours when you don't recognize it. You're going to see, with real output, how the same suite gives one failure in definition order and another different failure in reversed order —a test that was passing now fails, and one that was failing now passes—. That behavior, which seems haunted, has a one-line explanation once you diagnose it, and a one-fixture cure. When you finish, you won't just have the isolated suite: you'll have the complete method —reproduce, diagnose by the signature, choose the technique, verify in various orders— to fix any contaminated integration suite you come across.

Connection to the module: this mini-project is the synthesis of everything. It uses lesson 2's diagnosis (the order-dependent signature), lesson 4's technique (the fresh-database fixture with yield), lesson 5's resource decision (:memory:), lesson 6's seeding criterion (explicit state), and lesson 7's verification (various orders, various runs). And it closes the guide's integration part leaving you an impeccable suite —isolated and repeatable— that module 8, the capstone, will be able to use with confidence when joining contract and integration in a final deliverable. Here you prove you know how to make an integration suite tell the truth; there you'll combine it with the contract to close the guide.

The starting point: a haunted suite

You're handed this Reservo integration suite. Three tests, each correct if you read it alone, sharing a live real repository for the whole suite.

# tests/test_suite_shared.py — BEFORE: integration suite with a SHARED real repo
import sqlite3
from datetime import datetime
from reservo.calendar import Calendar
from reservo.doubles import FixedClock, SpyEmailSender, StubPaymentGateway
from reservo.models import Member, Room
from reservo.services import BookingService
from reservo.sqlite_repo import SqliteBookingRepository

FOCUS = Room("focus", "Focus", 4, 2500); ANA = Member("m-ana", "Ana", "pro")

# The sin: a live real DB for the whole suite.
REPO = SqliteBookingRepository(sqlite3.connect(":memory:"))

def service():
    return BookingService(Calendar(), FixedClock(datetime(2026, 3, 1, 9)),
                          StubPaymentGateway(True), SpyEmailSender(), REPO)

def test_book_creates_one_focus_booking():
    service().book(FOCUS, ANA, datetime(2026, 3, 10, 9), datetime(2026, 3, 10, 12))
    assert len(REPO.find_by_room("focus")) == 1

def test_focus_is_empty_before_my_booking():
    assert len(REPO.find_by_room("focus")) == 0     # assumes the DB is clean
    service().book(FOCUS, ANA, datetime(2026, 3, 11, 9), datetime(2026, 3, 11, 12))
    assert len(REPO.find_by_room("focus")) == 1

def test_exactly_one_confirmed_total():
    rows = REPO.find_by_room("focus")
    assert len(rows) == 1                            # assumes only its own

All three tests load onto the same module-level REPO, and since book commits, each booking one creates stays alive for the next ones. None cleans up. Let's run it in definition order.

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

python3 -m pytest tests/test_suite_shared.py -v
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 3 items

tests/test_suite_shared.py::test_book_creates_one_focus_booking PASSED     [ 33%]
tests/test_suite_shared.py::test_focus_is_empty_before_my_booking FAILED   [ 66%]
tests/test_suite_shared.py::test_exactly_one_confirmed_total PASSED         [100%]

One failure: the second test, which expected an empty database and inherited the first's booking. So far, nothing new compared with lesson 1. The revealing part comes now: let's run the same suite in reversed order.

python3 -m pytest tests/test_suite_shared.py::test_exactly_one_confirmed_total \
                  tests/test_suite_shared.py::test_focus_is_empty_before_my_booking \
                  tests/test_suite_shared.py::test_book_creates_one_focus_booking -v
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 3 items

tests/test_suite_shared.py::test_exactly_one_confirmed_total FAILED        [ 33%]
tests/test_suite_shared.py::test_focus_is_empty_before_my_booking PASSED   [ 66%]
tests/test_suite_shared.py::test_book_creates_one_focus_booking FAILED     [100%]

Look closely, because this is what drives crazy whoever doesn't recognize it. In definition order one failed (the second) and the other two passed. In reversed order two different ones fail (the first and the third) and the one that failed before passes. The same suite, the same code, without touching a line: three different verdicts depending on the order. A test that was passing now fails; one that was failing now passes. If you saw this without this module's diagnosis, you'd suspect the code, the environment, pytest, the position of the planets —and you'd lose hours—.

The diagnosis: the signature doesn't lie

With what you know, the diagnosis is immediate. The symptom —the verdict changes with the order— is the exact signature of the shared, non-isolated state from lesson 2. You don't have to read the tests' logic looking for a bug; you have to recognize the signature. The definitive confirmation is running each test alone:

python3 -m pytest tests/test_suite_shared.py::test_exactly_one_confirmed_total -v
# -> PASSED
python3 -m pytest tests/test_suite_shared.py::test_focus_is_empty_before_my_booking -v
# -> PASSED

Each test, isolated, passes. All three are correct separately; they only break when they share the REPO and inherit each other's bookings. The cause is in a single line —REPO = SqliteBookingRepository(sqlite3.connect(":memory:")) at the module level, shared by all— and book's commit that makes permanent what each one writes. Neither a bug in the assertions nor a pytest problem: real shared state, exactly what this module taught you to hunt.

With the problem understood, the reason for each verdict explains itself. In definition order: test_book_creates_one_focus_booking runs first over the empty database, books one, sees 1, passes —and leaves one booking—; test_focus_is_empty_before_my_booking runs second, expects 0, sees the inherited 1, fails; test_exactly_one_confirmed_total runs third, expects 1... and since the second test failed on its first line without getting to book, there are still exactly 1, so it passes by chance. In reversed order, the bookings accumulate differently and the expected 1s don't add up, and that's why two different ones fail. The exact detail of the counts matters less than the lesson: when the state is shared, each test's result depends on what the previous ones left, and that changes with the order. The cure isn't understanding each count; it's cutting the inheritance.

The cure: choose the technique and justify it

You have two isolation techniques from the module: the transaction rollback (lesson 3) and the fresh-database-per-test fixture (lesson 4). Which applies here? The decisive question is the rollback's precondition: does the code under test commit? This suite integrates through service().book(...), and book calls repo.save, which commits. Therefore, the rollback wouldn't work as-is: save's commit would make the bookings permanent before any teardown rollback could revert them —we demonstrated it in lesson 3—. The right technique is the fresh-database-per-test fixture: since each test gets its own :memory: database that dies when it ends, the commit stops mattering —it commits over a private database no one else will see—.

The resource: :memory:, because none of the three tests tests disk persistence; they all verify the service↔repository seam, which :memory: covers just as well and faster (lesson 5). Here's the cured suite:

# tests/test_suite_isolated.py — AFTER: the same suite, ISOLATED with a fixture
import sqlite3
import pytest
from datetime import datetime
from reservo.calendar import Calendar
from reservo.doubles import FixedClock, SpyEmailSender, StubPaymentGateway
from reservo.models import Member, Room
from reservo.services import BookingService
from reservo.sqlite_repo import SqliteBookingRepository

FOCUS = Room("focus", "Focus", 4, 2500); ANA = Member("m-ana", "Ana", "pro")

@pytest.fixture
def repo():
    conn = sqlite3.connect(":memory:")     # fresh DB per test
    yield SqliteBookingRepository(conn)
    conn.close()                           # destroyed in the teardown, no matter what

def service(repo):
    return BookingService(Calendar(), FixedClock(datetime(2026, 3, 1, 9)),
                          StubPaymentGateway(True), SpyEmailSender(), repo)

def test_book_creates_one_focus_booking(repo):
    service(repo).book(FOCUS, ANA, datetime(2026, 3, 10, 9), datetime(2026, 3, 10, 12))
    assert len(repo.find_by_room("focus")) == 1

def test_focus_is_empty_before_my_booking(repo):
    assert len(repo.find_by_room("focus")) == 0     # now it IS true: new DB
    service(repo).book(FOCUS, ANA, datetime(2026, 3, 11, 9), datetime(2026, 3, 11, 12))
    assert len(repo.find_by_room("focus")) == 1

def test_exactly_one_confirmed_total(repo):
    service(repo).book(FOCUS, ANA, datetime(2026, 3, 12, 9), datetime(2026, 3, 12, 12))
    assert len(repo.find_by_room("focus")) == 1     # its own booking, nothing else

The change is surgical and fits in three lines: the global REPO is replaced by a repo fixture that creates a fresh database per test and closes it in the teardown, and each test requests it by parameter. The assertions didn't change one bit: they're still the same reasonable statements as before. The only thing that changed is that now each one is true, because each test sees its own clean database. (Note: test_exactly_one_confirmed_total now seeds its own booking before counting, so its assertion == 1 asserts something about a state it established, not about what it was inheriting.)

The verification: green in any order, however many times

The cure isn't declared: it's demonstrated, with lesson 7's test. Definition order:

python3 -m pytest tests/test_suite_isolated.py -v
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 3 items

tests/test_suite_isolated.py::test_book_creates_one_focus_booking PASSED   [ 33%]
tests/test_suite_isolated.py::test_focus_is_empty_before_my_booking PASSED [ 66%]
tests/test_suite_isolated.py::test_exactly_one_confirmed_total PASSED      [100%]

============================== 3 passed in 0.01s ===============================

Reversed order:

python3 -m pytest tests/test_suite_isolated.py::test_exactly_one_confirmed_total \
                  tests/test_suite_isolated.py::test_focus_is_empty_before_my_booking \
                  tests/test_suite_isolated.py::test_book_creates_one_focus_booking -v
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 3 items

tests/test_suite_isolated.py::test_exactly_one_confirmed_total PASSED      [ 33%]
tests/test_suite_isolated.py::test_focus_is_empty_before_my_booking PASSED [ 66%]
tests/test_suite_isolated.py::test_book_creates_one_focus_booking PASSED   [100%]

============================== 3 passed in 0.01s ===============================

And twice in a row, without changing anything, for repeatability: 3 passed the first run, 3 passed the second, identical. Compare the before and after: the shared suite gave three different verdicts depending on the order; the isolated one gives the same green in all orders and all runs. That's exactly what the module was pursuing —independence and repeatability— achieved by changing three lines: the shared REPO for a fresh-database fixture. The haunting broke when the state inheritance was cut.

Your deliverable

Gather what you did, which is the complete method for fixing any contaminated integration suite:

  1. The contaminated suite in red, run in two orders, showing that the verdict changes with the order —the evidence that the problem is shared state, not a bug—.
  2. The diagnosis: name the signature (order-dependent verdict), confirm with the test of running each test alone (they pass isolated), and point out the exact cause (the shared module REPO plus save's commit).
  3. The chosen technique and its justification: fresh-database-per-test fixture, because the code integrates through a service that commits, which rules out the rollback; :memory: resource, because no test tests disk persistence.
  4. The isolated suite in green, run in normal order, in reversed order, and twice, showing the same green always —the proof of independence and repeatability—.

That package isn't just "I fixed the suite": it's the demonstration that you master the method —reproduce, diagnose by the signature, choose the technique with judgment, verify in various orders—, which is what you really take away from this module.

Common mistakes

Fixing the symptoms by adjusting the assertions. What happens: to make the shared suite pass, someone changes assert len == 0 to assert len >= 0 or adjusts the expected numbers until it "passes". Why it happens: it's the fastest fix in sight. How to detect it: if you touched the assertions instead of the isolation, you hid them; the suite "passes" but no longer verifies anything useful, and it still depends on the order underneath. How to fix it: the problem isn't in the assertions —they were correct—, it's in the shared state. Fix the isolation (the fixture) and leave the assertions intact.

Choosing the rollback without verifying the precondition. What happens: the rollback fixture is applied to this suite and it keeps failing, because book commits. Why it happens: the rollback is elegant and it's tempting to use it first. How to detect it: if you isolate with rollback and the contamination persists, check whether the code under test commits (here, via save). How to fix it: when the code integrates through a committing service, the technique is the fresh database per test; the rollback is for when you control the transaction. Choosing the technique is part of the work, not a detail.

Declaring victory with a single run in a single order. What happens: the isolated suite passes once in the usual order and is considered done. Why it happens: one green seems enough. How to detect it: you didn't prove it's independent until you ran it in another order; you didn't prove it's repeatable until you ran it twice. How to fix it: lesson 7's verification —various orders, various runs— is part of the deliverable, not an extra. A green in a single order doesn't distinguish an isolated suite from one that got lucky.

Exercises

Exercise 1 — Justify the technique in writing. Write the justification paragraph that would accompany your deliverable: explain in three or four sentences why you chose the fresh-database fixture and not the rollback for this suite, leaning on the rollback's precondition.

See solution

A possible justification:

"I chose the fresh-database-per-test fixture, not the rollback, because this suite integrates through service().book(...), and book internally calls repo.save, which commits. The rollback as an isolation technique can only undo what has not been committed; a commit inside the test makes the bookings permanent before any teardown rollback can revert them, so the rollback wouldn't isolate this suite —we demonstrated it in lesson 3, where an identical case failed—. The fresh-database fixture, in contrast, gives each test its own :memory: database that's destroyed when it ends, so the commit stops mattering: it commits over a private database that dies with the test and contaminates nobody. I used :memory: because none of the three tests tests disk persistence —they all verify the service↔repository seam—, so I gain speed without losing relevant realism."

The essence of the justification: name the rollback's precondition (it doesn't isolate if the code commits), note that booksave commits, and conclude that this is why the fresh database is the right technique. Choosing with judgment and being able to explain it is what separates applying a recipe from understanding the problem.

Exercise 2 — A test that does need disk. You're asked to add to the isolated suite a fourth test that verifies a booking survives closing the connection and reopening the database. Explain why that test can't use the :memory: fixture as-is, and write the fixture that does work, justifying the resource change.

See solution

That test can't use :memory: because it tests precisely what :memory: doesn't have: persistence beyond the connection. A :memory: database lives tied to its connection; when you close it, the whole database disappears, so "close and reopen" reopens nothing —the booking went away with the connection—. Verifying that the booking survives closing the connection requires a real file on disk, where the row persists even though the connection that wrote it is closed.

The temporary-file fixture, using tmp_path (lesson 5):

@pytest.fixture
def db_path(tmp_path):
    return tmp_path / "reservo.db"     # unique path per test, deleted by pytest

def test_booking_survives_reopen(db_path):
    conn1 = sqlite3.connect(db_path)
    service_with(SqliteBookingRepository(conn1)).book(FOCUS, ANA, START, END)
    conn1.close()                      # the connection that wrote it goes away
    conn2 = sqlite3.connect(db_path)   # new connection to the SAME file
    repo2 = SqliteBookingRepository(conn2)
    assert len(repo2.find_by_room("focus")) == 1   # it survived: it was on disk
    conn2.close()

The resource change is justified by what the test proves: most tests use :memory: (fast, enough), but this one verifies physical persistence, so it pays for disk with tmp_path —which also gives it a unique path per test and cleans it up on its own, keeping the isolation—. It's lesson 5's rule applied: the resource is decided by which boundary the test crosses, not by a uniform taste.

Exercise 3 — Armor the suite against order in the future. Your isolated suite passes in normal and reversed order. A colleague wants to guarantee that an order dependence never sneaks in again, not even in tests others add later. Propose a process practice (not a change to these three tests) that achieves it, and explain how it would detect a regression.

See solution

The practice is running the suite with the order shuffled on every CI execution, with a tool like pytest-randomly, which reorders the tests randomly on each run (and reports the seed used, to reproduce a failure). With that, every time someone adds a test, the suite is run in a different order; if the new test —or any other— depends on what ran before, sooner or later the shuffle will put it in an order that gives it away, and the run will fail.

How it detects a regression: in a well-isolated suite, shuffling changes nothing —green in any order, run after run—. The day someone introduces an order dependence (a test that shares state, a global REPO that sneaked back in), the shuffle will produce, in some run, an order where that test fails, and CI will go red with the seed that reproduces the problem. That way the order dependence is caught the first time it appears, not months later when it's already a consolidated flaky that nobody knows where it came from. It's the automatic and permanent version of the manual test —running in reversed order— that you did in this mini-project: turning "green in any order" into a guarantee that CI watches for you.

Mini-project and module summary

In this mini-project you walked through the complete method for fixing a contaminated integration suite. You reproduced the problem and saw its most confusing face —a suite that gives three different verdicts depending on the order—; you diagnosed it by the signature (order-dependent verdict) and confirmed it by running each test alone; you chose the technique with judgment —the fresh-database fixture, because the service commits and that rules out the rollback— over the right resource —:memory:, because nothing tests disk—; and you verified the cure by running the suite in normal order, reversed, and twice, seeing the same green always. The deliverable is that whole arc: order-dependent red → diagnosis → justified technique → green in any order.

And with this the guide's integration part closes. Review what the module left you: you understand why shared real state contaminates (the fake's dict dies with the test, SQLite's table persists) and you recognize its signature; you have the two isolation techniques —the transaction rollback, for when you control the transaction and recreating is expensive, and the fresh-database fixture with yield, for when the code commits or you want maximum simplicity—; you know how to choose the resource between :memory: and a temporary file with measured numbers; you seed minimal and explicit data and you know the boundary with the Builder; and you judge a suite by two criteria, independence and repeatability, which you verify by running it in various orders and various times. You have, in short, everything needed to write integration tests that tell the truth because each one starts from a known state and depends on nobody else.

What comes next is the capstone. Module 8 joins the guide's two disciplines —contract and integration— in a final deliverable: a consumer-driven contract for BookingRepository verified against the FakeBookingRepository and against the real SqliteBookingRepository, plus an integration test of BookingService with the real repository, isolated and repeatable as you learned here. The impeccable integration suites you now know how to build are half of that deliverable; the other half is the contract from modules 3 and 4. The capstone joins them and closes the guide.

Resources

  • pytest documentation — yield fixtures (recommended teardown) — the reference for the technique with which you cured the suite: the fresh-database-per-test fixture that isolates even when the code under test commits.
  • sqlite3 — Transaction control (Python documentation) — the reference for why save's commit rules out the rollback as a technique for this suite, the key to the mini-project's justified choice.
  • pytest-randomly — the plugin that shuffles the order on every run, the process practice from exercise 3 for armoring the suite against future order dependencies.
  • test-doubles-and-test-data-guide — the sister guide with the Builder pattern for complex test data, this guide's boundary; and with the FakeBookingRepository that module 8's capstone will verify against the contract alongside the real repository.