Module 7: Test Data And Isolation In Integration

1. Module introduction: the state that doesn't go away on its own

Description

You've reached the module that fixes the problem the previous six kept sweeping under the rug. You learned to test against the real thing —the contract that keeps the double honest, the first integration of BookingService with the SqliteBookingRepository, the boundaries of the database, the file, and HTTP—, and at each step we overlooked something that now has to be faced head-on: real state persists. When a unit test uses the FakeBookingRepository, it starts with a freshly created empty dict, does its thing, and when it ends throws the instance in the trash; the next test creates another empty dict and knows nothing of the previous one. Each test is born in a clean world, for free, without you having to think about it. With a real database, that stops being free. A booking one test writes to SQLite's bookings table doesn't evaporate when the test ends: it's still in the table, waiting. And the next test, which queries that same table, finds it there, without having asked for it, and makes decisions about a state it didn't create.

That's the module's whole problem, and it's more serious than it sounds. A test that inherits data from another doesn't always fail: it fails sometimes, depending on what ran before. It passes when you run it alone, because there's nobody to contaminate it, and it fails when you run it in the suite, after the test that left it a booking. Or worse: it passes today and fails tomorrow because someone added a new test that runs before. It's the flakiness the test-failure-diagnosis module taught you to fear, and its number-one cause in integration has a first and last name: shared real state that isn't isolated. This module gives you the two tools to close it at the root —the transaction rollback and the real-resource fixture— and the criterion so that each integration test starts from a known state and ends without leaving a trace, in any order and however many times you run it.

Connection to the module: this lesson is the map of the territory. Here you see the problem with your own eyes —an integration suite that changes color depending on the order— and the destination —the same suite, isolated, always green—, to know where we're going before studying the techniques. Lesson 2 dissects why real state contaminates and the fake doesn't; lesson 3 installs the transaction rollback; lesson 4, the fixture with yield; lesson 5 chooses between :memory: and a file with measured numbers; lesson 6 seeds data with criterion; lesson 7 names the two principles —independent and repeatable— and demonstrates them; and lesson 8 is the mini-project that brings it all together. The hard border: the Builder pattern for manufacturing complex test data is the doubles guide (test-doubles-and-test-data-guide), not this one; here the focus is the isolation of the real resource, not the data factory. And the capstone that unites contract and integration is module 8.

Analogy: the lab and the shared workbench

Think of two ways to work in a chemistry lab. In the first, each student receives a set of disposable equipment: a new beaker, a new pipette, a clean burner. They do their experiment, note the result, and throw everything away. The next student receives another new set. Nobody inherits anyone's residue; if your reaction came out blue, it came out blue because of what you put in, not because of a drop left from the previous experiment. That's the unit test with doubles: fresh, disposable equipment on each run, without memory between one and the next.

In the second way, everyone shares a single workbench with a single set of equipment, and nobody cleans it between experiments. The first student does their reaction and leaves the beaker with a residue. The second arrives, doesn't notice it, pours their reagents on top, and gets a color that corresponds neither to their experiment nor to the previous one: it's the mix of the two. Whose fault is the strange result? Nobody's in particular; it's the shared residue's. And worst of all: if you change the order the students go through the bench, the results change, because each inherits what the one before left. That dirty shared bench is your unisolated integration database. This module is learning to clean the bench between experiments —or to give each one their own bench— so that each test's result depends only on what that test did, and nothing else.

The problem, with the usual pieces

Let's recall the two faces of the repository, because the difference between them is the problem. The fake, which you already know, stores the bookings in a dict that lives inside the instance:

# reservo/doubles.py — the fake is born empty in each test
class FakeBookingRepository:
    def __init__(self):
        self._store = {}                  # new dict every time it's instantiated

    def save(self, booking):
        self._store[booking.id] = booking

    def find_by_room(self, room_id):
        return [b for b in self._store.values() if b.room_id == room_id]

When a test does FakeBookingRepository(), it gets an empty dict. When the test ends and the variable goes out of scope, Python collects the instance and the dict with it. The next test does another FakeBookingRepository() and starts from scratch. The isolation is automatic because the state lives in a Python object that dies with the test.

The real repository doesn't work that way. Its state doesn't live in a Python object that Python collects: it lives in a SQLite table, in a connection that —if you share it between tests— stays open and keeps having all the rows any test wrote to it.

# reservo/sqlite_repo.py — the state lives in the table, doesn't die with the test
class SqliteBookingRepository:
    def __init__(self, connection):
        self._conn = connection
        self._conn.execute(SCHEMA)         # CREATE TABLE IF NOT EXISTS bookings ...

    def save(self, booking):
        self._conn.execute("INSERT INTO bookings ... ON CONFLICT(id) DO UPDATE ...", (...))
        self._conn.commit()                # the row stays PERMANENT

    def find_by_room(self, room_id):
        rows = self._conn.execute(
            "SELECT ... FROM bookings WHERE room_id = ?", (room_id,)).fetchall()
        return [Booking(*r) for r in rows]

Notice save: it ends with commit(), which makes the row permanent. If two tests share the same connection, the booking the first saves is in the table when the second queries. The state didn't go away on its own, because there's no Python instance whose death erases it: it's in the database, which is exactly what we'd want in production and what ruins our lives in the tests if we don't isolate it.

Worked example: the gap, at a glance

Let's see the problem and its solution before studying any technique, to have the destination clear. First, the sin: an integration suite where three tests share a single real repository, alive for the whole suite. Each test reasonably assumes that the Focus room starts without bookings for it. But they share the table, so they step on each other.

# tests/test_suite_shared.py — the sin: a SHARED real repository
import sqlite3
from datetime import datetime
from reservo.calendar import Calendar
from reservo.doubles import FixedClock, SpyEmailSender, StubPaymentGateway
from reservo.models import Member, Room
from reservo.services import BookingService
from reservo.sqlite_repo import SqliteBookingRepository

FOCUS = Room("focus", "Focus", 4, 2500)
ANA = Member("m-ana", "Ana", "pro")

# A real DB alive for the WHOLE suite. Nobody cleans it between tests.
REPO = SqliteBookingRepository(sqlite3.connect(":memory:"))

def service():
    return BookingService(Calendar(), FixedClock(datetime(2026, 3, 1, 9)),
                          StubPaymentGateway(True), SpyEmailSender(), REPO)

def test_book_creates_one_focus_booking():
    service().book(FOCUS, ANA, datetime(2026, 3, 10, 9), datetime(2026, 3, 10, 12))
    assert len(REPO.find_by_room("focus")) == 1

def test_focus_is_empty_before_my_booking():
    assert len(REPO.find_by_room("focus")) == 0     # assumes the DB is clean
    service().book(FOCUS, ANA, datetime(2026, 3, 11, 9), datetime(2026, 3, 11, 12))
    assert len(REPO.find_by_room("focus")) == 1

Each test, read alone, is correct. The first books once and expects to see one booking; the second expects Focus to start empty for it and then books. The problem is in no test: it's that they share REPO, and book's commit leaves the first's booking alive when the second queries.

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

python3 -m pytest tests/test_suite_shared.py -v

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 2 items

tests/test_suite_shared.py::test_book_creates_one_focus_booking PASSED    [ 50%]
tests/test_suite_shared.py::test_focus_is_empty_before_my_booking FAILED  [100%]

=================================== FAILURES ===================================
_______________________ test_focus_is_empty_before_my_booking _______________________

    def test_focus_is_empty_before_my_booking():
>       assert len(REPO.find_by_room("focus")) == 0     # assumes the DB is clean
E       AssertionError: assert 1 == 0
E        +  where 1 = len([Booking(id='bk-m-ana-...', room_id='focus', ...)])

The second test fails on its first line: it expected 0 bookings in Focus and found 1 —the one the first test left—. It didn't fail because of a bug in its logic; it failed because it inherited a booking it didn't create. And the proof that the fault is the shared state and not the test: if you run that same test alone, it passes.

python3 -m pytest tests/test_suite_shared.py::test_focus_is_empty_before_my_booking -v

tests/test_suite_shared.py::test_focus_is_empty_before_my_booking PASSED  [100%]

The same test, two verdicts. Green alone, red in the suite. That's textbook flakiness, and its cause is real state that wasn't isolated.

Now the destination. The same suite, with one change: instead of a shared REPO, a fixture that gives each test a new database and destroys it when done. Nothing else changes in the assertions.

# tests/test_suite_isolated.py — the same suite, ISOLATED with a fixture
import sqlite3
import pytest
from datetime import datetime
from reservo.calendar import Calendar
from reservo.doubles import FixedClock, SpyEmailSender, StubPaymentGateway
from reservo.models import Member, Room
from reservo.services import BookingService
from reservo.sqlite_repo import SqliteBookingRepository

FOCUS = Room("focus", "Focus", 4, 2500)
ANA = Member("m-ana", "Ana", "pro")

@pytest.fixture
def repo():
    conn = sqlite3.connect(":memory:")     # new, empty DB per test
    yield SqliteBookingRepository(conn)
    conn.close()                           # destroyed when the test ends

def service(repo):
    return BookingService(Calendar(), FixedClock(datetime(2026, 3, 1, 9)),
                          StubPaymentGateway(True), SpyEmailSender(), repo)

def test_book_creates_one_focus_booking(repo):
    service(repo).book(FOCUS, ANA, datetime(2026, 3, 10, 9), datetime(2026, 3, 10, 12))
    assert len(repo.find_by_room("focus")) == 1

def test_focus_is_empty_before_my_booking(repo):
    assert len(repo.find_by_room("focus")) == 0     # now it IS true
    service(repo).book(FOCUS, ANA, datetime(2026, 3, 11, 9), datetime(2026, 3, 11, 12))
    assert len(repo.find_by_room("focus")) == 1

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

python3 -m pytest tests/test_suite_isolated.py -v

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 2 items

tests/test_suite_isolated.py::test_book_creates_one_focus_booking PASSED  [ 50%]
tests/test_suite_isolated.py::test_focus_is_empty_before_my_booking PASSED [100%]

============================== 2 passed in 0.01s ===============================

Green, both of them. The only difference from the broken suite is that each test receives its own database, fresh, through the repo fixture, and that database dies when the test ends. The assertion find_by_room("focus") == 0 is now true because the table that test sees really is empty: it's not the previous test's table, it's a new one. That's the whole module, summarized in two suites: going from a shared, dirty workbench to a clean bench per experiment. Everything that follows is understanding why real state contaminates, what the techniques for isolating it are, and how to choose between them.

The module's map: the eight lessons

It's worth seeing the journey, because each lesson installs a piece of the isolation.

Lesson	Topic	The idea in one sentence
1	The state that doesn't go away on its own (this one)	The fake is born empty in each test; the real database persists and contaminates
2	Why real state contaminates	The fake's `dict` dies with the test; the SQLite table survives, and the result depends on the order
3	The transaction rollback	Each test runs in a transaction that's reverted at the end; the DB stays clean without recreating it
4	Temporary DB fixtures with `yield`	A fixture creates a fresh database per test and destroys it in the teardown
5	`:memory:` versus a temporary file	Memory is fast and isolated per connection, or a real, persistent file but dozens of times slower
6	Seeding integration data	A known, minimal, spoken-aloud initial state, before each test
7	Independent and repeatable	Each test starts from a known state and gives the same result in any order
8	Mini-project	Take a contaminated suite and make it isolated and repeatable in any order

If you understand why real state contaminates and master the two isolation techniques —rollback and fixture—, know how to choose the resource and seed just enough, you have solved the problem that generates the most flakiness in integration. Lessons 3 to 6 are the tools; lesson 7 the principle that justifies them; lesson 8, the practice that brings them together.

What this module does NOT cover (the border)

It's worth marking the limits, because there are neighboring topics that look like they belong here.

The Builder pattern for manufacturing data is the doubles guide. Here you're going to seed test data, but minimally and explicitly: a few known bookings written by hand. When the test data becomes complex —many fields, many variants, factories with defaults and overrides—, the pattern that solves it elegantly is the Builder, and that's the topic of module 7 of test-doubles-and-test-data-guide, not this one. In this guide, the data is the means; the end is the isolation of the real resource. When the seeding starts to call for a real factory, we link you there.

The capstone is module 8. This module closes the integration technique —data and isolation— but doesn't yet bring together the guide's two disciplines. Module 8 is the integrative project: a consumer-driven contract verified against the fake and the real one, plus an integration test of BookingService with the real repository, turned in together. Here we leave the integration tests impeccable —isolated, repeatable— so the capstone can use them with confidence.

The web framework isn't this guide's. Reservo integrates in-process: BookingService plus a real SqliteBookingRepository. Isolating the data of a complete web app —with its database, its migrations, its framework fixtures— is testing-backend-applications-guide. The isolation principles you learn here transfer, but a framework's specific tools you'll see there.

Common mistakes

Believing "I used in-memory SQLite" already isolates. What happens: someone uses sqlite3.connect(":memory:") and concludes that, being in memory, each test starts clean. Why it happens: "in memory" sounds ephemeral. How to detect it: a :memory: database lives while the connection lives; if you share the connection between tests —like the example's REPO—, the database is the same for all and the state is inherited just like on disk. How to fix it: what isolates isn't :memory: versus a file, but a new database per test (a new connection per test, or a rollback between tests). You choose the resource by speed and by whether you want to touch the disk; the isolation is a separate decision, that of lessons 3 and 4.

Confusing a bug with an isolation problem (and vice versa). What happens: an integration test fails in the suite, and someone starts debugging the test's logic, which is perfect. Why it happens: the failure appears inside the test, so it seems the test's. How to detect it: run the test alone. If it passes alone and fails in the suite, it's not a test bug: it's state contamination. If it fails in both, it's a real bug. How to fix it: that test —alone versus suite— is the first diagnosis of every integration flake; you learned it in the test-failure-diagnosis module and it's the reflex this module asks you to install.

Cleaning "by hand" with a DELETE at the end of each test and trusting it's enough. What happens: someone puts conn.execute("DELETE FROM bookings") at the end of each test to clean up. Why it happens: it's intuitive and sometimes works. How to detect it: if a test fails midway, the DELETE line at the end doesn't run, and the next test inherits the garbage anyway —the manual teardown doesn't run if the test blows up before—. How to fix it: the isolation has to run no matter what, and that's what fixtures with yield are for (lesson 4) and the rollback in the teardown (lesson 3), which pytest runs even if the test fails. The cleanup can't depend on the test reaching the end alive.

Exercises

Exercise 1 — Why does the fake never suffer this? Explain, in your own words, why a suite of unit tests that uses FakeBookingRepository() never has the contamination problem we just saw, even if it has hundreds of tests that save bookings. Be precise about where the state lives in each case.

See solution

Because the fake's state lives in a dict that's an attribute of the repository's instance, and each test creates its own instance with FakeBookingRepository(). When the test ends, the variable that pointed to that instance goes out of scope, Python collects the object, and the dict —with all the bookings the test saved— goes with it. The next test creates another instance, with another empty dict, that shares nothing with the previous one. The isolation is automatic because the state's lifecycle is tied to the lifecycle of a Python object, and that object dies with the test.

The real repository breaks exactly that chain: its state doesn't live in a Python object that dies with the test, but in a SQLite table inside a connection. If you share the connection between tests, the table —and its rows— survives each test, because nothing erases it: save's commit made it permanent. That's why the isolation, which was free with the fake, in integration you have to provoke on purpose. That "on purpose" is the whole module.

Exercise 2 — Predict the effect of the order. In the example's shared suite, reverse the order of the two tests: put test_focus_is_empty_before_my_booking first and test_book_creates_one_focus_booking second. Without running anything, predict which passes and which fails, and why.

See solution

With the order reversed, test_focus_is_empty_before_my_booking runs first, when the table really is empty (nobody has written yet). Its first line, assert len(REPO.find_by_room("focus")) == 0, is now true, so it passes: it books one and ends, leaving one Focus booking in the shared table.

Then test_book_creates_one_focus_booking runs, which books again and expects len(...) == 1. But the table already had the booking the first test left, so now there are two, and assert len(...) == 1 fails with assert 2 == 1.

That is: by reversing the order, the test that was failing passes and the one that was passing fails. The same set of tests, the same code, opposite results depending on the order. That order dependence is the unmistakable signature of shared state that isn't isolated, and it's exactly what lesson 7 eliminates: a well-isolated suite gives the same verdict in any order.

Exercise 3 — The DELETE that didn't save. A colleague, to fix the shared suite, adds at the end of each test the line REPO._conn.execute("DELETE FROM bookings"); REPO._conn.commit(). The suite passes. Explain in what concrete situation this solution would stop working and why fixtures with yield don't have that problem.

See solution

The DELETE at the end works as long as each test reaches that last line alive. The problem appears when a test fails before reaching the DELETE: an intermediate assertion blows up, the test's execution is cut there, and the cleanup line at the end never runs. The table is left with that test's rows, and the next inherits them —exactly the problem it was trying to avoid—. So a single test that fails midway reintroduces the contamination for all that follow, and on top of that intermittently and confusingly.

Fixtures with yield don't have that problem because pytest runs the teardown —what comes after the yield— even if the test fails. The cleanup isn't just another line of the test body that can be skipped; it's a responsibility of the fixture that pytest guarantees no matter what. You'll see it in detail in lesson 4; the key idea is that the isolation can't depend on the test finishing well, because tests, sometimes, finish badly —and it's exactly when they fail that you most need the cleanup to run anyway—.

Summary and next step

In this lesson you saw the problem that names the module: real state persists between tests and contaminates them, something the unit test with doubles never suffers because the fake's dict dies with the test, while the SQLite table survives. With the shared workbench you understood that a contaminated test's result depends not on its logic but on what ran before, and you confirmed it with real output: a shared suite where a test passes alone and fails in the suite, and the same suite isolated with a fixture, green. You have the map of the eight lessons and the border with the Builder pattern (doubles guide) and the capstone (module 8).

Before moving on you should be able to: explain where the state lives in the fake and in the real repository, and why that makes one isolate itself and the other not; recognize the symptom of the integration flake —passes alone, fails in the suite, changes with the order—; and rule out the DELETE at the end as reliable isolation.

What comes next is understanding the mechanism in depth. In lesson 2 we're going to dissect why shared real state contaminates —the dict's lifecycle versus the table's—, see the contamination in slow motion with real output, and connect it explicitly with failure diagnosis: why this is the most common integration flake and how its signature —order-dependent— gives it away. Understanding the cause precisely is what lets you choose the right isolation technique in the lessons to come.

Resources

pytest documentation — Fixtures — the reference for the module's central tool: the fixture that creates a real resource and destroys it, which you saw in passing here and which lesson 4 develops in depth.
sqlite3 — DB-API for SQLite (Python documentation) — the reference for the real resource whose state persists; in particular, how a connection keeps its tables open while it lives, the technical root of the contamination.
test-doubles-and-test-data-guide — the sister guide where the Builder pattern for manufacturing complex test data lives, the border this module respects by seeding only the minimal and explicit.
Martin Fowler — Eradicating Non-Determinism in Tests — the frame that names the shared-state problem as one of the main causes of non-deterministic tests, the context of why isolation matters so much in integration.

← Previous8. Mini-project: Reservo's three boundaries Next →2. Why shared real state contaminates