Module 1: From Units To Integration

5. A green unit test can hide a broken integration

Next →

Description

We arrive at the heart of the guide. Everything before —the definitions, the pyramid, the seams— was to arrive prepared for a single sentence: a green unit test can hide a broken integration. Not "sometimes the suite has holes", not "you have to test more": something more uncomfortable and more precise. Your unit test can pass, entirely rightly, over code that in production is broken —and the test has no way of knowing, because the problem lives exactly where the test stopped looking—.

The cause is always the same: the unit test closes a seam with a double, and the double diverges from the real provider at that seam. The test verifies that your logic is correct assuming the double faithfully represents the real thing. If that assumption fails, the green is true for the double and false for the world. In lesson 1 you saw it with book; here we're going to isolate it down to its purest form —a single seam, a single operation, save followed by get— so you see the mechanism without noise. The fake and the real SqliteBookingRepository receive the same booking, are asked for the same booking back, and are asked the same question. One passes. The other fails. And the difference isn't a bug in your logic: it's a divergence at the seam that the double, by its very nature, couldn't reveal.

Connection to the module: this lesson is the distilled why of the entire module. Lesson 2 gave you the definitions, lesson 3 the proportion, lesson 4 the place (the seam); here you see the concrete failure that justifies integration's existence. We don't close the gap yet —the systematic how is the contract of modules 3 and 4, and integration in depth is 5 through 7—. What this lesson installs is the awareness: never again will you look at a green suite of pure unit tests and conclude, just like that, that the system works. You'll ask: which seams are doubled, and how sure am I that my doubles don't lie right there?

Analogy: the stunt double who rehearsed with a safety net pool

Think of an action movie with a dangerous scene: an actor must fall from a rooftop. The actor doesn't do it; a stunt double does, and before shooting, the double rehearses. But they rehearse falling onto a springy net pool set up on the practice set, not onto the cardboard boxes that will be there on the day of the real shoot. In the rehearsals, the fall comes out perfect: they land clean, get up, smile. A hundred rehearsals, a hundred successes, all green. On shoot day, they fall onto the real boxes —which compress differently, shift, have a harder edge— and twist their ankle. The hundred successful rehearsals didn't lie about the rehearsal: the double really does fall well onto the net. They lied about what mattered —the fall onto the real thing— because they rehearsed against something different from what they'd face.

Your unit test is that rehearsal, and the test double is the net pool. The test verifies, quite honestly, that your code "falls well" onto the double: book saves the booking in the dict, reads it back, and the datetime is intact. A hundred runs, a hundred greens. But production doesn't use the dict; it uses SQLite, which "cushions differently": it serializes the datetime to text and returns it as a str. The code that fell perfectly onto the net twists its ankle on the real boxes. The green didn't lie about the rehearsal; it lied about the premiere, because you rehearsed against a net pool that doesn't behave like the real ground. An integration test is rehearsing the fall onto the real boxes, even if only once, before the day of the shoot.

Worked example: the divergence, in its purest form

Let's reduce the problem to its minimum. No BookingService, no charges, no emails: just the repository, one booking, and the most basic operation there is —save it and read it back—. The two tests do exactly the same thing, with exactly the same assertions; the only thing that changes is which provider is behind the seam.

# tests/test_repo_roundtrip.py — the same round-trip, two providers
import sqlite3
from datetime import datetime

from reservo.doubles import FakeBookingRepository
from reservo.models import Booking
from reservo.sqlite_repo import SqliteBookingRepository

START = datetime(2026, 3, 10, 9)
END = datetime(2026, 3, 10, 12)


def a_booking():
    return Booking(id="bk-1", room_id="focus", member_id="m-ana",
                   start=START, end=END, status="confirmed", price_cents=6000)


# unit: against the double
def test_fake_roundtrips_a_saved_booking():
    repo = FakeBookingRepository()
    repo.save(a_booking())

    got = repo.get("bk-1")
    assert got.price_cents == 6000
    assert got.start == START            # datetime == datetime


# integration: against the real repository
def test_sqlite_roundtrips_a_saved_booking():
    repo = SqliteBookingRepository(sqlite3.connect(":memory:"))
    repo.save(a_booking())

    got = repo.get("bk-1")
    assert got.price_cents == 6000
    assert got.start == START            # <-- str != datetime

Read them carefully: they're twins. Same starting booking, same save, same get("bk-1"), same two assertions. If "save and read" were really the same in both providers, the two tests would give the same result. Let's run them and see.

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

python3 -m pytest tests/test_repo_roundtrip.py -v
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 2 items

tests/test_repo_roundtrip.py::test_fake_roundtrips_a_saved_booking PASSED [ 50%]
tests/test_repo_roundtrip.py::test_sqlite_roundtrips_a_saved_booking FAILED [100%]

=================================== FAILURES ===================================
____________________ test_sqlite_roundtrips_a_saved_booking ____________________

    def test_sqlite_roundtrips_a_saved_booking():
        repo = SqliteBookingRepository(sqlite3.connect(":memory:"))
        repo.save(a_booking())

        got = repo.get("bk-1")
        assert got.price_cents == 6000
>       assert got.start == START            # <-- str != datetime
E       AssertionError: assert '2026-03-10T09:00:00' == datetime.datetime(2026, 3, 10, 9, 0)
E        +  where '2026-03-10T09:00:00' = Booking(id='bk-1', ..., start='2026-03-10T09:00:00', ...).start

tests/test_repo_roundtrip.py:35: AssertionError
========================= 1 failed, 1 passed in 0.02s ==========================

There's the mechanism, bare. The test against the fake passes; the test against the real one fails, on the datetime line. The same question, two answers: assert got.start == START is True with the fake and False with SQLite. And notice the assertion that didn't fail: got.price_cents == 6000 passes in both. The divergence isn't in the whole object —it's in one field, the datetime, precisely the one the seam has to serialize—. The fake never serializes: it stores the whole object in a dict, so get returns you the very same object, with its datetime intact. SQLite has to put the data into a table of text and numbers; the datetime doesn't fit as such, it's stored as ISO text, and comes back as str. The fake and the real one don't do the same thing in save/get, and that difference is invisible to any test that only uses the fake.

Anatomy of the deception: why the green is true and deceptive

The most important thing to understand is that the green unit test is not wrong. test_fake_roundtrips_a_saved_booking passes because it's true: the FakeBookingRepository really does return the datetime intact. The test has no bug; it does exactly what it promises —verify the round-trip against the fake—. The deception isn't in the test; it's in the conclusion we draw from it. We see the green and think "saving and reading a booking works". What the green really says is narrower: "saving and reading a booking works with this double". The leap from the second to the first —from "works with the double" to "works"— is the deception, and we make it without realizing because the double resembles the real thing so much that we forget it isn't.

Let's break down the chain that produces the false sense of security:

  1. The double embodies an assumption. When you wrote FakeBookingRepository.get to return the saved object as-is, you encoded an assumption: "reading a booking returns it identical to how it was saved". It's reasonable. And for SQLite, it's false.
  2. The unit test inherits the assumption. By testing against the fake, the test doesn't verify that assumption: it takes it for granted. It can't do otherwise —the fake is the assumption made object—. A test can't catch an error that lives in its own premise.
  3. The conclusion jumps the seam. We interpret the green as a claim about the real system, when it's only a claim about the doubled system. The seam the double closed is exactly the seam the test stays silent about.

That's why no amount of unit tests closes this gap. You could have a thousand round-trip tests against the fake, with a thousand different bookings, and all thousand would pass, and all thousand would share the same false premise. The error doesn't dilute with volume because it isn't an error of this test: it's a limit of all the tests that double this seam. The only way to see the divergence is to stop doubling —cross the seam with the real piece—, and that is, by definition, an integration test.

The lesson you take away: suspect the seam, not the green

Don't draw the pessimistic conclusion that "unit tests lie" or "doubles are bad". Unit tests and doubles are indispensable —they're the wide base of the pyramid, your speed and your precision—. The correct conclusion is more surgical: a unit test green is a claim about the double, not about the real thing, and for every seam you double there's an open question that green doesn't answer: "does my double behave like the real provider at this seam?".

That question has two possible answers depending on the seam's risk. For cheap seams without a boundary —the in-memory Calendar—, the answer is "the double and the real thing are the same thing, no divergence is possible", and nothing more is needed. For boundary seams with serialization, network, or disk —the repository, the payment, the email—, the honest answer is "I don't know until I verify it against the real thing", and there you need either an integration test (cross the seam with the real piece) or a contract (a battery both sides must pass). The guide will give you both tools. What this lesson leaves you is the reflex of asking yourself the question: seeing a green suite and, instead of relaxing, looking at the map of seams and pointing out which ones are still unverified against the world.

Common mistakes

Reading "green" as "the system works". What happens: the unit suite passes entirely and someone deploys with total confidence. Why it happens: the green is such a strong and satisfying signal that it erases the fine print of what was verified. How to detect it: ask yourself "does this green claim something about the real piece, or about a double?". If all your greens speak of doubles at the boundary seams, you have no evidence about the real system. How to fix it: translate each green to its narrow claim —"works with the double"— and notice which seams are left without a green that speaks of the real thing. Those are your pending integrations.

Blaming the unit test when the bug appears in production. What happens: the datetime blows up in production and someone says "the unit test was poorly written". Why it happens: if the test didn't catch the bug, it seems the test failed. How to detect it: review what the test tested. If it honestly tested the round-trip against the fake, it isn't wrong: it's incomplete at the suite level, not wrong at the test level. How to fix it: the fix isn't "rewrite the unit test", it's "add the missing integration test". The unit test will keep giving you speed and precision over the logic; the integration one will cover the seam that the former, by design, couldn't see.

"Fixing" the bug by making the fake lie just like the real one. What happens: someone, to make the unit test reflect the datetime bug, makes FakeBookingRepository.get also return a str. Why it happens: it seems that "aligning the fake with the real" resolves the divergence. How to detect it: if your fake now reproduces an undesirable behavior of the real one, you're baking the bug into your double instead of fixing it. How to fix it: the divergence is closed by deciding what the correct behavior is (probably get should return a datetime, fixing SqliteBookingRepository so it converts back) and making both providers fulfill it —that's what a contract forces—. The fake shouldn't imitate the real one's defects; both should fulfill a correct contract, and the contract is what guarantees it. It's exactly the job of modules 3 and 4.

Exercises

Exercise 1 — Translate the green. The test test_fake_roundtrips_a_saved_booking passes. Write the narrow claim that green really guarantees, and then the broad claim it would be a mistake to deduce from it. Explain what seam separates the two.

See solution
  • Narrow claim (what the green guarantees): "Saving a booking with FakeBookingRepository.save and reading it with FakeBookingRepository.get returns a booking with the same price_cents and the same start that was saved." It's true, and it's all the test tested.
  • Broad claim (the deduction mistake): "Saving and reading a booking preserves its fields" —flat out, for any repository, including the production one—. This is false (SQLite returns start as a str), and the green doesn't back it.

What separates the two is the repository seam: the narrow claim lives on the double's side; the broad one jumps to the real provider's side. The green only authorizes saying things about the side the test touched (the fake). Every time a green tempts you to say something about the other side of a doubled seam, you're making the leap this lesson warns about.

Exercise 2 — Another divergence at the same seam. Besides the datetime, imagine that Reservo starts saving bookings with a notes field that can be None. The FakeBookingRepository stores the object as-is, so notes=None comes back as None. But a colleague configures the SQLite column as notes TEXT NOT NULL. Without running anything, predict: which unit test would pass and which integration test would fail, and at exactly what moment?

See solution

The unit test would pass. A test that saves a booking with notes=None against the FakeBookingRepository and verifies got.notes is None passes green: the fake stores the whole object in the dict, so notes=None comes back as None, without the NOT NULL constraint even existing in the fake's world.

The integration test would fail, and sooner than you think: not on the assertion, but on the save. When trying to INSERT a row with notes = NULL into a NOT NULL column, SQLite rejects the operation and raises an IntegrityError. So the integration test blows up on the line repo.save(...), not on repo.get(...) or the assert. The divergence here isn't of the data's shape (like the datetime), but of rules that only the real provider imposes: the fake accepts any object, the real database enforces its schema.

The moral reinforces the lesson: the same seam (the repository) can diverge in many ways —types that change, constraints only the real thing imposes, ids that collide—, and a double, however convenient, knows none of those rules unless you teach them to it. Only the real piece brings them built-in.

Exercise 3 — The plan to never fall again. Your team learned its lesson with the datetime. A colleague proposes: "from now on, let's ban fakes and test everything against real SQLite, so a divergence never escapes us". Critique the proposal using what you know from lessons 2 and 3, and propose a better plan.

See solution

The proposal trades one problem for a worse one. Banning fakes and testing everything against real SQLite inverts the pyramid (lesson 3): the suite becomes slow (every test opens and touches a database), fragile (it fails because of the real state, not because of your logic), and ambiguous on failure (was it the logic or the database?). It also loses the unit test's precision: when something breaks, you'll no longer know if the bug is in your orchestration or in the persistence. It's the ice-cream cone, motivated by a legitimate scare but resolved in the wrong direction.

The better plan keeps the pyramid and closes the gap:

  1. Keep the wide base of unit tests with fakes for the logic: fast, precise, most of your confidence about book/cancel/prices/refunds.
  2. Add a band of integration tests that cross the repository seam with real SQLite —a few, not one per business rule— to verify exactly what the fake can't: serialization, schema constraints, the real round-trip.
  3. Write a repository contract (modules 3-4): a single battery of tests you run against the fake and against the real one. If the fake and the real one diverge at any point of the contract, the battery screams it —that's the tool that turns "let's hope the fake doesn't lie" into "the fake can't lie without a red test giving it away"—.

That way you have speed (unit), truth in the joints (integration), and a systematic guarantee that the double doesn't diverge (contract), without sacrificing the pyramid's shape.

Summary and next step

In this lesson you looked head-on at the problem that gives the entire guide its reason for being: a green unit test can hide a broken integration, because the double that closes the seam diverges from the real provider right there. You isolated it down to its purest form —save and get, a single field— and saw the mechanism without noise: the test against the fake passes (and isn't wrong: it's true for the fake), the test against SQLite fails on the datetime, and no amount of unit tests could have prevented it, because they all share the false premise the fake embodies. With the stunt double and their net pool, you understood that the green doesn't lie about the rehearsal; it lies about the premiere, because you rehearsed against something that isn't the real ground. And you take away the key reflex: suspect the seam, not the green; for every doubled seam, ask whether your double behaves like the real thing there.

Before moving on you should be able to: translate a unit test green to its narrow claim ("works with the double") and detect the illegitimate leap to the broad one; explain why the volume of unit tests doesn't close the gap; and rule out the two false fixes (blaming the unit test, or making the fake imitate the real one's defect).

You now know that the divergence exists and why it hides. What's missing to complete the map of the "why" is to sharpen integration's vocabulary —because not all integration is the same— and honestly weigh its cost. Lesson 6 classifies the types of integration (solitary vs sociable, narrow vs broad, incremental vs big-bang) so you can name exactly what you're testing; lesson 7 measures what it costs and decides when it's worth it.

Resources

← Previous4. Seams: where components connectNext →6. Types of integration