Module 6: Real Boundaries Db Files Http

7. Fast and deterministic: when to touch the real thing and when to double

Next →

Description

You've now crossed the three boundaries: the database with its transaction, the file with tmp_path, the HTTP with http.server. In each one, without fully naming it, you applied two disciplines that now need distilling, because they're the ones that separate an integration suite people want to run from one they avoid like the plague. The first is the technique: how to make a boundary test —that touches disk, network, or a database engine— just as fast and reliable as a unit test. You achieved it with three repeated tools: a :memory: database or a tmp_path instead of a shared resource, a server in a thread with an ephemeral port instead of an external one, and a timeout that bounds the wait. Those three keys turn resources that are slow and non-deterministic by nature into millisecond, isolated, and repeatable tests.

The second discipline is the criterion, and it's the rule that governs the whole module —in fact, all of integration—. In any boundary test there's one boundary you're testing and several that are just passing through. The rule: touch the real thing at the boundary you're testing; double what you don't control or is slow on the path you're not testing. If you test that book persists in the database, the repository goes real (it's the seam under test) and the payment, the clock, and the email go doubled (they're passing-through collaborators, slow or external). If you test that book charges correctly over HTTP, the gateway goes real and the repository goes doubled. The same action —book— is tested with different real pieces depending on which boundary you're examining. This lesson makes that rule explicit, justifies it with the two extremes it avoids, and applies it boundary by boundary to Reservo with real output.

Connection to the module: this lesson is the synthesis. It gathers the three techniques you used in lessons 3 to 6 and names them as principles; and it formulates the decision rule that was implicit in every "what do I double and what do I leave real". It's also the bridge to the guide's module 7: the technique of ephemeral, self-cleaning resources you see here as "so it's fast and deterministic" becomes there the central topic —the systematic isolation of data and resources—. The border with that module is respected: here we give the rule and the three keys; the art of isolating with fixtures that create and destroy resources, and rollback as an isolation technique, is module 7. Here you decide what to touch real; there you learn to isolate the real thing you touch.

Analogy: the flashlight's beam

Imagine you're looking for something in a dark room with a flashlight. You don't light up the whole room equally —you can't, and you don't need to—: you point the beam at the corner you're checking and see it clearly, while the rest of the room stays in shadow, barely hinted at. When you finish with that corner and move to another, you move the beam: now that one is lit and the previous returns to shadow. The flashlight is useful precisely because it concentrates the light: if you tried to light up the whole room at the same intensity, you'd need an enormous lamp, waste a lot of energy, and on top of that the glare of everything at once would blind you and you'd see nothing clearly.

A boundary test is that flashlight. The boundary you're testing is the lit corner —real, sharp, examined up close—; everything else —the passing-through collaborators— stays in shadow, doubled, barely enough for the flow to run. When you switch boundaries, you move the beam: the one that was real gets doubled and the new one lights up. Lighting up the whole room —leaving everything real— is the mistake we'll talk about: expensive, slow, and so full of things failing at once that you don't know what you're testing. Leaving everything dark —doubling everything— is the other mistake: you see no boundary, you test no joint. The art is pointing the beam at one boundary, leaving the rest in shadow, and knowing exactly which corner you're looking at.

The three keys: fast and deterministic

Before the rule, let's consolidate the technique. A boundary test is, by nature, slower and less deterministic than a unit test —it touches resources you don't control—. Three tools, which you already used, return it to the terrain of the fast and reliable:

  • In-memory or temporary resources, never shared. For the database, sqlite3.connect(":memory:"): real SQLite, without the disk's cost, and isolated because each connection is born empty. For files, tmp_path: a real file in a directory unique per test, that pytest deletes itself. The alternative —a shared database, a file with a fixed name— is slow (disk, network) and fragile (one test leaves data for the next). The key: real but yours and ephemeral.
  • The server in a thread, on an ephemeral port. For HTTP, an http.server spun up in a daemon thread with port 0. It's a real server, local, that starts in microseconds and clashes with nothing. The alternative —pointing at an external server, or a fixed port— is slow, non-deterministic (depends on the other being alive and the network responding), and prone to collisions. The key: server real but local and ephemeral.
  • Timeouts that bound the wait. Every call that crosses the network carries a timeout, so that a resource that hangs doesn't freeze the suite. The alternative —a call without a limit— turns any stumble of the other end into a test hung forever. The key: never wait indefinitely at a boundary.

These three keys have something in common: they keep the resource real —you exercise the serialization, the transaction, the real HTTP protocol— but under your control, local and disposable. Real without being production; that's the balance of a good boundary test.

The decision rule

Now the criterion, in a sentence worth memorizing:

Touch the real thing at the boundary you're testing; double what you don't control, is slow, or is external on the path you're not testing.

Let's break it down. "The boundary you're testing" is the seam whose behavior you want to truly verify —its serialization, its transaction, its protocol—: that goes real, because doubling it would be doubling exactly what you want to examine. "The path you're not testing" is the other collaborators the flow touches in passing: those go doubled, because their reality adds nothing to what you test and does bring cost (slowness) or risk (non-determinism, external effects like charging a card). The rule is the flashlight turned into a criterion: light up one boundary, leave the rest in shadow.

Notice the practical consequence: the same action is tested with different configurations depending on the boundary under examination. book touches two boundaries —it persists in the database and charges via the gateway—. To test persistence, the repository goes real and the gateway doubled. To test the charge, the gateway goes real and the repository doubled. There's no single "book integration test"; there's one for each boundary you want to light up, each with its beam in a different place. Let's see it with code.

Worked example: the same book, two beams

Two book tests. In the first, the boundary under examination is the database: the repository goes real (:memory:) and everything else —payment, clock, email— goes doubled. In the second, the boundary is HTTP: the gateway goes real (against an http.server) and the repository goes doubled (FakeBookingRepository). The beam moves; the rule shows.

# tests/test_real_vs_double.py — the same book, the beam on a different boundary

# Boundary under test: DATABASE.
# The repo is REAL (:memory:); payment/clock/email are DOUBLES.
def test_db_boundary_real_everything_else_doubled():
    repo = SqliteBookingRepository(sqlite3.connect(":memory:"))   # REAL
    service = BookingService(
        Calendar(),
        FixedClock(CLOCK),              # double: the clock isn't the boundary here
        StubPaymentGateway(ok=True),    # double: the payment isn't the boundary here
        SpyEmailSender(),               # double
        repo,
    )
    booking = service.book(FOCUS, ANA, START, END)
    assert repo.get(booking.id).price_cents == 6000


# Boundary under test: HTTP.
# The gateway is REAL (http.server); the repo is a DOUBLE.
def test_http_boundary_real_repo_doubled(gateway_url):
    repo = FakeBookingRepository()                     # double: the DB isn't the boundary here
    service = BookingService(
        Calendar(),
        FixedClock(CLOCK),
        HttpPaymentGateway(gateway_url),               # REAL: the boundary under test
        SpyEmailSender(),
        repo,
    )
    booking = service.book(FOCUS, ANA, START, END)
    assert repo.get(booking.id).price_cents == 6000

Both do book(FOCUS, ANA, START, END) and both verify price_cents == 6000, but they light up different boundaries. The first puts the real SqliteBookingRepository and doubles the gateway with a StubPaymentGateway: it tests that the booking truly persists crossing the database boundary, without paying the cost or risk of a real HTTP call that adds nothing here. The second puts the real HttpPaymentGateway against the http.server and doubles the repository with a FakeBookingRepository: it tests that the charge crosses HTTP for real, without dragging in the disk's cost that doesn't matter here. Each beam where it belongs.

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

python3 -m pytest tests/test_real_vs_double.py -v
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 2 items

tests/test_real_vs_double.py::test_db_boundary_real_everything_else_doubled PASSED [ 50%]
tests/test_real_vs_double.py::test_http_boundary_real_repo_doubled PASSED [100%]

============================== 2 passed in 0.54s ===============================

Two greens that show the rule in action. The same book tested twice, with the beam on the database first and on HTTP after. Neither left everything real —that would be slow and fragile— or everything doubled —that would test no boundary—. Each lit up exactly one boundary and doubled the rest. That's a well-focused integration test: sharp where it matters, in shadow where it doesn't.

Why not everything real, and why not everything doubled

The rule is best understood through the two extremes it rules out.

Everything real is an end-to-end in disguise. If you leave the repository real and the gateway and the clock and the email, your "integration test" charges real cards, sends real emails, depends on the network, and takes seconds. When it fails, was it the repository, the gateway, the network, the email? You don't know: you lit up the whole room and everything could have broken at once. That's an end-to-end test —legitimate in its place, at the peak of the pyramid, few and slow—, but disguising it as a boundary test gives you the worst of both: slow, fragile, and without telling you which seam failed. Besides, some collaborators you can't leave real in a test: a production PaymentGateway would charge real money.

Everything doubled tests no boundary. If you double the repository and the gateway and everything else, your test runs extremely fast and is deterministic... but it crossed no boundary. It didn't exercise the database's serialization, or the HTTP protocol, or the write to a file. It's a unit test —good for BookingService's orchestration logic, useless for the joints with the real thing—. Remember module 1: a green unit test with everything doubled can hide a broken integration, because no double exercises the boundary where the bug lives. Doubling everything is turning off the flashlight.

The rule lives in the middle: exactly one real boundary (the one you test) and the rest doubled (what you don't test). That way you get the best of both worlds: you truly test the joint that matters to you —with its serialization, its transaction, its protocol— and keep the test fast, deterministic, and without external effects, because everything else is in shadow. And since each boundary has its own focused test, together they cover the system without any expensive test that tries everything at once.

Common mistakes

Doubling the boundary you claim to be testing. What happens: someone titles a test "repository integration" but uses the FakeBookingRepository. Why it happens: the fake is more convenient and fast, and habit weighs. How to detect it: ask yourself which boundary the test claims to examine, and whether that piece is real. If it's a double, you're not testing that boundary —you pointed the beam elsewhere—. How to fix it: the boundary under test goes real, always; that's the whole point. Double the passing-through collaborators, never the seam you examine.

Leaving real a collaborator that isn't the boundary under test. What happens: to "make it more realistic", someone leaves the PaymentGateway real in a test that examines database persistence. Why it happens: "more real is always better" is a seductive intuition. How to detect it: if your test of one boundary touches another real boundary you're not examining —it charges for real, sends emails, depends on the network—, you inherited its cost and risk without gaining anything. How to fix it: only the boundary under test goes real; the rest is doubled, even if you "could" leave it real. More real pieces isn't more rigorous: it's slower, more fragile, and harder to diagnose when it fails.

Testing a boundary with a shared resource or without a timeout. What happens: someone leaves the correct boundary real, but points at a shared database, a fixed file, or a server without a timeout. Why it happens: they focus on what to leave real and forget the three keys of how to make it fast and deterministic. How to detect it: if the test, correct in its choice of real pieces, is still slow, intermittent, or order-dependent, it fails in the technique, not in the criterion. How to fix it: apply the three keys —:memory:/tmp_path instead of shared, server in a thread with an ephemeral port, timeout on every network call—. The rule tells you what to touch real; the three keys, how to touch it without inheriting its slowness.

Exercises

Exercise 1 — Apply the rule. For each test goal, say which Reservo piece goes real and which go doubled: (a) verify that a booking persists correctly in SQLite; (b) verify that book charges the correct amount over HTTP; (c) verify that the CSV export writes the correct bookings; (d) verify the logic that cancel refunds 6000 at 72 h before the start.

See solution
  • (a) Persistence in SQLite. Real: the SqliteBookingRepository (the boundary under test). Doubled: the PaymentGateway (stub), the clock (fixed), the email (spy). It's test_db_boundary_real_everything_else_doubled.
  • (b) Charge over HTTP. Real: the HttpPaymentGateway against an http.server (the boundary under test). Doubled: the repository (fake), the clock, the email. It's test_http_boundary_real_repo_doubled.
  • (c) CSV export. Real: the file, with tmp_path (the boundary under test: export_bookings/import_bookings). Doubled: the repository the bookings come from can be a FakeBookingRepository (it isn't the boundary; it just provides the data). The payment, clock, and email don't even intervene if you build the bookings directly.
  • (d) refund_cents logic. No real boundary piece: it's pure logic. It's tested with a direct unit test (refund_cents(booking, 6000, now) == 6000), without a repository, without a gateway, without anything real. It crosses no boundary, so the "what to double" rule doesn't even apply: it's pure domain.

The rule you're sharpening: identify the boundary under test, make it real, double the rest. And if there's no boundary (pure logic), it's a unit test with nothing to double.

Exercise 2 — The test that leaves everything real. A colleague writes a "definitive integration test" of book with the real SqliteBookingRepository in a file, the real HttpPaymentGateway against the staging payment gateway, and a real clock. List three concrete problems of that test and rewrite, in words, how you'd split it according to the rule.

See solution

Three concrete problems of the "everything real" test:

  1. External effects and real cost. Pointing at the staging payment gateway can move real money or leave records in a shared system, and it depends on staging being alive and accessible. A test shouldn't charge cards.
  2. Slow and non-deterministic. The on-disk file (with its commit that synchronizes) plus the HTTP call over the real network add hundreds of milliseconds or seconds, and the staging network can lag or fail for reasons unrelated to your code. The test will be slow and intermittent.
  3. Impossible diagnosis. When it fails, you won't know if it was the repository, the gateway, the staging network, or the clock: you lit up the whole room, and anything could have broken. A red like that doesn't point to the guilty seam.

How to split it according to the rule: two focused tests, plus a unit test. One test with the beam on the database —real SqliteBookingRepository in :memory: (or tmp_path if you really test on-disk persistence), gateway/clock/email doubled— that verifies the booking persists. Another with the beam on HTTP —real HttpPaymentGateway against a local fake http.server, doubled repository— that verifies the charge crosses the network boundary correctly. And a unit test for the price and refund logic, without anything real. That way each test is fast, deterministic, without external effects, and when one fails it tells you exactly which boundary broke. What the colleague called "definitive" decomposes into sharp pieces; the "everything real" at most fits as a single end-to-end at the peak of the pyramid, not as the test of each boundary.

Exercise 3 — Real but badly set up. A test examines the HTTP boundary with the real HttpPaymentGateway —a good choice of real piece— but points it at a server spun up on the fixed port 8080, without a timeout, and that the test doesn't shut down when done. The choice of what's real is correct; what fails, and how do you fix it with the three keys?

See solution

The choice of the real piece is correct (the HTTP gateway is the boundary under test, so it goes real), but the technique of how to set it up is wrong in the three keys:

  1. Fixed port 8080. It clashes with anything else using that port —another test, a previous run, another app—, producing intermittent Address already in use failures. Fix: ephemeral port (0) and reading the one the system picked with server.server_address[1].
  2. No timeout. If the server hangs, the test waits forever and blocks the suite. Fix: pass a timeout to the client (as HttpPaymentGateway does) and, ideally, test that it cuts off with a slow server.
  3. Doesn't shut down the server. The thread and the port stay occupied between tests, dirtying later runs. Fix: wrap the startup in a fixture with try/finally (or yield) that calls shutdown(), join(), and server_close() when done, and run the server in a daemon thread.

The moral: the rule ("what to touch real") and the three keys ("how to touch it") are independent, and you need both. Here the criterion got it right —the gateway goes real— but the technique failed, and the result is a test correct in intention but slow, colliding, and fragile. Applying the three keys returns it to the terrain of the fast and deterministic without changing which boundary it examines.

Summary and next step

In this lesson you distilled the two disciplines that make testing at the boundaries viable. The technique —the three keys so a boundary test is fast and deterministic—: in-memory or temporary resources instead of shared ones (:memory:, tmp_path), server in a thread with an ephemeral port, and timeouts that bound the wait; all keep the resource real but yours, local and disposable. And the criterion —the whole module's decision rule—: touch the real thing at the boundary you test; double what you don't control, is slow, or is external on the path you don't test. With the flashlight's beam you saw why: lighting up one boundary and leaving the rest in shadow avoids the two extremes —the "everything real" that's a slow and blind end-to-end, and the "everything doubled" that tests no joint—. And you checked it with the same book tested twice, the beam on the database and then on HTTP, each real where it matters and doubled where it doesn't.

Before moving on you should be able to: name the three keys of a fast and deterministic boundary test; state and apply the rule of what to touch real and what to double; and decompose an "everything real" test into per-boundary focused tests.

What comes next is the module's close, where you bring the three boundaries together in a single flow. In lesson 8, the mini-project: a test that crosses the three real boundaries of Reservo —charge over HTTP, persist in SQLite, export to a file— and verifies them all, green, with the justification of what was left real and what doubled in each. It's this lesson's rule put to work in a complete flow, and the door to module 7, where you'll learn to isolate the data and resources these boundary tests leave everywhere.

Resources

← Previous6. The HTTP boundary with a real `http.server`Next →8. Mini-project: Reservo's three boundaries