Module 8: Project Contract And Integration For Reservo

7. Catch a breaking change with the contract

Next →

Description

You have the three deliverables complete: the contract from both sides and the isolated end-to-end integration. This lesson doesn't add a fourth deliverable —it cashes in the value of the three—. Everything you built was aiming at this moment: using the contract to catch a breaking change before deploying it. A breaking change is a modification in a component that breaks a promise another component depends on. The scenario you'll see is the most common and the most dangerous, because the change looks harmless: it's not an obvious error, it's a reasonable "simplification" a colleague would make in good faith, that compiles, and that passes all the tests that don't exercise exactly that case.

The change is the one the guide anticipated since module 2: SqliteBookingRepository.get, instead of raising when the id doesn't exist, starts returning None. One line. To whoever writes it, it looks cleaner —"I return None if I don't find it, like many APIs"—. But the contract promised that get of an absent id raises, and there's a real consumer —BookingService.cancel— leaning on that promise. You're going to see the same bug at its two destinations: the crime, what happens in production without a contract —a distant, confusing, and late AttributeError—, and the arrest, the contract's surgical red that catches it on your machine, in two hundredths of a second, before merging. The difference between those two destinations is, in one sentence, why contract testing exists.

Connection to the module: this lesson is the demonstration that the delivery isn't bureaucracy, but an active safety net of the deploy. The contract you wrote in lesson 2 and verified in lessons 3 and 4 —green on both sides— is the one that goes red here as soon as someone breaks a promise. It's the golden payoff of the whole process: the one that turns "I wrote a contract" into "my contract just saved me a production incident". Lesson 8 formally collects the delivery and closes the guide; this one gives it its reason for being.

Analogy: the smoke detector

A house can have an impeccable electrical installation and still catch fire: a cable that frays over the years, an appliance that fails, a forgotten candle. You can't prevent there ever being a start of a fire —that's part of living in a house—. What you decide is when you find out. Without a smoke detector, you find out late and far away: when the fire has already spread, the smoke filled the rooms, and the damage is big and expensive. With a smoke detector, you find out early and close: a beep in the kitchen when the fire is barely starting, with plenty of time to put it out with a glass of water. The detector doesn't prevent the fire; it prevents the fire from growing unseen.

The contract is the deploy's smoke detector. It doesn't prevent a colleague from committing a breaking change —that's human and inevitable, like the cable that frays—. What it does is change when you find out. Without a contract, you find out in production: an AttributeError at three in the morning, far from the cause, with users affected —the advanced fire—. With a contract, you find out on running the battery before merging: a red with a first and last name on your machine, with the fire barely starting —the beep in the kitchen—. The bug is the same in both cases; the only thing that changes is whether you put it out with a glass of water or with the firefighters. This lesson is seeing the fire with and without a detector.

Worked example, part 1: the crime (production without a contract)

Let's first see what happens if the change is deployed without the contract catching it. The provider's team edits SqliteBookingRepository.get. Before, it honored the contract:

    def get(self, booking_id):
        row = self._conn.execute(...).fetchone()
        if row is None:
            raise KeyError(booking_id)   # honors the contract: absent id -> raises
        return _row_to_booking(row)

After the "harmless" change:

    def get(self, booking_id):
        row = self._conn.execute(...).fetchone()
        if row is None:
            return None          # BREAKING CHANGE: it used to raise KeyError
        return _row_to_booking(row)

A single different line. Now, in production, someone requests to cancel a booking that no longer exists —a double click, an old id, whatever—. BookingService.cancel trusts that get raises for an absent id; with the changed provider, get doesn't raise: it returns None, and cancel proceeds with booking = None. Let's run that scenario exactly as it would happen:

What to expect. On my machine (Python 3.14.0):

python3 demo_prod_crime.py     # cancel of an absent id, with the provider already changed
Traceback (most recent call last):
  File "demo_prod_crime.py", line 17, in <module>
    service.cancel("does-not-exist")
  File "reservo/services.py", line 36, in cancel
    refund = refund_cents(booking, booking.price_cents, now)
AttributeError: 'NoneType' object has no attribute 'price_cents'

Look at the error carefully, because its shape is the moral. It doesn't say "the repository broke its contract". It doesn't mention get, or the change, or the absent id. It says AttributeError: 'NoneType' object has no attribute 'price_cents', on the refund_cents line, inside cancel —two steps after the real problem—. The symptom is far from the cause: whoever debugs this in production will see cancel failing on price_cents and will lose time suspecting the refund logic, the calculation, everything except the repository's get that, silently, returned None instead of raising. That's the cost of a breaking change that reaches production: a confusing, distant, and late error, with a trace uphill to a cause the message doesn't name. It's the fire discovered when it already filled the house with smoke.

Worked example, part 2: the arrest (the contract catches it before the deploy)

Now let's rewind to the correct moment: the change is made in the provider, but before merging and deploying, we run the contract battery. It's the same battery of your deliverable —without touching a line of the tests—; the only thing that changed is the provider's implementation. Let's run the battery with the provider already changed:

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

python3 -m pytest tests/test_repository_contract.py -v
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 8 items

tests/test_repository_contract.py::test_save_then_get_returns_the_same_booking[fake] PASSED [ 12%]
tests/test_repository_contract.py::test_save_then_get_returns_the_same_booking[sqlite] PASSED [ 25%]
tests/test_repository_contract.py::test_get_of_a_missing_id_raises[fake] PASSED [ 37%]
tests/test_repository_contract.py::test_get_of_a_missing_id_raises[sqlite] FAILED [ 50%]
tests/test_repository_contract.py::test_saving_the_same_id_twice_updates_not_duplicates[fake] PASSED [ 62%]
tests/test_repository_contract.py::test_saving_the_same_id_twice_updates_not_duplicates[sqlite] PASSED [ 75%]
tests/test_repository_contract.py::test_find_by_room_returns_only_that_rooms_bookings[fake] PASSED [ 87%]
tests/test_repository_contract.py::test_find_by_room_returns_only_that_rooms_bookings[sqlite] PASSED [100%]

=================================== FAILURES ===================================
___________________ test_get_of_a_missing_id_raises[sqlite] ____________________

repo = <reservo.sqlite_repo.SqliteBookingRepository object at 0x105ebb750>

    def test_get_of_a_missing_id_raises(repo):
>       with pytest.raises(KeyError):
E       Failed: DID NOT RAISE KeyError

tests/test_repository_contract.py:34: Failed
=========================== short test summary info ============================
FAILED tests/test_repository_contract.py::test_get_of_a_missing_id_raises[sqlite] - Failed: DID NOT RAISE KeyError
========================= 1 failed, 7 passed in 0.04s ==========================

1 failed, 7 passed. Compare this red with the production AttributeError: they're from another planet. Here the message is surgical and tells you three things. First, which clause broke: test_get_of_a_missing_id_raises —"get of an absent id raises"—. Second, who broke it: the bracket [sqlite], the real provider; the [fake] of the same clause stays green, because the fake didn't change. Third, how: Failed: DID NOT RAISE KeyError —the provider should have raised and didn't—. With that you know exactly what to check (SQLite's get), what the contract expected (a KeyError), and that neither the consumer nor the fake is to blame. And all this happened before merging: the breaking change never reached production, never blew up a cancel, never confused anyone with a distant AttributeError. The beep in the kitchen sounded with the fire barely starting.

Notice the asymmetry of the red: only the [sqlite] failed. That detail is pure diagnosis. That [fake] is green and [sqlite] is red in the same clause tells you, unambiguously, "the one that deviated from the contract is the real provider, not the contract itself or the fake". If both sides had failed, you'd suspect the test or the contract; that only one fails points the finger at the implementation that changed. The id in brackets, which in lesson 3 seemed a detail, is here half of the diagnosis.

Why the contract sees what the unit test suite doesn't

You might wonder: why didn't the team's unit test suite catch this? The answer closes the circle of the whole guide. BookingService's unit tests use the FakeBookingRepository, and the fake didn't change: its get still raises KeyError for an absent id. So cancel's unit tests still see a get that raises, still pass, and have no way of knowing that the other provider —the real one, the production one— stopped raising. The breaking change lives exactly in the divergence between the fake and the real one, and no test that uses only the fake can see it. It's module 1's gulf, again, now at the moment of a change.

The contract sees what the unit test doesn't because the contract runs against the provider that changed. The clause test_get_of_a_missing_id_raises[sqlite] exercises the real SqliteBookingRepository, not the fake, so when the real one stops raising, that line notices. It's the same lesson from module 1 —"cross the seam with the real piece"—, now systematized: you don't cross the seam once by hand, but on every run of the contract, for each clause, automatically. The contract is the net that makes "testing against the real thing" not depend on someone remembering to do it.

Shifting the risk to the left

There's an underlying idea worth naming, because it's the why of the whole capstone: moving the discovery of the error to the left in time. Draw the life of a change from left to right: you write it, you review it, the tests run, it's merged, it's deployed, it runs in production. The further to the right you discover a bug, the more expensive it is: in production it costs an incident, affected users, an urgent trace; in review it costs a comment; in your local suite it costs two hundredths of a second and a clear red.

The get breaking change is the same bug in the two examples of this lesson; the only thing that changes is where it's discovered. In part 1 it's discovered in production (far right): expensive, confusing, late. In part 2 it's discovered on running the local battery (far left): cheap, clear, immediate. The contract is the tool that pushes the discovery to the left —from the incident to the red test—. It doesn't make people stop committing breaking changes; it makes them found early, when fixing them is trivial. That is, in one sentence, the reason for being of the three deliverables you assembled: shifting the risk to the left.

Common mistakes

Seeing the red and "fixing the test" instead of the provider. What happens: someone sees test_get_of_a_missing_id_raises[sqlite] in red and changes the test to accept None, turning it green. Why it happens: a red feels like an annoying test, and "making it pass" seems like progress. How to detect it: if your fix of the red consisted of weakening the contract's assertion instead of touching the provider, you inverted the relationship —you let the implementation command over the contract—. How to fix it: the contract is the agreement; the red says the provider violated it. The correct decision is to revert the provider's change (make it raise again) or, if the team deliberately decides that get should return None, renegotiate the contract with the consumer —change the clause and adapt BookingService.cancel at the same time—. Never loosen the test silently: that reintroduces the bug and turns off the alarm.

Believing that "it compiles and passes the unit tests" is safe. What happens: the get change compiles, BookingService's unit tests (with the fake) stay green, and it's deployed with confidence. Why it happens: "it compiles and the tests pass" is the usual safety criterion. How to detect it: if the tests that passed use only the fake on the seam that changed, they didn't test the change —they tested the fake, which didn't change—. How to fix it: for a change in a provider, the safety criterion isn't "the unit tests pass", but "the provider's contract passes", because the contract is the only thing that runs against the implementation you touched. Add the contract battery to what you run before merging repository changes.

Not re-running the contract after a provider change. What happens: someone edits SqliteBookingRepository and doesn't run the battery, trusting that "it was already green". Why it happens: the contract passed yesterday, so re-running it seems unnecessary. How to detect it: if your flow doesn't run the battery on every provider change, a breaking change sneaks in just as a turned-off smoke detector doesn't warn of the new fire. How to fix it: the battery only protects if it runs again. Part 2's arrest happened because someone ran the battery after the change; if they hadn't run it, part 1's crime would have run its course. A detector nobody turns on is useless.

Exercises

Exercise 1 — Read the two errors. Put side by side the production AttributeError and the contract's Failed: DID NOT RAISE KeyError. For each one, say what it tells you about the cause, how far the symptom is from the cause, and at what moment of the change's life cycle it appeared.

See solution

The AttributeError: 'NoneType' object has no attribute 'price_cents' (production):

  • What it tells you about the cause: almost nothing. It names price_cents and NoneType, but doesn't mention get, or the absent id, or the breaking change. The cause —the get that returned None— doesn't appear in the message.
  • How far the symptom is from the cause: far. The error blows up in refund_cents, two steps after the line (get) where the contract really broke. You have to trace uphill to find it.
  • When it appeared: in production, at the far right of the cycle. Late and expensive: with users affected and an incident under way.

The Failed: DID NOT RAISE KeyError (contract):

  • What it tells you about the cause: almost everything. The clause (test_get_of_a_missing_id_raises) names the broken behavior, the bracket ([sqlite]) names the culprit, and the message (DID NOT RAISE) names the exact symptom.
  • How far the symptom is from the cause: right next to it. The red points directly at the clause and the provider that violated it; there's nothing to trace.
  • When it appeared: on running the local battery, at the far left of the cycle. Early and cheap: before merging, in two hundredths of a second.

It's the same bug, discovered at two points in time. The contract didn't prevent it; it moved it from the expensive end (production) to the cheap one (your machine). That's the whole value proposition of contract testing.

Exercise 2 — Another breaking change, another clause. The provider's team now "optimizes" save: to go faster, it stops doing self._conn.commit(). Without running, predict which contract clause(s) would go red, on which side, and why the [fake] one would stay green.

See solution

test_save_then_get_returns_the_same_booking[sqlite] would go red (and probably also test_saving_the_same_id_twice_updates_not_duplicates[sqlite] and test_find_by_room_...[sqlite], all the ones that save and then read). Without commit, depending on how it's read afterward, the get/find_by_room may not find the just-saved row, so "save-and-read returns the same booking" fails: repo.get("bk-1") finds nothing and raises KeyError where the test expected the booking. The clause that promises the round trip breaks.

The [fake] side would stay green because the fake has no transactions or commit: it saves the object directly in a dict, so a save followed by get always finds the booking. The fake didn't change and, by its nature, can't suffer a commit bug —it has no commits—. Again the asymmetry is the diagnosis: [fake] green + [sqlite] red = "the real provider broke a promise the fake still meets". And again, a one-line breaking change (removing a commit) that "compiles and passes the fake's unit tests" but that the contract catches before the deploy.

Exercise 3 — When the change is intentional. Suppose the team deliberately decides that get should return None for an absent id (to align with another API). The contract goes red. Explain the correct sequence to make that change without leaving a bug, and why "just changing the test to accept None" isn't that sequence.

See solution

The correct sequence treats the change as what it is: a renegotiation of the contract between the consumer and the provider, not a unilateral adjustment of the provider. The steps:

  1. Agree on the new contract with the consumer's side. The contract's owner is the consumer; if get is going to return None, you have to look at all the consumers that depended on it raising —starting with BookingService.cancel— and decide how they adapt.
  2. Change the contract's clause to reflect the new agreement: test_get_of_a_missing_id_returns_none instead of ..._raises, asserting assert repo.get("does-not-exist") is None.
  3. Adapt the consumers at the same time. cancel can no longer trust a KeyError; now it must check if booking is None: raise SomeError(...) explicitly before using booking.price_cents. Without this step, part 1's AttributeError comes back.
  4. Make the provider meet the new clause (make it return None) and run the battery: green on both sides, with the new promise.

Why "just changing the test to accept None" is not that sequence: that shortcut does step 2 (loosens the clause) but skips 1 and 3. The result is a green contract and a broken consumer: cancel still expects a KeyError that no longer arrives, and the production AttributeError reappears —only now with no alarm, because you turned off the only one that detected it—. Changing the contract is legitimate; changing it without adapting whoever depended on the old promise is how you introduce a bug with the blessing of a green suite. The red isn't the enemy: it's the list of who has to be notified.

Summary and next step

In this lesson you cashed in the value of the three deliverables: you caught a provider breaking change before deploying it. You saw the same bug —the get that returns None instead of raising— at its two destinations. In production, without a contract, it's an AttributeError: 'NoneType' object has no attribute 'price_cents' inside cancel, far from the cause, confusing and late: the crime. Before the deploy, with a contract, it's a surgical red —test_get_of_a_missing_id_raises[sqlite], DID NOT RAISE KeyError, with [fake] intact— that names the clause, the culprit provider, and the exact symptom in two hundredths of a second: the arrest. With the smoke detector you understood that the contract doesn't prevent the breaking change, but prevents it from growing unseen —the beep in the kitchen instead of the fire at three in the morning—. And you named the underlying idea: shifting the risk to the left, from the expensive incident to the cheap red test.

Before moving on you should be able to: distinguish the production error (distant, confusing) from the contract's red (surgical, immediate); explain why the unit test suite with the fake didn't catch the change and the contract did; read the [sqlite]/[fake] bracket as a diagnosis; and describe the correct sequence for an intentional provider change without leaving a bug.

You now have everything: the three deliverables, and the proof that they're worth it. What's missing is gathering it into a formal delivery and closing the guide. In lesson 8 you write the project statement —the three deliverables, the rubric that evaluates the method and not the quantity, and the complete reference solution with its real output—, and we close the journey: a review of the eight modules and the map of where to continue in the Testing ecosystem.

Resources

← Previous6. Isolate the integration with fixture and rollbackNext →8. Project: Reservo's contract + integration