Module 4: Verifying The Contract From Both Sides

6. Catching a consumer over-assumption

Description

Lesson 5 caught a provider that breaks a promise. This one catches the mirror error: a consumer that relies on a promise that was never made. They're the only two ways to break a contract. In the first, the agreement says X and the provider stops fulfilling X. In the second, the consumer takes Y for granted, but Y isn't in the agreement —it's a detail the current provider happens to have, not something it promised—. The first error is caught by the provider side; the second, more slippery, is caught by the consumer side, and only if you know how to look for it.

The over-assumption is slippery because it works. The consumer that relies on an unpromised detail doesn't fail today: its current provider has that detail, so the code runs, the tests pass, everything green. The bomb is armed for the day the provider changes —to another equally valid implementation, to a new version, to a different database engine— and the unpromised detail disappears. Then the consumer breaks, and the blame seems to be the provider's ("it changed and broke me!"), when in reality the provider fulfilled the contract to the letter and it was the consumer who relied on something outside it. In Reservo, the concrete case: BookingService assumes that find_by_room returns the bookings ordered by start, and the contract only promises the set of bookings, with no order at all. This lesson teaches the technique for catching that assumption before it blows up: running the consumer against a provider that returns a different legal order.

Connection to the module: it closes the pair lesson 5 opened. Together they cover the two directions of non-compliance —provider that breaks a promise (5), consumer that invents one (6)— and complete the idea from lesson 2, where the seed was planted: "the consumer must rely only on what's promised". Here that sentence becomes a concrete test with real output. Lesson 7 will go up a level to ask who defines what's promised, which is exactly what decides whether an assumption is legitimate or over.

Analogy: the one who memorized the drawer instead of reading the label

Imagine a pharmacy with a clear agreement: each medicine lives in a drawer labeled with its name, and the agreement —the contract— is "the drawer says what it contains". A new employee learns to read the label: to hand over ibuprofen, they look for the drawer that says "ibuprofen". They rely on what's promised: the label. Another, more veteran employee never reads the labels: they memorized that ibuprofen "is in the third drawer of the second row". And it works —for years— because in that pharmacy ibuprofen has been in that drawer for years. They rely on something the agreement doesn't promise: the position.

The day the shelving is reorganized —something perfectly legitimate, because the agreement never promised positions, only labels—, the employee who reads labels keeps working without noticing the change. The one who memorized positions hands over the contents of the third drawer of the second row, which is now something else, and swears "they moved everything wrong". But no one broke the agreement: the drawers are still labeled correctly. It was they who relied on the position, an unpromised detail, instead of the label, the only thing guaranteed.

find_by_room is the shelving: the contract promises which bookings there are (that room's, as the labels promise what each drawer contains), but doesn't promise the order (as it doesn't promise fixed positions). The consumer that does find_by_room(...)[0] to get "the first one" is the employee who memorized positions: it works while the provider returns the bookings in a certain order, and breaks silently the day an equally valid provider returns them in another. This lesson's technique is to hire an evaluator who deliberately reorganizes the shelving before testing the employee —to see whether they really read labels or memorized positions—.

Worked example: the consumer that assumes an order

Here's the consumer under suspicion. BookingService wants "a room's earliest booking", and solves it by taking the first element of find_by_room, assuming the list comes ordered by start ascending:

# reservo/scheduling.py — consumer logic that uses find_by_room
def earliest_booking_over_assuming(repo, room_id):
    # OVER-ASSUMES: that find_by_room already comes ordered by start ascending,
    # something the contract NEVER promised.
    return repo.find_by_room(room_id)[0]


def earliest_booking_correct(repo, room_id):
    # Relies ONLY on what the contract promises: the set of bookings,
    # without any order assumption. If it needs order, the consumer imposes it.
    return min(repo.find_by_room(room_id), key=lambda b: b.start)

The two functions answer the same question —"which is the earliest booking?"— but rely on different things. The first, over_assuming, hangs on find_by_room returning the bookings already ordered by start; it takes [0] and trusts. The second, correct, assumes nothing about the order: it receives the set and computes the minimum by start itself. If the contract promises the set but not the order, only the second is correct —the first is a time bomb—.

Now the technique for catching it. The key is the helper that prepares the repository: we save the bookings out of start order —first the 11 o'clock one, then the 9 o'clock one—. This is legal: the contract doesn't say what order you save in or what order they're returned in. A provider that returns the bookings in insertion order (like the fake, or like SQLite without ORDER BY) will return them [11am, 9am], and there the over-assumption is exposed:

# tests/test_consumer_over_assumption.py
from datetime import datetime

from reservo.doubles import FakeBookingRepository
from reservo.models import Booking
from reservo.scheduling import (earliest_booking_correct,
                                earliest_booking_over_assuming)


def booking(id, hour):
    start = datetime(2026, 3, 10, hour)
    end = datetime(2026, 3, 10, hour + 1)
    return Booking(id=id, room_id="focus", member_id="m-ana",
                   start=start, end=end, status="confirmed", price_cents=6000)


def a_repo_saved_out_of_start_order():
    repo = FakeBookingRepository()
    repo.save(booking("bk-11am", 11))    # saved FIRST
    repo.save(booking("bk-9am", 9))      # saved LATER (starts earlier)
    return repo                          # find_by_room -> [11am, 9am] (legal order)


# the consumer over-assumes: thinks [0] is the earliest
def test_over_assuming_consumer_returns_the_wrong_booking():
    repo = a_repo_saved_out_of_start_order()
    earliest = earliest_booking_over_assuming(repo, "focus")
    assert earliest.id == "bk-9am"       # expects the 9 o'clock one; the contract doesn't promise order


# the correct consumer only relies on the set and orders on its own
def test_correct_consumer_returns_the_earliest_regardless_of_order():
    repo = a_repo_saved_out_of_start_order()
    earliest = earliest_booking_correct(repo, "focus")
    assert earliest.id == "bk-9am"

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

python3 -m pytest tests/test_consumer_over_assumption.py -v

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
collected 2 items

tests/test_consumer_over_assumption.py::test_over_assuming_consumer_returns_the_wrong_booking FAILED [ 50%]
tests/test_consumer_over_assumption.py::test_correct_consumer_returns_the_earliest_regardless_of_order PASSED [100%]

=================================== FAILURES ===================================
____________ test_over_assuming_consumer_returns_the_wrong_booking _____________

    def test_over_assuming_consumer_returns_the_wrong_booking():
        repo = a_repo_saved_out_of_start_order()
        earliest = earliest_booking_over_assuming(repo, "focus")
>       assert earliest.id == "bk-9am"       # expects the 9 o'clock one; the contract doesn't promise order
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       AssertionError: assert 'bk-11am' == 'bk-9am'
E         
E         - bk-9am
E         ?    ^
E         + bk-11am
E         ?    ^^

tests/test_consumer_over_assumption.py:28: AssertionError
========================= 1 failed, 1 passed in 0.01s ==========================

There's the over-assumption, caught. The over_assuming consumer returned 'bk-11am' when the earliest booking is 'bk-9am': it took the [0] of a list that came in insertion order, not start order, and got it wrong. The correct consumer, which computes the minimum by start without assuming order, returned the right one. The technique worked because we prepared the repository out of order on purpose: we gave the provider a perfectly legal excuse to return the bookings in an order the consumer didn't expect, and so the consumer that relied on order gave itself away. A provider that happened to return everything ordered would have let the bug through; the trick is not to give it that coincidence.

The technique, named: the adversarial-but-legal provider

What you just did has a shape worth fixing, because it's the way to catch over-assumptions. The idea is this: to verify that a consumer relies only on what's promised, run it against a provider that fulfills the contract at the legal minimum, deliberately choosing the most uncomfortable behavior the contract still allows. If the contract doesn't promise order, your test provider returns a strange but legal order. If the contract allows None in an optional field, your provider returns None. If the contract doesn't promise that two calls return the same object, your provider returns different copies. It's an adversarial-within-the-law provider: it doesn't violate the contract, but it exploits every freedom the contract leaves it.

Why does it work? Because a consumer that relies only on what's promised survives any legal provider, including the most uncomfortable. And a consumer that over-assumes breaks exactly against the freedoms the adversary exploits. The adversarial-but-legal provider turns a tacit assumption ("surely it comes ordered") into a visible failure. In our case, a_repo_saved_out_of_start_order is that adversary: it uses the fake —which honors the contract— but feeds it so that it returns the most annoying order the contract allows. We don't lie about the contract; we exploit that the contract doesn't promise order.

Contrast this with the "friendly" consumer test from lesson 2, which used comfortable data and passed. A friendly consumer test verifies that the consumer works in the happy case; an adversarial-but-legal consumer test verifies that the consumer relies on nothing outside the contract. Both are useful: the friendly one documents normal use, the adversarial one catches the fragility. For over-assumptions, you need the adversary.

The two cures: tighten the consumer or widen the contract

When you catch an over-assumption, you have exactly two legitimate ways out, and choosing the right one is a design decision, not a matter of taste.

Cure A — tighten the consumer so it doesn't over-assume. If the order shouldn't be part of the contract —because other providers reasonably wouldn't guarantee it, or because asking for it would have a cost—, then the fix is for the consumer to stop depending on it. It's what earliest_booking_correct does: it receives the set and orders on its own with min(..., key=lambda b: b.start). The consumer takes responsibility for what it needs instead of demanding it of the provider. This is the default cure: it keeps the contract minimal and pushes the responsibility to whoever has the requirement.

Cure B — widen the contract so the order is a promise. If it turns out that many consumers need the bookings ordered by start, and it makes sense for every provider to guarantee it, then the way out is to add the order as a contract clause: a test_find_by_room_returns_bookings_sorted_by_start that runs against both providers, forcing the fake to order and SQLite to add ORDER BY start. With that, the order stops being an over-assumption and becomes a legitimate promise, guaranteed by the battery against both sides. The consumer can now rely on the order, because it's now promised.

The question that decides between A and B is: should this behavior be a promise of every provider, or is it a requirement of this consumer? If it's the consumer's, cure A (have it solve it itself). If it should be every provider's, cure B (raise it to the contract). What is not a legitimate way out is leaving the over-assumption untreated —neither tightening the consumer nor widening the contract—, because that leaves the bomb armed. Catching it forces a decision; the decision is A or B, never "I do nothing".

Common mistakes

Writing the consumer test with already-ordered data. What happens: someone tests earliest_booking by saving the bookings in start order (9am then 11am), the [0] gives the correct one, and the test passes —hiding the over-assumption—. Why it happens: it's natural to prepare the data in "natural" order. How to detect it: if your consumer test never feeds the provider an uncomfortable order, it can't catch an order dependency. How to fix it: to test that the consumer doesn't assume order, save out of order on purpose. The uncomfortable test data isn't a whim: it's the only thing that exercises the freedom the contract leaves and that the consumer might be violating. A consumer test that only uses comfortable data is a rehearsal with a net: it doesn't test the real fall.

Confusing "the fake returned it that way" with "the contract promises it". What happens: someone sees that the fake returns the bookings in insertion order and concludes "then the contract guarantees insertion order". Why it happens: an implementation's observed behavior is taken as if it were the agreement. How to detect it: ask yourself whether that behavior is written as a clause in the battery. The fake's insertion order has no test_... that promises it; it's an accident of a dict preserving insertion. How to fix it: the contract is what's written and verified against both sides, not what an implementation happens to do. If there's no clause promising the order, the order isn't promised —however much the fake does it today—. Relying on the observed instead of the promised is the very definition of the over-assumption.

"Fixing" the red by changing the test data instead of the consumer. What happens: someone sees test_over_assuming_... red and "fixes it" by re-saving the bookings in order, so [0] gives the correct one. Why it happens: reordering the data makes the red disappear with less effort than fixing the consumer. How to detect it: if your fix consisted of arranging the data so the over-assumption works again, you fixed nothing: you re-armed the bomb and turned off the alarm. How to fix it: the red is saying "the consumer relies on something unpromised"; the answer is one of the two cures (tighten the consumer or widen the contract), not touching up the data. The adversarial test must keep feeding out of order; what changes is the consumer (or the contract), not the data that gives it away.

Exercises

Exercise 1 — Another over-assumption. BookingService has a method that does repo.get(booking_id).start.strftime("%H:%M") to show the start time. What is it over-assuming about what get returns, on which provider would it break, and how would you catch it?

See solution

What it over-assumes: that get returns a start of type datetime (because .strftime(...) is a datetime method). The contract promises that get returns a booking with the correct fields —and in our contract, thanks to the save-and-read clause that verifies got.start == START as a datetime, the type is promised—. But if the contract did not verify the type of start (if it only compared, say, the id and the price_cents), then assuming it's a datetime would be an over-assumption.

On which provider it would break: on any that returned the start as a str instead of a datetime —exactly the broken SqliteBookingRepository from module 1, before the fix with fromisoformat—. A str has no .strftime, so it would blow up with AttributeError: 'str' object has no attribute 'strftime'.

How you'd catch it: with an adversarial-but-legal provider that returns the start as a str (if the contract allowed text) and running the consumer against it; or, better, guaranteeing in the contract that start is a datetime (the clause we already have) so the assumption stops being "over" and becomes legitimate. This exercise shows the link with module 1: that datetime divergence was, seen from this angle, a consumer over-assumption about the type —and the cure was cure B: raise the type to the contract and verify it against both sides—.

Exercise 2 — Choose the cure. For each case, decide whether the correct cure is A (tighten the consumer) or B (widen the contract), and justify: (a) a single internal report needs the bookings ordered by price; (b) all the consumers that list bookings for a user want them ordered by date, from most recent to oldest, and it would be strange for a provider not to guarantee it.

See solution

(a) Order by price for an internal report → cure A (tighten the consumer). It's a requirement of one specific and peculiar consumer (ordering by price isn't a general need of "listing bookings"). Forcing every provider to know how to order by price would load the contract with a promise almost no one uses. The right thing is for the report to receive the set and order by price itself (sorted(bookings, key=lambda b: b.price_cents)). The requirement lives where it's born: in the consumer that has it.
(b) All the consumers want order by descending date → cure B (widen the contract). When many consumers need the same thing and it's reasonable to demand it of every provider, raising it to the contract avoids each consumer reimplementing the order (and avoids one forgetting and being left fragile). A clause test_find_by_room_returns_bookings_sorted_by_start_desc is added, verified against both providers, and from there the order is a legitimate promise everyone can rely on.

The decision rule: does this consumer need it, or every consumer? An idiosyncratic requirement is solved in the consumer (A); a shared and reasonable requirement is promoted to a contract promise (B). The cost of getting it wrong: doing B when it should have been A inflates the contract with promises that tie every provider needlessly; doing A when it should have been B spreads the same ordering logic across all the consumers, inviting one to forget it.

Exercise 3 — An explicit adversarial provider. Instead of feeding the fake out of order, write a small ShufflingRepository provider that wraps another repository and returns find_by_room in reversed order —legal, because the contract doesn't promise order—. Explain why running the consumers against it is a more systematic way to catch order assumptions than preparing the data by hand.

See solution

An explicit adversarial provider:

class ShufflingRepository:
    def __init__(self, inner):
        self._inner = inner

    def save(self, booking):
        self._inner.save(booking)

    def get(self, booking_id):
        return self._inner.get(booking_id)          # relies on the inner's contract

    def find_by_room(self, room_id):
        return list(reversed(self._inner.find_by_room(room_id)))   # legal order, uncomfortable

It wraps any repository that honors the contract (for example the fake) and returns the bookings in reverse. Since the contract doesn't promise order, reversing is legal: ShufflingRepository still honors the contract (it saves, reads, raises on the missing id, filters by room) but exploits the order freedom to the max.

Why it's more systematic than preparing the data by hand: with data-by-hand, each consumer test has to remember to save out of order, and it's easy to forget (and go back to comfortable data). With the ShufflingRepository, you wrap any provider once and all the consumer tests that run against it are exposed to an uncomfortable order automatically, without depending on how the data was prepared. It's the adversarial-but-legal provider technique made reusable: a wrapper that forces the contract's freedom, so any consumer that assumes order gives itself away without you having to remember to set the trap. The same can be done for other freedoms (a CopyingRepository that returns copies on get, to catch object-identity assumptions).

Summary and next step

In this lesson you caught the breaking change's mirror error: a consumer over-assumption. BookingService took for granted that find_by_room came ordered by start —it took [0] as "the earliest"—, when the contract only promises the set of bookings, with no order. With the employee who memorized the drawer instead of reading the label you understood why the assumption works today and blows up tomorrow: it relies on a detail the current provider happens to have, not on something promised. And you saw the technique for giving it away —the adversarial-but-legal provider: feeding the provider so it returns the most uncomfortable order the contract still allows— with real output: the consumer that assumes order returned 'bk-11am' instead of 'bk-9am', while the correct consumer, which orders on its own, gave the right answer. And you named the two cures —tighten the consumer (A) or widen the contract (B)—, decided by a single question: does this consumer need it or should every provider promise it?

Before moving on you should be able to: distinguish an over-assumption from a legitimate dependency (is it written as a clause, or does only the current provider do it?); use an adversarial-but-legal provider to catch it; choose between tightening the consumer and widening the contract; and rule out the false fix of reordering the data to touch up the red.

With lessons 5 and 6 you have the two ways to break a contract, each caught on its side. There remains the governance question both brushed against: when the consumer assumes something and the provider does another, who's right? Who defines what's promised? Lesson 7 answers: the consumer owns the contract —it's consumer-driven—, and explains how that idea is automated between services in the concept of Pact.

Resources

pytest documentation — Assertion reports (assert) — how pytest generates the diff assert 'bk-11am' == 'bk-9am' that gives away the over-assumption; useful for reading exactly what value the fragile consumer returned versus the expected one.
docs.pact.io — Consumer-driven contracts — the frame that explains why the consumer must rely only on what the contract promises and why the consumer's expectations are the contract; the conceptual basis of what counts as an "over-assumption".
sorted and min with key (Python documentation) — the tool with which the correct consumer (min(..., key=lambda b: b.start)) takes responsibility for the order instead of demanding it of the provider; cure A made code.
Module 4, lesson 2 — The consumer side — where the seed "the consumer must rely only on what's promised" was planted; this lesson is that idea turned into a test with real output.

← Previous5. Catching a provider breaking change Next →7. Who owns the contract: consumer-driven