3. `--cov-fail-under` and the exit code

Description

This is the heart of the module. In the previous two lessons you installed the idea —a quality gate is metric, threshold, and consequence, and the consequence lives in the exit code—. Now you're going to execute it from start to finish, with your hands, on the Reservo suite: the gate breaking the build when a test is missing, you writing the missing test, and the same gate going green. Not a concept: a complete cycle from red to green, with real outputs and real exit codes, which is exactly what you'd do to set up this gate on a real project.

By the end you'll know how to use pytest --cov=reservo --cov-fail-under=80 precisely: what --cov=reservo does (measure the package's coverage), what --cov-fail-under=80 does (set the threshold that breaks the build), and how to read the exit code it emits —1 when coverage is below, 0 when it reaches it—. You're going to see the real cycle: with cancel_with_refund untested, the coverage is 65.38% and the gate breaks the build; you add the test, the coverage rises to 91.03%, and the gate passes. And you'll meet the command's cousin, the coverage CLI —coverage run followed by coverage report --fail-under=80—, with its different exit code (2 instead of 1) and the .coveragerc file that defines what gets measured, a piece you're going to need so the gate doesn't lie.

Connection to the module: this lesson turns lesson 2's theory into muscle. Lesson 1 showed you the gate break the build; lesson 2 defined it by its three parts; this one executes it whole, there and back. And it prepares the two that follow: lesson 4 starts from this same fixed-floor gate to show its blind spot (a drop that doesn't cross the floor), and lesson 5 reuses the exit-code mechanics with another metric (the smoke markers). If you master this lesson's red→green cycle, the others are variations. That's why here we go slowly and really execute each step.

Turn off the alarm by fixing the fire, not by disconnecting the sensor

When your home smoke detector goes off while you're cooking, there are two ways to silence it. The first, the correct one: put out the fire or air out the smoke —solve the cause—. The second, the tempting and dangerous one: rip out the detector's battery so it stops beeping. Both silence the alarm. But one solves the problem and the other just turns off the signal, leaving your house unprotected for the next real fire.

A coverage gate that breaks the build is that alarm. When it goes off —"coverage below 80%"— you have the same two options. The correct one: write the missing tests, raise the coverage for real, and see the gate pass because the problem was solved. The tempting one: lower the threshold (--cov-fail-under=60) or delete the untested code so the number rises without writing a test —rip out the detector's battery—. Both make the build green. But one closes the untested-code gap and the other just hides that it exists.

This lesson teaches you to put out the fire, not to disconnect the sensor. The cycle you're going to execute —gate red, I write the test, gate green— is the correct response made routine. And in lessons 6 and 7 you're going to see, by name, the ways of "ripping out the battery" (the fetish threshold, the tautological tests) and why they do more harm than leaving the alarm ringing. For now, the image: the gate isn't your enemy when it turns red; it's the detector doing its job, pointing you to code you left uncovered.

When the coverage gate breaks the build, the correct response is to write the missing test —put out the fire—, not lower the threshold or delete the untested code —rip out the detector's battery—. Both make the build green; only one closes the gap.

Anatomy of the command: --cov and --cov-fail-under

Before running it, let's take the command apart piece by piece, because each flag does a different thing and it's worth not confusing them.

python -m pytest --cov=reservo --cov-fail-under=80

• python -m pytest — runs the suite, as always. This part discovers and runs Reservo's tests; without the other flags, it's your usual pipeline.
• --cov=reservo — turns on the coverage measurement and tells it what to measure: the reservo package. This is what pytest-cov (the plugin) does while the suite runs: it observes which lines of the reservo package execute and which don't. It's the gate's metric. Without this flag, there's no coverage to report. Note the detail: we measure reservo (the code), not tests (the tests) —we want to know how much of the code the tests exercise, not how much of the tests ran—.
• --cov-fail-under=80 — sets the threshold and arms the consequence: "if the total coverage of the measured code ends up below 80%, finish with a failure exit code". This is the flag that turns the measurement into a gate. Without it, --cov=reservo only reports (you saw it in lesson 2: exit 0 even if coverage is low); with it, it imposes.

Lesson 2's three parts, in one line: metric (--cov=reservo), threshold (80), consequence (the exit code --cov-fail-under emits). Now let's run it.

The cycle, step 1: the gate breaks the build

We start with the suite in the state that opened the module: Reservo debuted cancel_with_refund, but it arrived without tests. The suite has ten tests —three pricing, three refunds, four availability— and they all pass. We run the gate, for real, on Python 3.14.0 with pytest 9.1.1, coverage 7.15.2, and pytest-cov 7.1.0. We add --cov-report=term-missing, which besides the percentage prints which lines ended up uncovered —the exact map of the gap—:

python -m pytest --cov=reservo --cov-report=term-missing --cov-fail-under=80

What to expect (real output, measured by executing):

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
rootdir: /private/tmp/reservo-m6
plugins: cov-7.1.0
collected 10 items

tests/test_availability.py ....                                          [ 40%]
tests/test_pricing.py ...                                                [ 70%]
tests/test_refunds.py ...
ERROR: Coverage failure: total of 65 is less than fail-under=80
                                                                         [100%]

================================ tests coverage ================================
_______________ coverage: platform darwin, python 3.14.0-final-0 _______________

Name                       Stmts   Miss  Cover   Missing
--------------------------------------------------------
reservo/__init__.py            0      0   100%
reservo/calendar.py           30      7    77%   24, 26, 34, 49-51, 55
reservo/cancellations.py      17     17     0%   7-36
reservo/models.py             12      2    83%   34-35
reservo/pricing.py            10      1    90%   14
reservo/refunds.py             9      0   100%
--------------------------------------------------------
TOTAL                         78     27    65%
FAIL Required test coverage of 80% not reached. Total coverage: 65.38%
============================== 10 passed in 0.03s ==============================

Read it in layers. At the top, 10 passed: the ten tests pass. The coverage table tells you why the gate complains anyway. Look at the row reservo/cancellations.py 17 17 0% 7-36: it has 17 code lines (Stmts), all 17 unexecuted (Miss), 0% coverage, and the uncovered lines are 7 to 36 —the whole file—. That's cancel_with_refund and refund_reason, real code no test touches. The reservo/calendar.py row is at 77%: it's missing lines 24, 26, 34, 49-51, 55 —the branches of is_available no test walks, the raise in book, and the cancel method, which was only called from the untested cancellation—. Summing the whole package: TOTAL 78 27 65% —78 lines, 27 uncovered, 65.38%—.

And at the bottom, the gate: FAIL Required test coverage of 80% not reached. Total coverage: 65.38%, with its echo above, ERROR: Coverage failure: total of 65 is less than fail-under=80. The suite passed, but the gate didn't. Let's confirm the build really broke, by looking at the only datum CI would read:

python -m pytest --cov=reservo --cov-fail-under=80 > /dev/null 2>&1; echo "exit code: $?"

exit code: 1

Exit code 1. On the runner, this 1 would paint the step red and stop the merge —just like a failed test, even though here the ten tests passed—. The gate did its job: it detected that a third of the code isn't tested and blocked the change. Now, let's put out the fire.

The cycle, step 2: I write the missing test

The correct response isn't to lower the threshold; it's to test cancel_with_refund. We write a test file for the cancellation that really exercises the function —cancel a booking, confirm the refund, and verify that the attempt to cancel twice fails—:

# tests/test_cancellations.py
from datetime import datetime

from reservo.calendar import Calendar
from reservo.cancellations import cancel_with_refund, refund_reason
from reservo.models import Member, Room

focus = Room(id="r-focus", name="Focus", capacity=1, hourly_cents=2500)
ana = Member(id="m-1", name="Ana", tier="basic")


def _confirmed_booking(cal, start, end):
    return cal.book(focus, ana, start, end)


def test_cancel_with_full_refund_frees_the_slot():
    cal = Calendar()
    booking = _confirmed_booking(cal, datetime(2026, 1, 4, 9), datetime(2026, 1, 4, 12))
    # basic 3h booking = 7500; cancel 72h ahead -> 100% back
    now = datetime(2026, 1, 1, 9)
    refund = cancel_with_refund(cal, booking, now)
    assert refund == 7500
    assert booking.status == "cancelled"
    # the slot is free again after cancelling
    assert cal.is_available(focus, datetime(2026, 1, 4, 10), datetime(2026, 1, 4, 11)) is True


def test_cancel_twice_raises():
    cal = Calendar()
    booking = _confirmed_booking(cal, datetime(2026, 1, 4, 9), datetime(2026, 1, 4, 12))
    now = datetime(2026, 1, 1, 9)
    cancel_with_refund(cal, booking, now)
    try:
        cancel_with_refund(cal, booking, now)
        assert False, "expected ValueError on double cancel"
    except ValueError:
        pass


def test_refund_reason_text():
    booking = _confirmed_booking(Calendar(), datetime(2026, 1, 4, 9), datetime(2026, 1, 4, 12))
    assert "full refund" in refund_reason(booking, datetime(2026, 1, 1, 9))
    assert "half refund" in refund_reason(booking, datetime(2026, 1, 2, 21))
    assert "no refund" in refund_reason(booking, datetime(2026, 1, 4, 0))

Notice these aren't filler tests: they verify real behavior. test_cancel_with_full_refund_frees_the_slot confirms the refund anchor number (7500, the 100% of a basic 3 h booking), that the booking ended up cancelled, and that its slot was freed. test_cancel_twice_raises confirms that cancelling twice is an error. test_refund_reason_text walks the three refund tranches. They're tests that assert something, not just that touch the code —the distinction lesson 7 makes central—. Let's run the gate again.

The cycle, step 3: the gate passes

Same command, now with the test added:

python -m pytest --cov=reservo --cov-report=term-missing --cov-fail-under=80

What to expect (real output):

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
rootdir: /private/tmp/reservo-m6
plugins: cov-7.1.0
collected 13 items

tests/test_availability.py ....                                          [ 30%]
tests/test_cancellations.py ...                                          [ 53%]
tests/test_pricing.py ...                                                [ 76%]
tests/test_refunds.py ...                                                [100%]

================================ tests coverage ================================
_______________ coverage: platform darwin, python 3.14.0-final-0 _______________

Name                       Stmts   Miss  Cover   Missing
--------------------------------------------------------
reservo/__init__.py            0      0   100%
reservo/calendar.py           30      4    87%   26, 34, 50, 55
reservo/cancellations.py      17      0   100%
reservo/models.py             12      2    83%   34-35
reservo/pricing.py            10      1    90%   14
reservo/refunds.py             9      0   100%
--------------------------------------------------------
TOTAL                         78      7    91%
Required test coverage of 80% reached. Total coverage: 91.03%
============================== 13 passed in 0.03s ==============================

Look at what changed. collected 13 items —the three new tests—, all green. The reservo/cancellations.py row went from 0% to 100%: its 17 lines are now exercised by the test. reservo/calendar.py rose from 77% to 87%, because the cancellation test also walks calendar.cancel and is_available with a cancelled booking in the mix. The total jumped from 65.38% to TOTAL 78 7 91% —91.03%—. And at the bottom, the gate changed tone: Required test coverage of 80% reached. Total coverage: 91.03%. No FAIL, no ERROR. Let's confirm the exit code:

python -m pytest --cov=reservo --cov-fail-under=80 > /dev/null 2>&1; echo "exit code: $?"

exit code: 0

Exit code 0. Green. The build would pass on the runner. And —this is what matters— we didn't achieve it by lowering the threshold or deleting code: we achieved it by testing the code that was untested. We put out the fire. The complete cycle —gate red (exit 1, 65.38%), I write the test, gate green (exit 0, 91.03%)— is the routine this module teaches you to make natural. And notice there are still uncovered lines (calendar.py at 87%, models.py at 83%): the gate doesn't require 100%, it requires passing 80. That margin between "enough" and "perfect" is precisely lessons 6 and 7's topic.

The CLI cousin: coverage report --fail-under and the .coveragerc

pytest --cov-fail-under is the most common way to set the gate, but there's a second, more explicit one, worth knowing because you'll sometimes see it in pipelines and because it separates the two phases —measuring and evaluating— into two commands. It's the CLI of the coverage tool (which pytest-cov leans on internally):

1. coverage run -m pytest — runs the suite while measuring the coverage, and saves the data in a .coverage file. It evaluates nothing yet; it only measures.
2. coverage report --fail-under=80 — reads that data, prints the report, and applies the gate: if the total is below 80, it finishes with a failure exit code.

But there's a detail that bites if you ignore it: by default, coverage run measures everything that runs, including the test files, and ignores files that were never imported. That's a double problem: it inflates the number (the tests are always 100% "covered", because they ran) and, worse, cancellations.py would disappear from the report —since no test imports it, coverage doesn't even see it, and can't report a 0% of a file it doesn't know exists—. A gate that doesn't see the untested code protects from nothing.

The solution is a configuration file, .coveragerc, that tells coverage exactly what to measure:

# .coveragerc
[run]
source = reservo

[report]
show_missing = True

source = reservo does two things: it limits the measurement to the reservo package (excluding the test files) and —key— it tells coverage to discover all the package's files, even if they weren't imported, and count them as 0% if no one touched them. That way cancellations.py without tests appears with its honest 0%. show_missing = True adds the uncovered-lines column, like term-missing. With this .coveragerc in place, we run the CLI on the incomplete suite (without the cancellation test):

coverage run -m pytest
coverage report --fail-under=80

What to expect (real output):

Name                       Stmts   Miss  Cover   Missing
--------------------------------------------------------
reservo/__init__.py            0      0   100%
reservo/calendar.py           30      7    77%   24, 26, 34, 49-51, 55
reservo/cancellations.py      17     17     0%   7-36
reservo/models.py             12      2    83%   34-35
reservo/pricing.py            10      1    90%   14
reservo/refunds.py             9      0   100%
--------------------------------------------------------
TOTAL                         78     27    65%
Coverage failure: total of 65 is less than fail-under=80

Same 65.38%, same table, same cancellations.py at 0% —because source = reservo made it visible—. And the gate fails with its message: Coverage failure: total of 65 is less than fail-under=80. But look at the exit code, which is different:

coverage report --fail-under=80 > /dev/null 2>&1; echo "exit code: $?"

exit code: 2

Exit code 2, not 1. Here's the nuance lesson 2 anticipated: the coverage CLI uses exit code 2 for "fail_under wasn't reached", while pytest-cov uses 1. For CI it doesn't matter —any non-zero number is red—, but for you the number tells a story: a 2 tells you "it was the CLI coverage gate", while a 1 from pytest could be "the gate or a test that failed". When you add the cancellation test and run coverage run -m pytest && coverage report --fail-under=80 again, the total rises to 91% and the exit code drops to 0 —the gate passes—, just like with pytest-cov.

Which to use? For most projects, pytest --cov=reservo --cov-fail-under=80 in a single command is simpler and is what we'll put in the mini-project's workflow. The two-step CLI shines when you want to separate measurement from evaluation —for example, measuring across several matrix jobs and combining the data before applying the gate a single time—, an advanced case we only mention. What matters is that you recognize both, and above all that you understand the .coveragerc, because source = reservo is what keeps the gate from blinding itself to exactly the untested code it should catch.

Common mistakes

Lowering the threshold so the gate passes (ripping out the detector's battery). What happens: the gate breaks the build at 65.38%, and someone "solves" the red by changing --cov-fail-under=80 to --cov-fail-under=60. The build goes green again, but the untested code stays exactly as untested. Why it happens: lowering a number is faster than writing a test, and the green build gives the false sense of having solved something. How to spot it: if the real coverage didn't rise but the gate passed, someone moved the threshold, not wrote tests. Check the --cov-fail-under history. How to fix it: treat the threshold as a floor that only rises (lesson 4's ratchet). The red gate is an invitation to write the missing test, not to lower the requirement. Lowering the threshold is legitimate only as an explicit and justified team decision, never as a reflex to silence a red.

Measuring tests instead of reservo, or not scoping with .coveragerc. What happens: someone runs coverage run -m pytest without .coveragerc (or puts --cov=.), and the report includes the test files —always "100% covered"— and ignores cancellations.py, which was never imported. The number comes out inflated and the real gap, invisible. Why it happens: by default coverage measures what runs, and the tests run, so they "count". How to spot it: if your report has test_*.py rows at 100% and you don't see the untested modules, you're measuring the wrong thing. How to fix it: source = reservo in .coveragerc (or --cov=reservo with pytest-cov) to measure the code, not the tests, and so the unimported modules appear with their honest 0%. The gate must see exactly the code no one tested; scoping it wrong blinds it.

Reading the report and not the exit code. What happens: someone sees the coverage report printed, with its table and its percentage, and assumes the gate "is working", without verifying the command finishes with a non-zero exit code when it should. Why it happens: the report is what's visible; the exit code has to be asked for with echo $?. How to spot it: run the gate with coverage below the threshold and confirm the exit code (1 for pytest-cov, 2 for the CLI). If it's 0, you have a report but no gate. How to fix it: always verify the gate bites —just as you verify a test bites by breaking the code—, putting coverage below the threshold on purpose and seeing the failure exit code. CI reads the exit code, not the table.

Exercises

Exercise 1 — Predict the exit code. For each command on Reservo, predict the exit code (0 or non-zero, and which) and explain why in one sentence. Assume the real coverage in each state. (a) Incomplete suite (65.38%): pytest --cov=reservo --cov-fail-under=80. (b) Complete suite (91.03%): pytest --cov=reservo --cov-fail-under=80. (c) Incomplete suite (65.38%): pytest --cov=reservo (without --cov-fail-under). (d) Incomplete suite (65.38%): coverage report --fail-under=80 (with data from a previous run).

Exercise 2 — Write the test that puts out the fire. Reservo has another untested function: refund_reason(booking, now), which returns the refund text ("full refund", "half refund", "no refund") based on how many hours before it's cancelled. The gate is red because this function lowers the coverage. Write a test that exercises it in its three tranches (≥ 48 h, 24–48 h, < 24 h) and explain why writing the test is a better response than lowering the threshold.

from datetime import datetime

from reservo.calendar import Calendar
from reservo.cancellations import refund_reason
from reservo.models import Member, Room

focus = Room(id="r-focus", name="Focus", capacity=1, hourly_cents=2500)
ana = Member(id="m-1", name="Ana", tier="basic")


def test_refund_reason_covers_the_three_tiers():
    booking = Calendar().book(focus, ana, datetime(2026, 1, 4, 9), datetime(2026, 1, 4, 12))
    # >= 48h before -> full
    assert "full refund" in refund_reason(booking, datetime(2026, 1, 1, 9))    # 72h before
    # 24-48h before -> half
    assert "half refund" in refund_reason(booking, datetime(2026, 1, 2, 21))   # 36h before
    # < 24h before -> none
    assert "no refund" in refund_reason(booking, datetime(2026, 1, 4, 0))      # 9h before

Exercise 3 — Diagnose the lying .coveragerc. A teammate sets up a gate with the coverage CLI, but without a .coveragerc. They run coverage run -m pytest and coverage report --fail-under=80, and it gives them exit 0 —the gate passes— even though cancellations.py has not a single test. Why didn't the gate catch it, and how do you fix it?

[run]
source = reservo

[report]
show_missing = True

Summary and next step

In this lesson you executed the heart of the module: the complete cycle of a coverage gate on Reservo, from red to green, with real outputs and exit codes. You took the command apart —--cov=reservo measures (metric), --cov-fail-under=80 imposes (threshold + consequence)— and saw the three phases: the gate breaking the build with cancel_with_refund untested (65.38%, Coverage failure: total of 65 is less than fail-under=80, exit 1); you writing the missing test —one that verifies, not just executes—; and the same gate passing (91.03%, Required test coverage of 80% reached, exit 0). The correct response to the red was to put out the fire (write the test), not rip out the detector's battery (lower the threshold).

You also met the CLI cousin: coverage run -m pytest followed by coverage report --fail-under=80, with its 2 exit code (instead of pytest-cov's 1) and —most importantly— the .coveragerc with source = reservo, which keeps the gate from blinding itself by measuring the tests and ignoring the never-imported code. A gate is worth what its what-to-measure configuration is worth.

Before moving on you should be able to: write pytest --cov=reservo --cov-fail-under=80 and explain what each flag does; read a term-missing report and locate the file that sinks the coverage; predict a gate's exit code based on the coverage and the tool (1 pytest-cov, 2 CLI, 0 if it passes); and explain why source = reservo in .coveragerc is what makes the gate honest.

What's next, in lesson 4, is a blind spot of this same gate. The fixed floor of 80 protects against coverage dropping below 80, but not against it dropping while above: if your coverage is 91% and someone adds untested code that lowers it to 84%, the floor-80 gate passes —84 > 80— and doesn't see the seven-point drop. Lesson 4 shows that gap with a real demo and how to close it with a gate that fails on the coverage drop, not just against a distant floor: the ratchet that only goes up.

Resources

• pytest-cov: --cov-fail-under and the report options — the reference for the flags you used (--cov, --cov-report=term-missing, --cov-fail-under). The first place to consult when you set the gate on a real project.
• Coverage.py: the report command and --fail-under — the two-step CLI (run + report) with its own exit code. Note the exit-codes table: 2 is "below fail_under".
• Coverage.py: configuration with .coveragerc and source — how to tell coverage what to measure. The source option in [run] is the one that keeps the gate from blinding itself; read it carefully.
• pytest exit codes — the official table that confirms the 1 (there were failures, including the coverage gate) versus the 0 (all passed). The CI language.