Module 4: Markers And Configuration

8. Mini-project: mark and configure the Reservo suite

Next →

Overview

By the end of this lesson you will have gathered the whole module into a real delivery: take the Reservo suite of module 3 —thirteen tests in layers, unmarked and without central configuration— and build the complete configuration layer for it. You are going to design the marker vocabulary, stick them on the tests (with module-level pytestmark where it fits), register them in pyproject.toml, activate --strict-markers, and configure addopts = "-m smoke" so the suite runs the smoke one by default and the slow one on demand. The deliverable is double: the marked suite and the configuration contract that governs it, verified with the -m expressions you mastered in lesson 7.

This matters because it is the first time in the module you make the whole journey without anyone giving you the pieces already in place. Each previous lesson showed you an isolated ingredient —what a marker is, how to register it, what addopts is—; here you assemble them in the correct order on a raw suite, which is exactly what you will do in a real project when you inherit a layered suite and it is your turn to give it its marker and config layer. And there is an honest "done" criterion this project trains: the configuration layer is not ready when the suite passes —an unmarked suite also passes—, but when plain pytest runs only the smoke one, when pytest -m slow runs only the slow ones, when a marker typo breaks collection, and when the header proves the contract loaded. Building well is that each of those behaviors happens and you verify it by running.

Connection to the module: this lesson is the synthesis. Each step activates a previous one: designing the vocabulary is lesson 3; sticking with pytestmark is lesson 3; registering and --strict-markers is lesson 4; writing the contract (testpaths, markers) is lesson 5; setting addopts = "-m smoke" is lesson 6; verifying with -m expressions is lesson 7. And it closes the module with a deliverable piece. Remember the boundary: here the contract is single —one behavior for the whole team—; making it vary by environment (local versus CI) is module 7. What you deliver is the fixed base on which that variation will be mounted later.

The workshop you fit with a control panel

Think of it this way, closing the metaphor of module 3. In that mini-project you remodeled the workshop: you set up the box with compartments —tests/unit/, tests/integration/—, so the tools had their place. The workshop ended up navigable, but mute: to run "just the fast ones" or "just the critical ones" you had to know by hand what folder or what file to ask for, and each person did it their own way.

This mini-project fits the workshop with its control panel. It is the wall panel with labeled buttons: a "quick check" button that runs the smoke suite, a "full test" button that runs everything, a "slow flows" button that runs only the heavy ones. Behind each button there is a marker and a -m expression; the panel is the configuration that connects and labels them so anyone —not only whoever set up the workshop— can operate the suite without knowing the internal mechanics. Marking the tests is wiring the buttons; writing the contract is labeling them and setting which is pressed by default. In the end, plain pytest is pressing the "quick check" button, and the workshop stopped being mute: it tells you how it is operated, and it operates the same for everyone.

The starting point: the layered suite, unmarked

This is the suite you receive —the one of module 3, grown to thirteen tests, in two layers, with a conftest.py per folder but without a single marker and without central config—. The tree:

reservo/                     # the domain (models, pricing, refunds, calendar, service)
tests/
├── conftest.py              # domain: focus, ana, bruno, monday_9am
├── unit/
│   ├── test_overlaps.py     # 2 tests: pure overlaps()
│   ├── test_pricing.py      # 3 tests: pure price_cents()
│   └── test_refund.py       # 3 tests: pure refund_cents()
└── integration/
    ├── conftest.py          # heavy: calendar, service
    ├── test_booking_flow.py # 3 tests: BookingService + Calendar
    └── test_cancel_flow.py  # 2 tests: BookingService + Calendar + refund

The starting state, running the suite as is:

python3 -m pytest -q

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

.............                                                            [100%]
13 passed in 1.24s

Thirteen green in 1.24s. This is the "before": a suite that passes but mute —you cannot cut it by criticality or by cost, there is no net against marker typos (because there are no markers), and there is no file that says how it is run—. The project is turning this into a suite with a control panel, in five steps.

Step 1: design the vocabulary

Before touching code, decide what labels the suite deserves —lesson 3, the librarian's criterion: short, orthogonal, each with its pytest -m—. For Reservo, three:

  • smoke — the critical happy path, the handful you run before deploying. Command: pytest -m smoke. Dimension: criticality.
  • slow — the test whose body takes long, the one you want to be able to exclude while developing. Command: pytest -m "not slow". Dimension: cost.
  • integration — several real pieces together (BookingService + Calendar). Command: pytest -m "integration and smoke" and other combinations. Dimension: nature.

Three orthogonal dimensions, each with a real command. We do not add important (subjective), or fast (redundant with not slow), or unit (duplicates the folder without gaining combinability) —the rejections of exercise 3 of lesson 3—.

Step 2: stick the markers

Now wire the buttons. Where each label goes:

smoke with decorator, on the four critical ones. It is selective within its files (not all the pricing tests are smoke), so it goes with the decorator, one by one. In tests/unit/test_pricing.py:

import pytest

from reservo.pricing import price_cents


@pytest.mark.smoke
def test_basic_member_pays_hourly_rate_times_hours(focus, ana):
    # 2500 * 3 = 7500 cents.
    assert price_cents(focus, ana, 3) == 7500


@pytest.mark.smoke
def test_pro_member_gets_twenty_percent_off(focus, bruno):
    # 7500 - 20% = 6000 cents.
    assert price_cents(focus, bruno, 3) == 6000


def test_zero_hours_costs_nothing(focus, ana):
    # Edge case, NOT critical path: no smoke.
    assert price_cents(focus, ana, 0) == 0

The third test, test_zero_hours_costs_nothing, does not carry smoke on purpose: it is an edge case, not the critical path. A marker is worth for what it excludes. Likewise, in tests/unit/test_refund.py you mark test_full_refund_at_72h with smoke (the full refund is critical) and leave the other two unmarked. And the fourth smoke is integration: test_booking_a_room_charges_the_pro_price.

integration with pytestmark, at module level. The two files of tests/integration/ have all their tests integration, so the label goes with pytestmark —one line that marks the whole file and that new tests inherit on their own—. In tests/integration/test_booking_flow.py:

"""Integration tests of the booking flow: BookingService + Calendar."""
import time
from datetime import timedelta

import pytest

from reservo.service import SlotTakenError

# Every test in this module is integration (module-level mark).
pytestmark = pytest.mark.integration


@pytest.mark.smoke
def test_booking_a_room_charges_the_pro_price(service, focus, bruno, monday_9am):
    booking = service.book(focus, bruno, monday_9am, monday_9am + timedelta(hours=3))
    assert booking.price_cents == 6000
    assert booking.status == "confirmed"


def test_room_is_unavailable_after_it_is_booked(service, calendar, focus, ana, monday_9am):
    start, end = monday_9am, monday_9am + timedelta(hours=3)
    service.book(focus, ana, start, end)
    assert calendar.is_available(focus.id, start, end) is False


@pytest.mark.slow
def test_double_booking_the_same_slot_is_rejected(service, focus, ana, bruno, monday_9am):
    time.sleep(0.4)  # simulates a slow integration flow
    start, end = monday_9am, monday_9am + timedelta(hours=3)
    service.book(focus, ana, start, end)
    with pytest.raises(SlotTakenError):
        service.book(focus, bruno, start, end)

Notice the accumulation: test_booking_a_room_charges_the_pro_price carries integration (inherited from the pytestmark) and smoke (its own decorator); test_double_booking_the_same_slot_is_rejected carries integration and slow. A test can have several labels, and that is what makes the combinations possible (integration and smoke). The other integration file, test_cancel_flow.py, carries the same pytestmark = pytest.mark.integration, and its two cancellation tests additionally carry @pytest.mark.slow (their body takes long).

The final distribution of the labels: 4 smoke (two pricing, one refund, one booking), 3 slow (one booking, two cancellation), 5 integration (the five of the folder, by pytestmark).

Step 3: register the markers and activate strict

With the labels in place, the suite already works but runs with the PytestUnknownMarkWarning on every test —and without a net against typos—. Close it with the contract: create pyproject.toml at the root, register the three markers, and (in step 4) activate --strict-markers. Let us start with the registration and the base options of lesson 5:

# pyproject.toml — Reservo's contract (step 3)
[tool.pytest.ini_options]
testpaths = ["tests"]
markers = [
    "smoke: the critical happy path; run before deploying.",
    "slow: the test takes long; excluded with -m 'not slow' while developing.",
    "integration: assembles several real pieces of Reservo together.",
]

With this, the warnings disappear (the three markers are known) and testpaths sets that plain pytest start with tests/. Verify it with pytest --markers, which now lists your documented vocabulary.

Step 4: set the defaults with addopts

Now the control panel: what plain pytest runs, and what guarantees go always. Add addopts with --strict-markers (the typo net, lesson 4), -ra (useful summary, lesson 6) and -m smoke (the smoke suite by default, lesson 6):

# pyproject.toml — Reservo's complete contract
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-ra --strict-markers -m smoke"
markers = [
    "smoke: the critical happy path; run before deploying.",
    "slow: the test takes long; excluded with -m 'not slow' while developing.",
    "integration: assembles several real pieces of Reservo together.",
]

This is the final contract: seven lines that define how the suite runs for the whole team. Plain pytest runs the smoke one with the strict net on; any other selection is asked for on the command line, and that -m wins over the -m smoke of the contract (lesson 6).

Worked example: the control panel, button by button

Let us verify each button of the panel. First, the default button —plain pytest, which must run the smoke suite—:

python3 -m pytest

What to expect:

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
rootdir: /path/to/reservo
configfile: pyproject.toml
testpaths: tests
collected 13 items / 9 deselected / 4 selected
...
======================= 4 passed, 9 deselected in 0.02s ========================

Read the header as a receipt (lesson 5): configfile: pyproject.toml proves the contract loaded, testpaths: tests that the key applied, and collected 13 items / 9 deselected / 4 selected that the -m smoke of addopts filtered the smoke suite without you typing any -m. Four critical tests, in 0.02s: the "quick check" button works, and it is the simplest thing in the world to press —pytest—.

Second button —the slow ones on demand, pytest -m slow—:

python3 -m pytest -m slow

What to expect:

======================= 3 passed, 10 deselected in 1.22s =======================

Three slow tests, in 1.22s (they ran for real, that is why the long second). Your -m slow beat the -m smoke of addopts —the last one wins—. The "slow flows" button works.

Third button —the complete suite, which with a -m smoke by default you have to ask for explicitly—:

python3 -m pytest -m ""

What to expect:

============================== 13 passed in 1.22s ==============================

Thirteen green: the whole suite. The -m "" (empty expression) filters nothing, so it overrides the -m smoke of addopts and runs everything. This is the "full test" button, and it is the one to document, because it is not obvious that with a smoke default you run everything with -m "". And the development button —pytest -m "not slow", everything but the slow ones—:

python3 -m pytest -m "not slow"

What to expect:

======================= 10 passed, 3 deselected in 0.01s =======================

Ten tests in 0.01s: the fast net you run all the time while programming. Four buttons, four behaviors, all verified by running.

The proof that the net is on

We are missing verifying the silent guarantee: that a marker typo breaks the collection, without anyone typing --strict-markers (it comes from addopts). Simulate the slip —change a @pytest.mark.smoke for @pytest.mark.smoek— and run plain pytest:

python3 -m pytest

What to expect:

collected 10 items / 1 error
==================================== ERRORS ====================================
_________________ ERROR collecting tests/unit/test_pricing.py __________________
'smoek' not found in `markers` configuration option
=========================== short test summary info ============================
ERROR tests/unit/test_pricing.py - Failed: 'smoek' not found in `markers` con...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.06s ===============================

The typo stopped the collection with 'smoek' not found in markers configuration option, and you did not even type --strict-markers —the contract carried it on—. This is the proof that the panel includes its fuse: the same smoek that in the module without config would have thrown a critical test out of the smoke suite silently, here is a wall that does not let you advance. You fix smoek → smoke and the suite runs again.

Your delivery

The project asks you for two deliverables, and each one exercises one half of the module:

  1. The marked suite. Take the layered Reservo suite (the one of this lesson, or the one you assembled in the module-3 mini-project grown to thirteen tests) and stick the three-marker vocabulary on it: smoke with decorator on the four critical ones, slow with decorator on the three that take long, integration with pytestmark on the two files of the layer. The correct distribution is 4 smoke, 3 slow, 5 integration, with accumulation where it fits (the booking flow is integration and smoke; the cancellation ones are integration and slow).
  2. The configuration contract. Write the pyproject.toml with [tool.pytest.ini_options]: testpaths = ["tests"], the three markers registered in markers, and addopts = "-ra --strict-markers -m smoke" so the suite runs the smoke one by default, with the strict net on and the useful summary.

And the verification, which is half the deliverable —a config layer without verification is not finished—. Capture these five runs:

  • pytest → 4 passed, 9 deselected (the default is the smoke suite; header with configfile and testpaths).
  • pytest -m slow → 3 passed, 10 deselected in ~1.2s (the slow ones on demand; the -m from the command line wins).
  • pytest -m "" → 13 passed (the complete suite, overriding the default).
  • pytest -m "not slow" → 10 passed, 3 deselected in ~0.01s (the fast development net).
  • A test smoek → ERROR ... 'smoek' not found (the strict net, without typing --strict-markers).

An honest "done" criterion, in the spirit of the guide: the config layer is ready not when the suite passes —the unmarked suite also passed, 13 passed—, but when the control panel responds: plain pytest runs the smoke one, each -m button cuts what it says, the header proves the contract loaded, and a marker typo breaks the collection instead of hiding a test. If those five runs give what is expected, you set up the panel.

Common mistakes

Registering the markers but forgetting --strict-markers (of half-net). What happens: someone completes step 3 (registering) and considers the contract done, skipping the --strict-markers of addopts. The suite runs without warnings and seems ready, but the typo hole is still open —a smoek would fall silently again—. Why it happens: registering silences the warnings, and "I no longer see warnings" gets confused with "I am now protected" (the first common mistake of lesson 4). How to detect it: put a test smoek and run pytest; if it gives you passed instead of an ERROR ... not found, you are missing --strict-markers in addopts. How to fix it: the contract needs both pieces —markers = [...] and --strict-markers in addopts—. Registering is half; strict is the one that closes the hole.

Marking smoke too much and emptying its meaning (of excess). What happens: when sticking smoke, someone puts it on the three pricing tests (including test_zero_hours_costs_nothing) and on the two refund ones "so as not to leave critical ones out", and ends up with seven or eight smoke. Now plain pytest —which runs the smoke one— executes half the suite, and "smoke" no longer means "the critical handful". Why it happens: it is scary to leave a test without the label, and over-marking feels safe. How to detect it: if pytest (with -m smoke by default) runs more than four or five tests, the smoke suite inflated and lost its edge. How to fix it: smoke is the critical happy path, not "everything important". Four tests in Reservo: basic price, pro price, full refund, booking flow. The edge cases (test_zero_hours_costs_nothing, test_half_refund_at_36h) do not go; a marker is worth for what it excludes.

Delivering without documenting the selection default (of a panel without labels). What happens: the contract ends up with addopts = "-m smoke" working, but no one writes anywhere that pytest runs only the smoke one or how to run everything. The next one who arrives types pytest, sees 4 passed, and believes the suite has four tests —or that their new test does not run (the surprise default of lesson 6)—. Why it happens: the -m smoke of addopts is invisible in the command, so it acts behind the scenes. How to detect it: if someone asks "why does my test not appear?" or "why do only four run?", the default is not documented. How to fix it: label the panel. A comment in the pyproject.toml and a note in the README: "pytest runs the smoke suite; use pytest -m '' for the complete suite, pytest -m slow for the slow ones". A control panel without labels confuses whoever did not wire it.

Exercises

Exercise 1 — Predict the complete panel. With the final contract (addopts = "-ra --strict-markers -m smoke", and the suite marked 4 smoke / 3 slow / 5 integration), predict how many tests run and the count line of each command. (a) pytest. (b) pytest -m integration. (c) pytest -m "integration and smoke". (d) pytest tests/unit. (e) pytest -m "".

See solution
  • (a) pytest → 4 run. The -m smoke of addopts applies; the four smoke ones run. 4 passed, 9 deselected.
  • (b) pytest -m integration → 5 run. Your -m integration wins over the -m smoke of the contract; the five of the integration folder run (marked by pytestmark). 5 passed, 8 deselected in ~1.2s (three of them are slow).
  • (c) pytest -m "integration and smoke" → 1 runs. The intersection: only test_booking_a_room_charges_the_pro_price is integration and smoke. 1 passed, 12 deselected.
  • (d) pytest tests/unit → 3 run. Here you did not type a -m, so the -m smoke of addopts is still in force; you passed a path, which adds to the filter. Pytest collects tests/unit and from there selects the smoke ones: the three unit smoke ones (basic, pro, full_refund). 3 passed. (Subtle point of lesson 6: a path does not erase the -m of addopts; only a new -m does.)
  • (e) pytest -m "" → 13 run. The empty expression filters nothing and overrides the -m smoke; it runs the whole suite. 13 passed in ~1.2s.

The key is in (d): passing a path is not passing a -m, so the selection default keeps acting over that path. Combining path and default is a real case (running the smoke ones of a folder) that surprises if you do not remember how addopts composes.

Exercise 2 — Find the badly done marking. A coworker delivers the marked suite, but on verifying, odd counts come out: pytest -m smoke gives 7, and pytest -m "integration and slow" gives 0 (you expected 3). Without seeing their code, what two marking errors explain each anomaly, and how would you confirm them?

See solution
  • -m smoke gives 7 instead of 4 → they marked smoke too much. They probably put @pytest.mark.smoke on tests that are not critical path —the edge cases (test_zero_hours_costs_nothing, test_half_refund_at_36h, test_no_refund_at_12h) or the overlaps ones—, inflating the smoke suite. How to confirm it: pytest -m smoke --collect-only -q lists the seven; you compare against the four correct critical ones (basic, pro, full_refund_72h, booking_charges_pro) and see which are extra. Cure: remove smoke from the ones that are not the critical happy path.
  • -m "integration and slow" gives 0 instead of 3 → the slow ones did not inherit integration. The three slow tests live in the integration folder, but if their file does not have pytestmark = pytest.mark.integration (or the coworker marked slow with a decorator but forgot the module's pytestmark), then they are slow but not integration, so the intersection is empty. How to confirm it: pytest -m integration --collect-only -q —if it gives fewer than 5, the pytestmark is missing in some integration file; if pytest -m slow gives 3 but integration and slow gives 0, the slow ones exist but without the integration label. Cure: add pytestmark = pytest.mark.integration at module level in the two files of tests/integration/.

The general technique: when a count does not check out, --collect-only -q gives you the list of what is selected, and comparing it with the expected you see exactly what test is badly labeled. The counts are the symptom; the list is the diagnosis.

Exercise 3 — Close the module: extend the contract with a new requirement. The Reservo team decides two things: (1) that in CI everything but the slow ones runs (so the pipeline is fast), and (2) that any warning breaks the run (rigor). Answer: (a) Does requirement (1) go in this module's pyproject.toml, or somewhere else? Justify with the module's boundary. (b) Write the line you would add to the contract for requirement (2). (c) With that line, what would happen if someone leaves a @pytest.mark.wip unregistered? (d) Reflect: what did this configuration layer give you that the "raw" marked suite of module 3 did not?

See solution
  • (a) Requirement (1) does NOT go in this pyproject.toml; it is per-environment config, which is module 7. "Run differently in CI than locally" is exactly the per-environment variation that the module's boundary leaves out —here the contract is single, one behavior the same for everyone—. Putting -m "not slow" as default would break the local smoke check; the correct thing is for CI to invoke pytest -m "not slow" in its pipeline configuration (or, in module 7, an --env option that changes the behavior). This module's contract sets the base default; the environment overrides it from outside.
  • (b) The line is filterwarnings in error mode (lesson 5): 
    filterwarnings = ["error"]
  • (c) The collection would fail with an error. With filterwarnings = ["error"], the PytestUnknownMarkWarning of @pytest.mark.wip (unregistered) becomes an error and stops the run —a second path to the same rigor as --strict-markers, but via the warnings route—. The unregistered wip would break just like a typo, forcing you to register it or remove it.
  • (d) What the config layer gave you: a control panel operable by anyone. The raw marked suite of module 3 (well, unmarked yet) passed, but it was mute and fragile: to cut it you had to know the commands by hand, each person ran differently, and a marker typo passed silently. The config layer gave it (1) named cuts —pytest is the smoke one, -m slow the slow ones, without remembering paths—; (2) a single behavior for everyone —the contract in a file, not in each person's memory—; (3) a net against typos —--strict-markers turns the silent slip into a wall—; and (4) living documentation —pytest --markers and the header prove what exists and what loaded—. That is the module's leap: from "thirteen tests that pass" to "a suite with a contract that governs it", operable by anyone, protected against human error, and the same for the whole team.

What you closed is the framework's configuration layer: markers as a second axis, and a central contract that registers them, protects them, and sets how the suite runs. It is one of the four layers of a test framework —together with fixtures (module 2), structure (module 3) and the ones that come—, and the one that turns an ordered suite into a governed suite.

Summary and next step

In this mini-project you gathered the whole module into a real delivery: you took the layered Reservo suite —thirteen tests, unmarked and without config— and mounted its control panel in five steps. You designed the vocabulary (three orthogonal markers, each with its command), stuck the labels (smoke with decorator on the four critical ones, slow on the three that take long, integration with pytestmark on the two files of the layer: 4/3/5 with accumulation where it fits), registered the markers in pyproject.toml, activated --strict-markers, and set addopts = "-ra --strict-markers -m smoke". And you verified it by running each button of the panel: pytest runs the smoke one (4 passed, with the header proving configfile and testpaths), pytest -m slow the slow ones (3 passed in 1.2s), pytest -m "" the complete suite (13 passed), pytest -m "not slow" the fast net (10 passed in 0.01s), and a test smoek breaks the collection ('smoek' not found) without typing --strict-markers, because the contract carries it on. The workshop stopped being mute: it tells you how it is operated, and it operates the same for everyone.

With this you closed the framework's configuration layer: markers as a second categorization axis, and the central file as the contract that registers them, protects them with the strict check, and sets how the suite runs. You now know how to go from a suite ordered in folders to a suite governed by a contract.

What comes next in the guide is module 5: the shared utilities library and the harness. So far you gave structure (module 3) and control (module 4) to the suite, but the tests still repeat logic among themselves —each one assembles its setup, each one writes its same assertions—. Module 5 attacks that duplication: a shared utilities library —helpers, custom assertions like assert_refund(...), the setup/teardown harness— treated as a module of the framework, not as copy-paste. You are going to see the delicate balance of DRY without hidden magic: reusing the common logic without hiding what each test verifies. The foundations —fixtures, structure, markers and config— you already have; what comes is the shared-code layer the tests use so as not to repeat themselves.

Resources

← Previous7. Selecting with `-m` expressionsNext →1. Module introduction: the support-code library