5. Marker gates and smoke tests

Description

So far, every gate looked at coverage: a percentage against a threshold. But in lesson 2 you established that coverage isn't special —any metric with a threshold and a consequence is a gate—. This lesson demonstrates it by changing the metric: from "what percentage of the code ran?" to "did this subset of critical tests pass?". It's the marker gate, and its most common case is the smoke gate: a CI job that runs only the tests marked smoke —the few that verify the essentials work— and requires them to pass before allowing any merge.

By the end you'll know how to set up a marker gate from start to finish: registering a marker (smoke) in pytest's configuration so it's official, marking Reservo's anchor tests with @pytest.mark.smoke, and selecting them with pytest -m smoke so the gate runs only that group. You're going to see it executed for real: pytest -m smoke selects 3 tests and deselects 10 (exit 0, green); and when someone breaks the pro-discount rule —the heart of Reservo's business— the smoke gate turns red (exit 1, assert 5625 == 6000) and would block the merge. And you're going to understand the pattern this enables: staged gates —the fast smoke first, the complete suite after— so the feedback on the critical stuff arrives in seconds, not minutes.

Connection to the module: this lesson moves the metric knob (lesson 2): the same recipe —metric, threshold, consequence via exit code— with "coverage ≥ 80%" replaced by "the smoke pass". It complements the coverage gates of lessons 3 and 4: coverage and markers answer different questions —"how much code did I test?" vs "does the most important stuff still work?"— and in a mature pipeline they coexist. It also prepares lesson 6 (when a gate helps), because the smoke is the clearest example of a gate that pays off: cheap, fast, and protecting exactly what can't break.

The "does it start and brake?" check before the full inspection

When you take the car to the shop for the complete annual inspection —which takes hours: oil, filters, brakes, suspension, emissions—, the mechanic doesn't start with the slowest part. Before putting it up on the lift, they do a thirty-second check: does the engine start? does it brake? do the wheels turn? If the car doesn't even start, there's no point checking the suspension: there's a problem so basic that the full inspection would be a waste of time. That quick check of the essentials —"does the minimum work?"— filters out the fundamentally broken cases before investing in the deep analysis.

In aviation and engineering, that minimum check is called a smoke test, and the name has a literal history: when a new electronic device was tested, the first thing was to turn it on and see if smoke came out. If smoke came out, something was so wrong it wasn't worth testing anything else; you had to turn off and fix. If no smoke came out, at least the basics were standing and you could proceed with the detailed tests. The smoke test doesn't verify everything is perfect; it verifies the essentials aren't catastrophically broken.

A smoke gate in your CI is that "does it start and brake?" check. Of your whole suite, you mark the few tests that verify the essentials of Reservo —that the basic price gives 7500, that the pro discount gives 6000, that a full refund returns everything— and you set up a gate that runs only those and requires them to pass. If the pro price breaks, the smoke gate catches it in seconds, before running the hundred edge-case tests. It's fast (few tests), it's cheap (seconds, not minutes), and it protects what can never break. It doesn't replace the complete suite —the deep inspection is still necessary—; it's the first line, the filter that catches the obvious disasters immediately.

A smoke gate runs only a subset of critical tests —the ones that verify the essentials work— and requires them to pass before merging. Like the "does it start and brake?" check before the full inspection, it's fast, cheap, and catches the obvious disasters immediately, without waiting for the whole suite.

Step 1: register the marker

A marker in pytest is a label you put on a test to group it or give it an attribute. You already saw one in module 4: @pytest.mark.skipif, which skips a test under a condition. Here we use a custom marker, smoke, to tag the critical tests. But before using it you have to register it, and this step is important for a concrete reason: if you mark a test with @pytest.mark.smoke without registering smoke anywhere, pytest accepts it but throws a warning —PytestUnknownMarkWarning: Unknown pytest.mark.smoke - is this a typo?—, because it doesn't know whether you wrote smoke on purpose or made a typo. Registering the marker tells pytest "yes, smoke is a real marker, I use it deliberately", and the warning disappears.

It's registered in pytest's configuration file, pytest.ini (or in the equivalent section of pyproject.toml):

# pytest.ini
[pytest]
markers =
    smoke: fast, critical tests that must pass before any merge (the smoke gate).

The syntax is name: description. The description isn't decorative: it appears when someone asks which markers exist, and it documents what the marker is for —here, "fast, critical tests that must pass before any merge"—. You can see the registered markers with pytest --markers, which confirms smoke is official:

python -m pytest --markers

@pytest.mark.smoke: fast, critical tests that must pass before any merge (the smoke gate).

There it is, registered and documented. A registered marker is one the team understands; an unregistered one is a loose label that generates warnings and that no one knows if it's intentional or a typo.

Step 2: mark the critical tests

Now we tag the tests that make up Reservo's "does it start and brake?". Which are critical? The ones that verify the business's anchor numbers: the basic price, the pro discount, the full refund. If any of those break, Reservo charges a customer wrong or returns their money wrong —a disaster of the kind that can't reach production—. We don't mark all the tests as smoke; that would turn them into the complete suite and lose the point. We mark the few, essential ones whose failure means "don't continue, there's smoke".

It's marked by putting @pytest.mark.smoke above the test:

# tests/test_pricing.py (extract)
import pytest

from reservo.models import Member, Room
from reservo.pricing import price_cents

focus = Room(id="r-focus", name="Focus", capacity=1, hourly_cents=2500)
basic = Member(id="m-1", name="Ana", tier="basic")
pro = Member(id="m-2", name="Bruno", tier="pro")


@pytest.mark.smoke
def test_basic_three_hours():
    # basic, 3h -> 7500 (essential price, no discount)
    assert price_cents(focus, basic, 3) == 7500


@pytest.mark.smoke
def test_pro_three_hours():
    # pro, 3h -> 6000 (the 20% discount, heart of the business)
    assert price_cents(focus, pro, 3) == 6000


def test_basic_one_hour():
    # basic, 1h -> 2500 (correct, but not critical: it's one more case)
    assert price_cents(focus, basic, 1) == 2500

# tests/test_refunds.py (extract)
@pytest.mark.smoke
def test_full_refund_72h_before():
    # cancel 72h ahead (>= 48h) -> 100% of 6000 = 6000 (customer's money)
    booking = _booking(datetime(2026, 1, 4, 12, 0))
    now = datetime(2026, 1, 1, 12, 0)
    assert refund_cents(booking, 6000, now) == 6000

We mark three tests: the basic price (7500), the pro discount (6000), and the full refund (6000). Notice what we don't mark: test_basic_one_hour (2500) is correct and useful, but it's "one more case", not the heart of the business —it goes in the complete suite, not the smoke—. The decision of what to mark is a judgment: the smoke are the tests whose failure should stop everything immediately, not all the tests you have. A smoke inflated with fifty tests stops being fast and loses its point.

Step 3: the gate runs only the smoke

With the tests marked, the gate is armed with the -m flag (for marker), which selects tests by their marker. pytest -m smoke runs only the tests marked smoke and skips the rest. Let's run it for real, on Python 3.14.0 with pytest 9.1.1, with the -v flag to see each test:

python -m pytest -m smoke -v

What to expect (real output):

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
configfile: pytest.ini
plugins: cov-7.1.0
collecting ... collected 13 items / 10 deselected / 3 selected

tests/test_pricing.py::test_basic_three_hours PASSED                     [ 33%]
tests/test_pricing.py::test_pro_three_hours PASSED                       [ 66%]
tests/test_refunds.py::test_full_refund_72h_before PASSED                [100%]

======================= 3 passed, 10 deselected in 0.02s =======================

Read the key line: collected 13 items / 10 deselected / 3 selected. pytest found Reservo's 13 tests, but -m smoke deselected 10 (the ones without the marker) and selected 3 (the smoke). It ran only those three, all three passed, and the summary confirms it: 3 passed, 10 deselected. The smoke gate is green. The exit code:

python -m pytest -m smoke > /dev/null 2>&1; echo "exit code: $?"

exit code: 0

Exit code 0. Green: the critical tests pass, the merge can proceed (as far as the smoke gate is concerned). And the complement —running everything except the smoke, for the complete suite in another job— is done with pytest -m "not smoke":

python -m pytest -m "not smoke"

collected 13 items / 3 deselected / 10 selected
======================= 10 passed, 3 deselected in 0.01s =======================

There it's inverted: 3 deselected, 10 selected. The three smoke are skipped (the smoke gate already ran them) and the other ten run. Between pytest -m smoke and pytest -m "not smoke", you cover the 13 tests, split into two gates with different purposes.

Step 4: the smoke gate bites

A gate you've only seen green is useless if you don't know it turns red when it should. Let's verify it by breaking the rule one of the smoke protects: the pro discount. In reservo/pricing.py, we change the discount from 20% to 25% —a bug that would undercharge the pro customer— and run the smoke gate:

# after changing PRO_DISCOUNT_PERCENT from 20 to 25 in reservo/pricing.py
python -m pytest -m smoke

What to expect (real output):

collected 13 items / 10 deselected / 3 selected

tests/test_pricing.py::test_pro_three_hours FAILED                       [ 66%]
...
>       assert price_cents(focus, pro, 3) == 6000
E       AssertionError: assert 5625 == 6000
tests/test_pricing.py:20: AssertionError

FAILED tests/test_pricing.py::test_pro_three_hours - AssertionError: assert 5625 == 6000
================== 1 failed, 2 passed, 10 deselected in 0.03s ==================

The smoke gate caught the bug: 1 failed, 2 passed, 10 deselected. test_pro_three_hours failed with assert 5625 == 6000 —with the discount at 25%, the pro would pay 5625 instead of 6000—. The exit code:

python -m pytest -m smoke > /dev/null 2>&1; echo "exit code: $?"

exit code: 1

Exit code 1. Red. In CI, this 1 would block the merge: the "does it start and brake?" check detected that the pro-discount brake doesn't work, and doesn't let the car onto the road. And notice the speed —in 0.03s—: the smoke gate gave its verdict in hundredths of a second, running only 3 tests. It didn't have to wait for the other 10 to tell you "there's a serious problem with the price". We restore the discount to 20% and the gate goes back to green (3 passed, 10 deselected, exit 0), as you verified it bites and heals.

The pattern: staged gates

The smoke gate really shines when you combine it with the complete suite in a staged pipeline. The idea: don't run everything at once; run the fast and critical stuff first, and only if that passes, run the slow and exhaustive stuff. In a CI workflow it would look like this (content, remember: no runner here):

# .github/workflows/tests.yml (extract, staged)
jobs:
  smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-python@v5
        with:
          python-version: "3.14"
      - run: pip install -r requirements.txt
      - name: Smoke gate (critical tests only)
        run: python -m pytest -m smoke

  full-suite:
    needs: smoke          # only runs if the smoke gate passed
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-python@v5
        with:
          python-version: "3.14"
      - run: pip install -r requirements.txt
      - name: Full suite with coverage gate
        run: python -m pytest --cov=reservo --cov-fail-under=80

The key is needs: smoke in the second job: it tells CI "don't run the complete suite until the smoke gate passes". So, if someone breaks the pro price, CI stops at the smoke gate in seconds, without spending the minutes of the complete suite and the version matrix (module 4). It's fast feedback on the critical stuff —you fail early and cheap— and deep analysis only when the basics are standing. The "does it start?" check before putting the car on the lift, translated to CI jobs.

An honest nuance: staging has a cost —the second job waits for the first, so in the happy case (all green) the pipeline is a bit slower than if both ran in parallel—. The trade-off pays off when the critical failures are relatively frequent or the complete suite is expensive (big matrix, many tests): there, catching the disaster at the smoke gate saves a lot. If your complete suite is seconds long, staging might not be worth it and you'd run everything in parallel. Like every CI decision, it depends on your numbers —a topic for lesson 6—.

Common mistakes

Marking too many tests as smoke. What happens: someone marks twenty or thirty tests as smoke "just in case", and the smoke gate takes almost as long as the complete suite. Why it happens: it's tempting to mark everything that "seems important", and almost everything does. How to spot it: if your smoke gate runs in a time comparable to the whole suite's, it stopped being a quick check. How to fix it: the smoke are the "does it start and brake?" —the very few tests whose failure should stop everything immediately—, not a second suite. For Reservo there are three: basic price, pro discount, full refund. If you doubt whether a test is smoke, it probably isn't: the smoke is a brutal filter of the essentials, not a summary of what's important.

Not registering the marker and living with the warning. What happens: someone uses @pytest.mark.smoke without registering it in pytest.ini, and each run spits out PytestUnknownMarkWarning: Unknown pytest.mark.smoke - is this a typo?. Over time, the team ignores the warnings, and the day someone really misspells a marker (@pytest.mark.smoek), the warning that would give it away is lost in the noise. Why it happens: registering the marker is an easy step to skip, and the warning breaks nothing... until it hides a real error. How to spot it: if you see PytestUnknownMarkWarning in your output, you have unregistered markers. How to fix it: register all your markers in pytest.ini with their description. A registered marker generates no noise and documents its purpose; also, pytest --strict-markers can turn an unregistered marker into an error (not just a warning), catching the typos immediately.

Confusing the smoke gate with the complete suite. What happens: a team puts only the smoke gate in CI and believes that with that "the tests run on every push", never running the complete suite. Why it happens: the smoke is fast and green, and gives a false sense of total coverage. How to spot it: ask yourself "if an edge case breaks —an odd overlap, a refund at the exact 48 h limit—, would my CI catch it?". If you only run smoke, the answer is no: the smoke only cover the essentials. How to fix it: the smoke is the first gate, not the only one. Behind it goes the complete suite (with its coverage gate). Staged, not one instead of the other: the smoke catches the disasters fast; the complete suite catches everything else.

Exercises

Exercise 1 — Predict the selection. Reservo's suite has 13 tests, 3 marked smoke. For each command, predict how many tests are selected and how many deselected, and the exit code if all that run pass: (a) pytest -m smoke. (b) pytest -m "not smoke". (c) pytest (without -m). (d) pytest -m smoke with the pro discount broken (20→25).

See solution

• (a) pytest -m smoke: selects 3, deselects 10. Runs only the smoke; if they pass, exit 0. (3 passed, 10 deselected.)
• (b) pytest -m "not smoke": selects 10, deselects 3. Runs everything but the smoke; if they pass, exit 0. (10 passed, 3 deselected.) It's the exact complement of (a).
• (c) pytest without -m: selects 13, deselects 0. Without a marker filter, it runs the whole suite; if everything passes, exit 0. (13 passed.) The marker doesn't change what runs when you don't filter by it.
• (d) pytest -m smoke with the discount broken: selects 3, deselects 10, but now test_pro_three_hours fails (assert 5625 == 6000): 1 failed, 2 passed, 10 deselected, exit 1. The smoke gate bites: it catches the pro-price bug in the 3 critical tests, without running the other 10.

The rule: -m smoke runs only the marked ones, -m "not smoke" runs only the unmarked ones, and without -m it runs them all. The exit code depends on whether the ones that run pass; in (d), since a smoke fails, the gate breaks the build.

Exercise 2 — Choose the smoke of a new feature. Reservo adds cancellation with refund (cancel_with_refund, from the previous lessons). It has tests for: (i) cancelling with a full refund frees the slot, (ii) cancelling twice raises an error, (iii) the email text says "full refund" in the correct tranche, (iv) cancelling an already-past booking. Which one(s) would you mark as smoke and why? Which would you leave only in the complete suite?

See solution

Smoke: (i) cancelling with a full refund frees the slot. It's the feature's essential happy path —what 95% of real cancellations do— and it touches two critical things at once: that the money is returned correctly (anchor number) and that the slot is freed (availability). If this breaks, the cancellation is catastrophically broken; it's exactly a "there's smoke, don't continue".

Complete suite (not smoke): (ii), (iii), and (iv). They're important and should exist, but they're edge or secondary cases, not the heart:

• (ii) cancelling twice raises an error protects against a misuse, not against the main flow. It matters, but its failure isn't a "doesn't start" disaster.
• (iii) the email text is a presentation detail; a wrong text doesn't charge or return money wrong (though it's worth testing, it's not business-critical).
• (iv) cancelling a past booking is an edge case: rare, and its failure doesn't break the normal flow.

The criterion: a smoke is the essential happy path whose failure means "the feature doesn't work at all". (i) is; (ii)-(iv) are the deep inspection. Marking all four as smoke would inflate the gate and make it slow without gaining real protection on the critical stuff. Remember: the smoke is a brutal filter, not a summary.

Exercise 3 — Design the staged pipeline. Your Reservo suite takes: smoke 0.02 s, complete suite with coverage 0.03 s, and a matrix of three Python versions (module 4) that multiplies the complete suite. A teammate proposes: "Let's run smoke, complete suite, and matrix, all three in parallel, to finish as soon as possible." Another says: "Let's stage them: smoke first, and the matrix only if smoke passes." Which is right for Reservo, and would your answer change if the matrix took 12 minutes?

See solution

For Reservo as it stands (everything in hundredths of a second), the first teammate's parallel is right. Staging has a cost: the job that waits (needs: smoke) doesn't start until the previous one finishes, so in the happy case the pipeline is slower. That cost is only justified if what you avoid —running the expensive suite when smoke already failed— is expensive. With a matrix that takes hundredths of a second, there's nothing expensive to avoid: running all three in parallel finishes sooner and the "saving" from staging is nonexistent. Staging here would be paying extra latency without buying anything.

If the matrix took 12 minutes, the answer inverts: staging (the second teammate) wins. Now there is something expensive to avoid. If someone breaks the pro discount, the smoke gate catches it in 0.02 s; staged with needs: smoke, the 12-minute matrix never starts, and you save those 12 minutes (× the matrix cells, × each broken push). The cost of staging —waiting 0.02 s for smoke to pass in the happy case— is negligible against the 12 minutes you save in the broken case. The rule: stage when what goes behind the gate is expensive and the failures the gate catches are frequent; run in parallel when everything is cheap. The number —how long the expensive suite takes— decides, not the reflex of "staging is always better".

Summary and next step

In this lesson you set up a gate that doesn't look at coverage, demonstrating that lesson 2's recipe —metric, threshold, consequence— applies to any metric. The smoke gate runs only a subset of critical tests (Reservo's "does it start and brake?") and requires them to pass before merging. You armed it in three steps: registering the smoke marker in pytest.ini (so it's official and generates no warnings), marking the anchor tests with @pytest.mark.smoke (basic price, pro discount, full refund —the few essential ones, not all—), and selecting them with pytest -m smoke.

You saw it executed for real: pytest -m smoke selected 3 tests and deselected 10 (exit 0, green); and when you broke the pro discount (20→25), the smoke gate turned red in 0.03 s —1 failed, 2 passed, 10 deselected, assert 5625 == 6000, exit 1— catching the disaster without waiting for the other 10 tests. And you saw the pattern it enables: staged gates with needs: smoke, the fast smoke first and the complete suite (with its coverage gate) after, for fast feedback on the critical stuff —with the honest nuance that staging pays off only when what goes behind is expensive—.

Before moving on you should be able to: register and mark a test with a custom marker; run only a subset with pytest -m smoke and its complement with -m "not smoke"; explain why a smoke should be few essential tests; and design a staged pipeline with needs, knowing when staging pays off and when it doesn't.

What's next, in lesson 6, is taking a step back from the "how" to the "when": the judgment. You already know how to set up coverage and marker gates; the question now is when a gate helps —it stops erosion, makes a decision explicit, protects the critical stuff— and when it gets in the way —a too-high threshold that blocks legitimate work, a metric that doesn't measure what matters—. It's the lesson that turns you into someone who decides which gates to set, not just someone who knows how to set them.

Resources

• Markers in pytest: how to register and use them — the reference for @pytest.mark, how to register them in pytest.ini, and --strict-markers to turn an unregistered marker into an error. The basis of the marker gate.
• Selecting tests by marker with -m — how pytest -m smoke and pytest -m "not smoke" filter the suite, with examples of more complex marker expressions (-m "smoke and not slow"). The flag that arms the gate.
• Job dependencies with needs — GitHub Actions — how needs: smoke makes one job wait for another, the piece that arms the staged pipeline. Read it for the mini-project.
• Smoke testing (concept) — the marker is the tool; the idea of the smoke test —first testing that the essentials aren't catastrophically broken— is older than pytest and applies in any stack. Think about which three tests would be your own project's "does it start?".