5. Splitting the suite: fast and slow

Description

Lesson 4's parallelism distributes all the tests across workers, and that already speeds things up a lot. But there's an earlier, finer optimization that parallelism doesn't replace: not all tests are worth the same or cost the same. In Reservo, eleven tests fly —prices, refunds, availability, all in hundredths of a second— and only twelve weigh: the monthly-report ones, with their half a second of wait each. When you run the whole suite, those twelve slow ones dominate the clock and make you wait six seconds for a verdict that, for the other eleven, was ready in 0.01. Splitting the suite fixes that: you separate the fast from the slow and decide what runs first and what runs separately.

By the end you'll know how to mark the slow tests with @pytest.mark.slow, select them with -m "not slow" (for fast feedback: only the fast ones, in hundredths) and -m slow (for the expensive part, separately), and find the culprits of the slowness with --durations, which lists the slowest tests by real time. You'll see the two-speed strategy in CI —a fast-tests job that gives a verdict in seconds and a slow-tests job that runs in parallel— and you'll peek at the idea of distributing the suite across several jobs (sharding). All of this runs for real: the times and counts come from running the Reservo suite locally.

Connection to the module: this lesson organizes what lesson 4 parallelizes. Marking and splitting the suite is what lets you apply parallelism where it hurts (the slow ones) and instant feedback where it matters (the fast ones, first). It leans on the @pytest.mark.slow marker Reservo already carried in test_reports.py since lesson 1, and prepares the ground for lesson 6, which will explain why those tests, to be distributed, must be isolated. A boundary note: here we split by speed (fast/slow) to speed up; splitting by type of test (unit, integration) as a strategy decision is a topic for another guide. Our criterion is one: time.

The supermarket express lane

In any supermarket there's a register with a sign: "10 items or fewer". It doesn't exist by whim: it exists because mixing in a single line the person with three things and the person with a full cart punishes the first. The one with three things would wait twenty minutes behind three carts, for a checkout that takes fifteen seconds. Separating the flows —an express lane for few items, the normal registers for big shops— makes the fast feedback arrive fast, without the heavy shop getting in the way.

Your test suite has exactly that mix. The fast tests —Reservo's unit ones, which verify a pricing formula or a refund calculation— are "three items": they should give you a verdict in a blink. The slow tests —the monthly-report ones, which wait for a data source— are "the full cart": they take time, and it's fine that they take time, but they shouldn't make the fast ones wait. When you run everything together, you put the three items behind the cart: you wait six seconds for a result that, for the eleven fast ones, was in 0.01.

Splitting the suite is putting up the "express lane" sign. You mark which tests are the full cart (@pytest.mark.slow), and from there you can run only the fast ones when you want immediate feedback —while coding, on every save— and leave the slow ones for a separate run —complete, in parallel, when it's time—. The urgent verdict arrives urgently; the heavy one, when it can.

Not all tests cost the same. Marking the slow ones and separating them lets you run the fast ones first (feedback in hundredths) and the slow ones separately (in parallel, when it's time). The express lane doesn't delete the heavy shop: it puts it in its own line.

Marking the slow tests

A pytest marker is a label you put on a test with a decorator. Reservo already marked its twelve slow tests in test_reports.py since lesson 1:

import pytest


@pytest.mark.slow
def test_report_january_basic_hours():
    total = _slow_report_total([price_cents(ROOM, BASIC, 3) for _ in range(4)])
    assert total == 30000

The @pytest.mark.slow on top doesn't change what the test does: it still verifies that four basic 3 h bookings sum to 30000 cents. What it does is stick a label —"this test is slow"— that you can later use to select or exclude it. slow isn't a magic pytest word; it's a name we chose. It could be called heavy or integration; we use slow because it precisely describes why we separate it: by its time.

Custom markers should be registered, so pytest knows they exist and doesn't warn you of a possible typo. Reservo registers it in pyproject.toml:

[tool.pytest.ini_options]
markers = [
    "slow: marks tests that take a noticeable amount of time (deselect with '-m \"not slow\"')",
]

Registering the marker has two benefits. One: if you write @pytest.mark.slwo (with a typo), pytest warns you "unknown marker slwo", instead of silently creating a new label that will select nothing. Two: pytest --markers lists your markers with their description, so anyone on the team sees what slow means and how to exclude it. The description in quotes is living documentation.

Selecting by marker: -m

With the tests marked, the -m flag filters the suite by marker. It's an expression, so you can ask for a marker, its negation, or combinations.

Only the fast ones —the express lane, everything but the slow— with -m "not slow":

python -m pytest -m "not slow"

What to expect (real output):

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
rootdir: /private/tmp/reservo-m5
configfile: pyproject.toml
testpaths: tests
plugins: xdist-3.8.0
collected 23 items / 12 deselected / 11 selected

tests/test_availability.py ....                                          [ 36%]
tests/test_pricing.py ....                                               [ 72%]
tests/test_refunds.py ...                                                [100%]

====================== 11 passed, 12 deselected in 0.01s =======================

Read it slowly: 11 passed, 12 deselected in 0.01s. pytest collected the 23 tests, deselected the 12 marked slow, and ran only the 11 fast ones —in a hundredth of a second—. There's the express lane: while coding, you run -m "not slow" and know in a blink whether you broke a pricing or refund rule, without waiting the six seconds of the report. deselected means "they exist, but this time we didn't run them" —they didn't fail, they weren't skipped by a condition like module 4's skipif; we simply left them out of the filter—.

Only the slow ones —the heavy shop, separately— with -m slow:

python -m pytest -m slow

====================== 12 passed, 11 deselected in 6.11s =======================

The twelve slow ones, without the fast ones. This is the expensive run, the one you want to parallelize. Combined with lesson 4's -n auto, those 6.11 s drop to 1.17 s. And that's exactly the point: you parallelize the slow part, because that's where parallelism pays off; the fast part is already at 0.01 s and doesn't need workers.

Finding the culprits: --durations

Before marking something as slow, it's worth measuring who's really slow, instead of guessing. The --durations=N flag tells pytest "when you finish, show me the N slowest tests with their time". It's the sink-2 meter lesson 2 promised.

python -m pytest --durations=8

What to expect (real output, trimmed to the durations block):

============================= slowest 8 durations ==============================
0.51s call     tests/test_reports.py::test_report_january_basic_hours
0.51s call     tests/test_reports.py::test_report_october_basic_two_hours
0.51s call     tests/test_reports.py::test_report_november_pro_two_hours
0.51s call     tests/test_reports.py::test_report_december_year_end
0.51s call     tests/test_reports.py::test_report_september_pro_hour
0.51s call     tests/test_reports.py::test_report_april_single_basic
0.51s call     tests/test_reports.py::test_report_july_two_pro
0.51s call     tests/test_reports.py::test_report_june_basic_long
23 passed in 6.10s

The list is unmistakable: the eight slowest tests are all from test_reports.py, each with 0.51s call (the half second of the sleep, plus a smidge). The call indicates the time went into executing the test (there's also setup and teardown, which here are negligible). If you had a slow test hidden among the "fast" ones, --durations would give it away at once, and you'd know which one to stick the @pytest.mark.slow on. The module's rule, again: measure before optimizing. --durations is how you measure.

The two-speed strategy in CI

Splitting the suite isn't just local comfort; it's a structure for the pipeline. The idea: two jobs with distinct purposes.

A fast job that runs only the fast tests and gives an almost instant verdict:

- name: Fast tests
  run: pytest -m "not slow"

And a slow job that runs the heavy ones, in parallel:

- name: Slow tests
  run: pytest -m slow -n auto

What does the team gain with this? Layered feedback. The fast job finishes in seconds and already tells you if you broke something basic —a pricing formula, a refund calculation—; you don't have to wait for the monthly report to run to find out you broke price_cents. The slow job runs in parallel with -n auto and gives the complete verdict a bit later. If the fast one fails, you know immediately and there's no need to wait for the slow one. It's the supermarket express lane applied to CI: the urgent result arrives urgently, the heavy one when it can, and both run without getting in each other's way.

An honesty detail: splitting into two jobs also lets each cache and prepare its environment separately, and lets the heavy one use a runner with more cores if it really needs it —that's already fine-tuning lesson 7's trade-off—. The basic structure, however, is this: separate by speed, run the fast one first, parallelize the slow one.

Distributing the suite across jobs: sharding, briefly

There's a second way to split, complementary, worth naming even though its detail exceeds this lesson. Splitting by speed (fast/slow) organizes by cost. But if you have thousands of tests all of similar cost, another tactic is to split them into chunks (shards) and give a chunk to each CI job, which run on different machines at once. It's parallelism at the level of CI jobs, not processes on one machine: instead of -n auto distributing across one runner's cores, you have four runners running a quarter of the suite each.

Module 4's matrix already gave you the mechanics to launch several jobs at once; sharding uses it to distribute the suite instead of repeating it across several versions. Tools like pytest-split distribute the tests into chunks of even duration based on prior measurements. We don't develop it here —it's a technique for very large suites, and Reservo doesn't need it—, but keep the idea: there are two levels of parallelism, within a machine (xdist, -n) and across CI machines (job sharding), and they combine. For most projects, -n auto plus the fast/slow split is more than enough.

Common mistakes

Not registering the marker and losing the typo warning. What happens: someone uses @pytest.mark.slow without registering it in pyproject.toml, one day writes @pytest.mark.slow (typo), and that test isn't marked as slow —but pytest doesn't warn, because it accepts any marker—. The slow test sneaks into the express lane and slows it down. Why it happens: pytest, without --strict-markers or registration, creates new markers silently. How to spot it: run pytest --markers and verify your markers are there; a typo won't appear. How to fix it: register the markers in pyproject.toml (like Reservo) so pytest warns you of an unknown name instead of accepting it quietly.

Confusing deselected with skipped or with a failure. What happens: someone sees 12 deselected and worries —"did twelve tests not run? is something broken?"—. Why it happens: deselected is a state distinct from passed, failed, and skipped, and isn't always explained. How to spot it and understand it: deselected means "these tests exist but your -m filter left them out on purpose", like asking for "10 items or fewer" and the big shops not entering that lane —they didn't fail or skip by a condition, they simply weren't what you asked for—. How to act: if you wanted to run them, remove or change the filter; if not, deselected is exactly what you wanted and there's nothing to fix.

Guessing what's slow instead of measuring it. What happens: someone marks as slow the tests that "seem" heavy —the ones with long names, or from the module they don't like— and leaves unmarked a test that really takes three seconds hidden among the "fast" ones, which keeps slowing down the express lane. Why it happens: intuition about what's slow usually fails. How to spot it: run --durations=10 and compare the real list with your marks; if a slow test isn't marked, or a marked one is actually instant, your criterion doesn't match the data. How to fix it: mark based on what --durations measures, not on what you assume. The meter rules.

Exercises

Exercise 1 — Choose the command. For each goal, write the pytest command that fulfills it on the Reservo suite. (a) Instant feedback while coding: only the fast tests. (b) The expensive reports run, parallelized. (c) See the five slowest tests with their time. (d) The complete suite in parallel (fast and slow together).

Exercise 2 — Design the two jobs. Write the two CI steps of the two-speed strategy for Reservo —a fast job and a slow one— and explain in one sentence what the team gains with this separation instead of a single pytest.

- name: Fast tests
  run: pytest -m "not slow"

- name: Slow tests
  run: pytest -m slow -n auto

Exercise 3 — Read the --durations and act. You run pytest --durations=5 on a new project and see this. Which tests would you mark as slow and why, and what would you do next with the slow part?

============================= slowest 5 durations ==============================
2.10s call     tests/test_email.py::test_sends_welcome_email
1.95s call     tests/test_export.py::test_generates_pdf_report
0.88s call     tests/test_email.py::test_sends_reminder
0.01s call     tests/test_pricing.py::test_basic_rate
0.01s call     tests/test_pricing.py::test_pro_rate

Summary and next step

In this lesson you learned to split the suite by speed, so the urgent feedback arrives fast and the expensive part runs separately. You saw it with the supermarket express lane: separating the three items from the full cart so the first doesn't wait behind the second. You marked the slow tests with @pytest.mark.slow (registered in pyproject.toml so pytest warns of typos), selected them with -m "not slow" for the express lane —11 passed, 12 deselected in 0.01s, feedback in a hundredth— and with -m slow for the expensive run, which combined with -n auto drops from 6.11 s to a little over one. And you measured the culprits with --durations, which pointed to the twelve test_reports.py tests without you having to guess.

You also saw the structure for CI: the two-speed strategy —a fast job that gives a verdict in seconds and a slow one in parallel—, and you peeked at sharding, the second level of parallelism (across CI machines, not within one), which combines with -n for enormous suites. The rule running through it all: measure with --durations, mark based on the data, run the fast ones first, and parallelize the slow ones.

Before moving on you should be able to: mark and register a slow marker; select with -m "not slow" and -m slow; distinguish deselected from skipped and from a failure; use --durations to find slow tests; and design the two-job strategy for a pipeline.

What's next, in lesson 6, is the condition that makes all of lesson 4's parallelism and this splitting possible: isolation. Distributing tests across workers only works if each test is independent —if it doesn't depend on what another test left behind—. You're going to see, executed for real, a test with shared state that passes serially (3 passed) and fails under -n (2 failed), because each worker is a process with its own memory; and you're going to fix it with a fixture that gives each test its own world. Isolation isn't a luxury of parallelism: it's its precondition.

Resources

• How to mark tests with attributes — pytest documentation — the official reference for markers: how to apply them, register them, and use --strict-markers. The basis of splitting by speed.
• Working with custom markers — pytest documentation — examples of -m with expressions (not slow, slow and reports), exactly the selection we used here. Useful for finer combinations.
• Profiling test execution duration — pytest documentation — the --durations section, the meter with which you find the slow tests before marking them. The "measure before optimizing" made a command.
• pytest-split on PyPI — the tool that distributes the suite into chunks of even duration for sharding across CI jobs. For when a suite grows beyond what -n auto reaches on a single machine.