Module 2: Your First Pipeline Pytest In Ci
8. Mini-project: the `tests.yml` that runs the Reservo suite
Description
This is the module's capstone. In the previous seven lessons you were gathering pieces —the anatomy of the YAML, the triggers, the steps that prepare the ground, installing dependencies, running pytest with its exit code, and reading the log—. Now you use them all together, you, from start to finish, to produce something deliverable: the complete tests.yml that runs Reservo's suite on every push, plus the proof that it does what it says. You're not going to learn a new concept; you're going to demonstrate that you already know how to set up a project's first pipeline.
The deliverable has three parts, and we build them together: (1) the complete workflow, with named steps, written and explained decision by decision; (2) local parity —you run on your machine, for real, the same commands CI would execute, and paste the real output to verify that the pipeline and your terminal do the same—; and (3) a short note of what your pipeline covers and what's explicitly left for the following modules. That last part —saying what your workflow doesn't do yet and why— is as important as the YAML: knowing the scope of what you set up is part of the craft.
Connection to the module: this lesson introduces nothing; it integrates. Each decision you make here —which events in on:, which version in setup-python, why install from requirements.txt, how to name the steps— comes from a previous lesson, and the idea is that you apply them without being reminded. It's also the bridge to the rest of the guide: the "what's left for later" note points straight to module 3 (reproducing a CI failure, deterministic environments), module 4 (the version matrix), module 5 (cache and speed), and module 6 (coverage gates). Here you set up the base pipeline; those modules make it more robust, faster, and more demanding.
The practical driving exam, not the written one
Think of getting your driver's license. There's a written exam —they ask you what a sign means, at what distance you brake— and there's a practical exam, where you get in the car and actually drive with the instructor beside you. Both matter, but they're different: the written one proves you know the rules; the practical one proves you can apply them all at once, without anyone telling you which one applies at each moment.
Lessons 1 to 7 were the written exam: each taught you a piece and tested it separately. This mini-project is the practical exam: you get in the car and write a real workflow, making yourself the decisions that were previously given to you. Which events trigger? Which Python version? Do I install by hand or from the file? Do I name the steps? No one tells you; you decide with what you learned. And as in the practical exam, the goal isn't theoretical perfection but real competence: at the end, having a tests.yml that runs Reservo's suite on every push, that reads well, and that —verified with local parity— does exactly what it says.
The project you're going to protect
Let's recall what Reservo has, because the workflow is built around its structure. It's a pure Python project (meeting-room booking logic, money in int cents) with its code in a reservo/ package and its suite split across three test files:
reservo/ ← the domain code
├── models.py (Room, Member, Booking with price_cents)
├── pricing.py (price_cents)
├── refunds.py (refund_cents)
├── calendar.py (Calendar)
└── schedule.py (overlaps, is_available, book, cancel)
test_pricing.py ← 4 pricing tests
test_refunds.py ← 3 refund tests
test_availability.py ← 4 overlap and availability tests
requirements.txt ← the dependency list
Its requirements.txt, which you already know from lesson 5, is one line —Reservo only needs pytest to test itself—:
# requirements.txt
pytest==9.1.1
Your job is to write the workflow that, on every push, starts a clean machine and runs those eleven tests.
Step 1: write the complete workflow
Create the file at the exact path —.github/workflows/tests.yml— and write this, with named steps so the log reads well (lesson 7):
# .github/workflows/tests.yml
name: tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Check out the code
uses: actions/checkout@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.14"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Run the test suite
run: pytest
Review each decision, all from the previous lessons, because they're the ones you'd make in a real project:
name: tests(lesson 2) — the label in the Actions tab. Descriptive and stable.on: [push, pull_request](lesson 3) — the standard pair: early feedback on every push to your branch, and threshold guardian on every pull request towardmain. Both doorbells.runs-on: ubuntu-latest(lesson 2) — a clean Linux machine; enough and cheap for a pure-Python suite.Check out the codewithactions/checkout@v5(lesson 4) — first of all, because nothing can be done without the code on the runner. Remember to pin the major and check the current one on the action's page.Set up Pythonwithpython-version: "3.14"(lesson 4) — the guide's version, in quotes so YAML doesn't read it as a number. It goes before installing, so pip and pytest run on this Python.Install dependencies(lesson 5) — the|groups the two commands: upgrade pip (hygiene) and install fromrequirements.txt(a single source of truth, pinned versions).Run the test suitewithpytest(lesson 6) — the heart: it discovers and runs the eleven tests the same as locally, and its exit code paints the job.
That's the whole workflow. Fourteen lines of content that turn "I run the tests when I remember" into "the tests run on their own on every change". Remember the module's rule: this file is content you wrote and understand; we're not going to spin up a runner. What we are going to do —and it's the more valuable half of the deliverable— is prove locally that it does what it says.
Step 2: local parity (this runs for real)
Here's the idea that holds up the whole module: what CI does to your suite is the same thing your machine does. You can verify it by running, in your terminal, exactly the same steps the workflow would run on the runner. If you get the same green output, you have direct evidence that your pipeline will do the right thing. This is called local parity: running locally the same as CI, so as not to have surprises.
Let's follow the workflow's steps, one by one, locally. Each command below was executed for real with Python 3.14.0 and pytest 9.1.1; the outputs are real.
The Check out the code step locally is, simply, standing in your already-cloned project (you already have the code; the runner brings it, you have it). There's no command to run.
The Set up Python step locally is having the correct Python active. You verify it and, as in the fundamentals guide, you work in a clean virtual environment:
python3 -m venv .venv
source .venv/bin/activate
python3 --version
Python 3.14.0
Same Python that setup-python: "3.14" would leave on the runner. Version parity: confirmed.
The Install dependencies step locally is the two same commands from the run::
python3 -m pip install --upgrade pip
pip install -r requirements.txt
What to expect (real output of the installation from requirements.txt):
Collecting pytest==9.1.1 (from -r requirements.txt (line 1))
Using cached pytest-9.1.1-py3-none-any.whl.metadata (7.6 kB)
Collecting iniconfig>=1.0.1 (from pytest==9.1.1->-r requirements.txt (line 1))
Collecting packaging>=22 (from pytest==9.1.1->-r requirements.txt (line 1))
Collecting pluggy<2,>=1.5 (from pytest==9.1.1->-r requirements.txt (line 1))
Collecting pygments>=2.7.2 (from pytest==9.1.1->-r requirements.txt (line 1))
Installing collected packages: pygments, pluggy, packaging, iniconfig, pytest
Successfully installed iniconfig-2.3.0 packaging-26.2 pluggy-1.6.0 pygments-2.20.0 pytest-9.1.1
Successfully installed ... pytest-9.1.1: the same dependencies the runner would install from the same list. Dependency parity: confirmed.
The Run the test suite step locally is the workflow's pytest (in your terminal, the unambiguous form is python3 -m pytest):
python3 -m pytest
What to expect (real output of Reservo's suite):
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
rootdir: /home/ana/reservo
collected 11 items
test_availability.py .... [ 36%]
test_pricing.py .... [ 72%]
test_refunds.py ... [100%]
============================== 11 passed in 0.03s ==============================
And the exit code CI would read to paint the job:
python3 -m pytest -q > /dev/null; echo "exit code: $?"
exit code: 0
There's the complete parity. The only difference between this run and the runner's is the platform line: here darwin (macOS), on the runner linux (Ubuntu). Everything else —Python 3.14.0, pytest 9.1.1, collected 11 items, 11 passed, exit code 0— is identical, because it's the same command on the same suite with the same dependencies. That match is the proof that your tests.yml will do the right thing: you just ran, with your hands, the same thing CI will run on its own. (That platform difference, darwin vs linux, is exactly the kind of detail that sometimes makes a test pass on one side and fail on the other —and reproducing and closing that gap is module 3—.)
Step 3: verify the pipeline bites (the intentional red)
A pipeline you've only seen green has a silent problem: you don't know if it would turn red when it should. Just as in the fundamentals guide you verified that your tests bite by breaking the code on purpose, here you verify that your pipeline bites. Break a Reservo rule —the pro discount from 20% to 25% in reservo/pricing.py— and run the same commands. What to expect (real):
test_pricing.py .F.. [ 72%]
...
=========================== short test summary info ============================
FAILED test_pricing.py::test_pro_member_gets_twenty_percent_off_the_subtotal - AssertionError: assert 5625 == 6000
========================= 1 failed, 10 passed in 0.03s =========================
python3 -m pytest > /dev/null 2>&1; echo "exit code: $?"
exit code: 1
Exit code 1: on the runner, that number would paint the Run the test suite step red with a ✗, and the whole job red (lessons 6 and 7). You confirmed that your pipeline not only passes when everything is fine, but also fails when something breaks —which is the whole point of having it—. Now restore the code (back to 20%) and confirm it returns to green:
python3 -m pytest -q > /dev/null; echo "exit code: $?"
exit code: 0
Back to 0. This cycle —green gives 0, red gives 1, I restore, it goes back to 0— is the CI machine verified in your own terminal. A pipeline you never saw turn red is a pipeline you shouldn't fully trust.
Step 4: hang the badge
With the workflow in place, add the badge to the README so the suite's health is visible at a glance (lesson 7). Assuming your user ana-dev and repo reservo:
[](https://github.com/ana-dev/reservo/actions/workflows/tests.yml)
Rendered, it will show a little tests | passing label in green that updates on its own with each run on main, linked to the workflow's runs tab. It's the project's public state front page.
Step 5: the scope note
The last deliverable isn't code: it's a short note —three or four lines— of what your pipeline does and what it doesn't do yet, on purpose. Knowing the scope of what you set up, and saying it, keeps someone (or you, in a month) from believing the pipeline covers more than it does. An example note:
Scope of
tests.yml. Runs Reservo's complete suite (11 tests: prices, refunds, availability) on every push and every pull request, on Python 3.14 on Ubuntu, installing fromrequirements.txt. The job turns red if any test fails (exit code ≠ 0). Left for following modules: reproducing locally a failure that only happens in CI and pinning deterministic environments (module 3); running across several Python versions and operating systems with a matrix (module 4); speeding up with dependency caching and parallelism (module 5); and requiring a coverage threshold that breaks the build (module 6). This pipeline is the floor, not the ceiling.
Notice what that note does: it acknowledges that the pipeline is the simplest one that works —one Python version, one operating system, no cache or coverage gates— and says why it's fine for it to be so for now (those improvements come in specific modules). That's engineering honesty. A pipeline presented as "complete" when it's the basic one deceives; one that says "it does this, I left that for later, for this reason" is reliable and makes the improvement path clear.
Common mistakes
Delivering the workflow without having run anything locally (unverified pipeline). What happens: someone writes a tests.yml that looks correct, pushes it, and discovers in the real CI that something doesn't add up —a misspelled version, an incomplete requirements.txt— that a local run would have revealed in seconds. Why it happens: the YAML "looks fine" and you trust the look. How to spot it: if you haven't run the workflow's commands on your machine, you don't know if they work. How to fix it: always do step 2's local parity —the same commands, in your terminal, with their green output— before trusting the pipeline. CI isn't the place to discover that your requirements.txt was wrong; your terminal is.
Presenting the basic pipeline as "complete CI" (false scope). What happens: someone sets up this one-version, one-OS workflow and announces "I have CI now, everything's covered", when it's missing the matrix, the cache, and the coverage gates. Why it happens: setting up the base pipeline feels like finishing, because it's the big leap (from nothing to something). How to spot it: try to list what your pipeline doesn't do; if you come up with several things in thirty seconds (does it run on another Python version?, does it require coverage?, does it cache?), it's not complete. How to fix it: write step 5's scope note. Naming what's missing isn't admitting a failure; it's making the improvement map clear, and those improvements are literally the following modules.
Copying the YAML without understanding each step, and being helpless when it fails (blind copy). What happens: someone pastes a tests.yml from the internet, it works by luck, and the day it turns red for an infrastructure reason (an absent checkout, a misplaced version) they don't even know where to start. Why it happens: copying is faster than understanding, until you have to fix. How to spot it: if you can't explain what each step does and why it's in that order, you copied without understanding. How to fix it: for each step of your workflow, be able to say what it prepares and what would happen without it —exactly what we reviewed in step 1—. A workflow you understand is one you can fix; one you copied is a black box that will block you the day it fails.
Exercises
Exercise 1 — Detect the three defects. A teammate gives you this tests.yml "that isn't working well". It has three problems from what you learned in the module. Find and fix them.
name: tests
on: [push]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-python@v5
with:
python-version: 3.10
- run: pip install -r requirements.txt
- run: pytest
See solution
The three defects:
- The
checkoutis missing. There's noactions/checkout, so the code never reaches the runner:pip install -r requirements.txtwon't find the file andpytestwon't find tests (exit code 5). It's the most serious defect. It's fixed by adding the checkout as the first step. python-version: 3.10without quotes. YAML reads it as the number 3.1 (the trailing zero is lost), asking for Python 3.1 instead of 3.10. It's fixed with quotes:python-version: "3.10".- The pip upgrade / install hygiene is missing, and —more subtly— the
on:only haspush, withoutpull_request, so it loses the threshold guardian on pull requests. (Depending on how you count, the "third defect" is either of the two; both count.)
Fixed:
name: tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Check out the code
uses: actions/checkout@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Run the test suite
run: pytest
The most important one to detect is the absent checkout: without it, the pipeline tests nothing and (thanks to pytest's exit code 5) at least turns red instead of lying green.
Exercise 2 — Justify the parity. A teammate says: "Why do I run the commands locally if CI is going to run them anyway? It's double work." Answer them by explaining what local parity gives you that waiting for CI doesn't, using the module's central idea.
See solution
Local parity isn't double work; it's faster and cheaper feedback on the pipeline itself. Three reasons:
- Diagnosis speed. If your
requirements.txthas an error or you're missing a dependency, locally you discover it in seconds, in your terminal, with all the context at hand. Waiting for CI means: commit, push, wait for the runner to start, install, and run, read the log, and only then find out —a cycle of minutes for each attempt—. For silly errors (a misspelled version), local is orders of magnitude faster. - Confidence before pushing. Running the same commands locally and seeing them green is direct evidence that the pipeline will do the right thing, before anyone else sees your code. You push with confidence, not with hope.
- The module's central idea makes it valid. "What CI does to your suite is the same thing your machine does": same command, same suite, same dependencies. That's why the local output predicts CI's (except for details like the platform). It's not that you run something different "just in case"; you run the same, and that's why the local result tells you what to expect from CI.
In one sentence: local parity turns CI from "the place where I discover something was wrong" into "the automatic confirmation of something I already know is fine". It's not double work; it's moving the discovery to the cheapest place.
Exercise 3 — Write the scope note for a change. Imagine you expand the pipeline so that, besides running on Python 3.14, it also runs on 3.12 and 3.13 (a matrix, which is module 4). Rewrite step 5's scope note reflecting that change: what it covers now and what still remains for later.
See solution
A reasonable note after adding the version matrix:
Scope of
tests.yml. Runs Reservo's complete suite (11 tests) on every push and pull request, now on three Python versions (3.12, 3.13, and 3.14) on Ubuntu, installing fromrequirements.txt. The job turns red if any test fails on any of the versions (this way we catch version-specific incompatibilities). Left for following modules: reproducing a failure that only happens in CI and pinning deterministic environments (module 3); speeding up the matrix with dependency caching and parallelism, which now matters more because we run the suite three times (module 5); and requiring a coverage threshold that breaks the build (module 6). It also still runs on only one operating system (Ubuntu); expanding to Windows/macOS is another dimension of the matrix, to be evaluated based on where our users run.
The important thing about this note: it accurately reflects what changed (three versions instead of one, and what it implies —red if it fails on any—) and updates what's left for later. Notice that adding the matrix makes module 5 more relevant (cache/speed), because now the suite runs three times and the cost starts to matter —the note acknowledges it explicitly—. And it keeps the honesty about another dimension still not covered (a single operating system). A good scope note evolves with the pipeline: it says where it is today and where it can grow.
Summary and next step
In this mini-project you integrated the whole module by producing a real deliverable: the tests.yml that runs Reservo's suite on every push. You wrote the complete workflow with named steps, justifying each decision with the lesson it comes from —on: [push, pull_request], runs-on: ubuntu-latest, checkout first, setup-python with "3.14" in quotes, install from requirements.txt, pytest—. You established local parity: you ran in your terminal, for real, the same steps CI would execute, and saw the same green output (11 passed, exit code 0), with the only expected difference in the platform. You verified that the pipeline bites, provoking a real red (exit code 1) and restoring it. You hung the badge. And you wrote the scope note: what your pipeline does and what's left, on purpose, for the following modules.
With this you close module 2. Look at everything you can do now that you couldn't at the start: write a GitHub Actions workflow from scratch and explain each line; choose the triggers with judgment; prepare the runner with checkout and setup-python; install dependencies from a pinned list; understand how pytest's exit code paints the job; read a CI log by going straight to the failure; and hang the badge. You set up your first pipeline —the one that turns "I run the tests when I remember" into "the tests run on their own on every change"—, and you verified with your hands that it does what it says.
What's next is what happens when that pipeline, one day, gives you a surprise. So far, when you ran the suite locally and in CI (conceptually), you got the same. But remember that single difference you saw in the platform line: your machine is darwin, the runner is linux. Sooner or later, a test is going to pass on your machine and fail in CI —or the reverse—, and that environment gap is baffling the first time. Module 3 is dedicated entirely to it: why CI red and your local green don't contradict each other, how pinned dependencies and deterministic installs close the gap, and how to reproduce on your machine the failure you only saw in CI. You already know how to set up the pipeline; now you're going to learn to trust it when its verdict doesn't match yours.
Resources
- Build and test Python (GitHub Actions documentation) — GitHub's official guide to testing Python projects, with the complete example workflow you integrated here. The first place to consult when you set up a real pipeline.
- How to invoke pytest (pytest documentation) — the reference for the ways of running pytest (complete, filtered, quiet) you used in local parity, and the exit-codes section that connects with the job's color. The day-to-day reference.
- GitHub Actions quickstart guide — the official "create your first workflow" tutorial, which walks the same path as this mini-project from GitHub's side. Useful if you want to set it up in a real repository of yours, step by step, with screenshots of the Actions tab.