Module 2: Your First Pipeline Pytest In Ci

1. Module introduction: from your machine to the pipeline that runs pytest on its own

Description

In module 1 we built the why. You saw that "it works on my machine" isn't a guarantee but a fortunate coincidence, that the cheapest moment to catch an error is the instant it's written, and that a continuous integration pipeline exists for a single thing: running your tests automatically on every change, without depending on someone remembering. You already have that idea. What you don't have yet is the file that makes it real.

This module is that file. By the end of it you'll have written, and understood line by line, a GitHub Actions workflow —a text file called tests.yml that lives in .github/workflows/— that tells GitHub: "every time someone pushes to this repository, start a clean machine, bring the code, install Python and the dependencies, and run pytest". It's the guide's first real pipeline, the simplest one that does its job, and once you understand it, all the following modules (the matrix, the cache, the coverage gates) are variations on this same skeleton.

We're going to work on Reservo, the meeting-room booking system you already know from the sibling guides, with its test suite already written. The module's twist is this: the same suite you ran in your terminal with python3 -m pytest is now going to run on its own, in the cloud, on every push. What's tested doesn't change —that's already done—; who presses the button and when changes. The answer goes from "me, when I remember" to "the pipeline, always".

Connection to the module: this is the map lesson. We don't write the complete workflow yet (that starts in lesson 2) or touch GitHub for real; here we build the vocabulary and the mental model the rest of the module takes for granted. Lessons 2 to 6 assemble the workflow piece by piece —the anatomy of the YAML, the triggers, the steps that prepare the ground, installing dependencies, and running pytest with its exit code—. Lesson 7 teaches you to read the log and hang the badge. Lesson 8 pulls it all together into a complete tests.yml for Reservo, with the local run that proves it does what it says.

The difference between remembering to water the plants and automatic watering

Imagine you have plants at home. They need water every two or three days or they die. You have two ways to keep them alive.

The first is remembering yourself. It works the first few days, when the intention is fresh. But one weekend you travel, or you have a heavy week, or it simply slips your mind, and by the time you remember, the plant is wilted. The problem isn't that you don't know how to water —you know perfectly—; the problem is that the system depends on your memory, and memory fails exactly when you're busiest. And worst of all: when you fail, there's no alarm. The plant doesn't tell you it's thirsty; you just find it withered days later.

The second way is automatic watering: a timer connected to a drip hose. You set it up once —"every two days, at seven in the morning, thirty seconds"— and from then on it waters on its own, whether you're there or not, whether you remember or not. It doesn't water better than you; it waters the same as you, but without depending on you being present and attentive. Setting it up costs an afternoon; after that, the plant simply lives.

Running the tests by hand is remembering to water. You know how to do it —python3 -m pytest, you have it in your fingers— but the system depends on you remembering, and on you remembering right before sharing the code, which is when you're under the most pressure and it's easiest to slip. A CI pipeline is the automatic watering: you set it up once, with the tests.yml file, and from then on the tests run on their own on every push, whether you're there or not, whether you remember or not. They don't test better than you; they test the same as you, but without depending on your memory. And when something fails, unlike the withered plant, the pipeline does warn you: the push is painted red. This whole module is about writing that timer.

What a workflow is, in one sentence

Before we dive in, let's hold on to a definition you'll be able to repeat from memory:

A workflow is a text file that tells GitHub when to run something, where to run it, and what to run.

That's all. It's not a program you write in a new language or a tool you install; it's a .yml file you put in a specific folder of your repository (.github/workflows/), and GitHub reads it and obeys it. When the event you declared as the trigger happens —for example, a push—, GitHub starts a clean virtual machine on its servers, runs the steps you listed, and reports the result to you. That machine is called a runner, and it is, for all intents and purposes, a freshly formatted computer: it doesn't have your code, it doesn't have your dependencies, it doesn't have anything of yours. Your workflow is the list of instructions that turns it, from scratch, into a machine capable of running your suite.

Notice that the three pieces of the definition —when, where, what— are exactly the ones you're going to write:

  • When: the on: block of the YAML. "Run this on every push and every pull request." That's lesson 3.
  • Where: the runs-on: key. "Run on a clean Ubuntu machine." That's part of lesson 2, and the ground is prepared by lesson 4's steps.
  • What: the list of steps:. "Bring the code, install Python, install the dependencies, run pytest." Those are lessons 4, 5, and 6.

The whole module is learning to fill in those three answers precisely.

Worked example: the whole workflow, at a glance

Don't write it yet —it's the picture of where we're going, not today's work—, but look at it carefully, because it's the whole module in fourteen lines. This is the tests.yml you'll understand from the inside by the end:

# .github/workflows/tests.yml
name: tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-python@v5
        with:
          python-version: "3.14"
      - run: python -m pip install --upgrade pip
      - run: pip install -r requirements.txt
      - run: pytest

Read it as the three questions, which almost read themselves even if you don't know YAML yet:

  • name: tests — what this workflow will be called in GitHub's interface. Cosmetic but useful: it's the label you'll see in the "Actions" tab.
  • on: [push, pull_request] — the when. Runs on every push and every pull request. This line is what makes everything "automatic".
  • jobs:test:runs-on: ubuntu-latest — the where. Defines a job called test that runs on a freshly formatted Ubuntu machine.
  • The five steps — the what, in order: (1) checkout brings your code to the runner; (2) setup-python installs Python 3.14; (3) upgrades pip; (4) installs your dependencies from requirements.txt; (5) runs pytest.

That last step, run: pytest, is exactly the command you type in your terminal. It's not a special cloud version of pytest or a different magic: it's the same pytest, running the same suite, on a machine your workflow just prepared to be like yours. That's the heart of the idea, and that's why we can show it to you for real: when we get to lesson 6, you're going to run pytest on your own machine, see the real output, and understand that it's identical to what CI would produce.

What runs for real and what's "this is how it would look" (the module's honesty rule)

There's a distinction I want you to be clear on from now, because it's what makes this module honest instead of magical.

The pytest run is executed for real. Every pytest output you see in these lessons —the 11 passed, the 1 failed, the diff of a broken assert, the exit code— was produced by running Reservo's suite for real, locally, with Python 3.14.0 and pytest 9.1.1. When I say "what to expect" and show you a green block, that block came out of a real terminal. You can reproduce it on your machine and get the same.

The CI workflow is content that gets explained, not a CI we run. We don't have a GitHub runner running while you read these lines, and we're not going to pretend we do. The tests.yml file we write and take apart piece by piece; the format of the log a runner would produce we show you as honest illustration —"this is how it would look in the Actions tab"—, with the real pytest output embedded where it would go. The reason this works without cheating is exactly the one from the previous paragraph: what CI does to your suite is the same thing your machine does. CI starts a computer, prepares it to be like yours, and runs pytest. That's why we can run pytest locally, show you the true output, and tell you with full confidence "this is what CI would see", without having spun up a CI.

Keep this rule, because you'll see it applied in every lesson: real pytest, explained workflow. It's the difference between learning how a thing works and being sold it.

Why GitHub Actions (and what the alternatives are)

We're going to use GitHub Actions as the CI platform for the whole guide, and it's worth saying why, so you don't take it as the only option in the world.

GitHub Actions is the CI/CD system integrated into GitHub. If your code already lives on GitHub —as the vast majority of projects do today—, you don't have to install or hire anything separate: you put a .yml file in .github/workflows/ and you have a pipeline. That closeness —CI lives in the same place as the code— is the reason it's, by far, the most common platform for open-source projects and for a great many teams, and that's why it's the one you learn here. What you practice applies directly to a real repository of yours.

But it's not the only one. GitLab CI/CD does the same with a .gitlab-ci.yml file; CircleCI, Jenkins, Azure Pipelines, and others cover the same territory. The key names and some details change, but the mental model is identical across all: a file declares when to run, where to run, and what steps to execute; a clean machine spins up, follows the steps, runs your tests, and reports green or red based on the exit code. If tomorrow you switch to GitLab, you won't relearn the concept, only the syntax. Learning one in depth —and we're going to learn GitHub Actions in depth— is learning the pattern they all share.

What happens between the push and the verdict

Before closing, it's worth seeing the complete movie of a run, from start to finish, even though the details of each step arrive in the following lessons. Having the sequence in your head helps you understand why the pipeline takes "a few minutes" and isn't instant: there's a machine that's born, works, and dies on each run.

1. You do a git push.
        │
2. GitHub sees the event and checks .github/workflows/.
   It finds tests.yml and verifies: does its "on:" include push? Yes.
        │
3. GitHub starts a runner: a clean Ubuntu virtual machine,
   freshly formatted, without your code and without your dependencies.
        │
4. The runner executes the steps in order, top to bottom:
      checkout      → brings your code
      setup-python  → installs Python 3.14
      pip install   → installs your dependencies
      pytest        → runs the suite and returns an exit code
        │
5. GitHub reads the exit code of the last step:
      0  → green job ✓
      ≠0 → red job ✗
        │
6. The runner is destroyed completely (the machine disappears).
   The result —green or red— stays in the Actions tab and in the badge.

Three things this movie makes clear that hold for the whole module:

  • The machine is ephemeral. It's born in step 3 and dies in step 6. Nothing is saved between runs: next time, another clean machine starts from scratch. That's what gives reproducibility —no run drags garbage from the previous one— and it's the reason each run has to rebuild the environment with the steps, instead of "remembering" it.
  • The bulk of the time goes into preparing the ground, not testing. For Reservo, pytest takes hundredths of a second, but starting the machine, bringing the code, installing Python, and installing dependencies takes most of those "couple of minutes". That's why module 5 is dedicated to speeding up precisely that preparation (by caching dependencies): the suite is already fast; the slow part is the setup.
  • The whole verdict boils down to a number. Step 5 is all of CI's intelligence: reading pytest's exit code. Module 1 told you that CI warns; here you see the exact mechanism —a 0 or a non-0— and lesson 6 takes it apart completely.

Common mistakes

Believing CI "tests your code for you" or finds bugs on its own (magical expectation). What happens: someone sets up a pipeline expecting it to discover problems their tests don't cover, and is disappointed when CI passes green over buggy code. Why it happens: "continuous integration" sounds like the intelligence is in the platform. It isn't. CI runs the tests you wrote, not one more; if your suite doesn't cover a case, CI won't cover it either. How to spot it: if you expect the pipeline to warn you of a bug for which there's no test, you have the model backwards. How to fix it: remember that CI automates the when and the where of running your suite, not the what you test. The quality of the protection is set by your suite (that's the fundamentals guide); CI only guarantees that suite is always run.

Thinking the runner has your environment already set up (shared-machine). What happens: someone writes a workflow that runs pytest directly, without bringing the code or installing anything, imagining the runner is "like my machine but in the cloud". It fails with "no such file" or "no module named pytest". Why it happens: you take for granted that CI inherits your local environment. It doesn't: the runner starts empty, freshly formatted, without your code and without your packages. How to spot it: if your workflow doesn't have a checkout step and an install-dependencies one, it's missing the ground. How to fix it: every workflow starts by preparing the machine —bring the code, set up Python, install dependencies— before running anything. They're exactly lessons 4 and 5's steps, and they exist precisely because the runner knows nothing about you.

Confusing "green workflow" with "deployed code" (scope). What happens: someone sees the green badge and believes their change is already in production, or that CI "published" something. Why it happens: two ideas get mixed —continuous integration (CI) and continuous deployment (CD)— that usually go together but aren't the same. How to spot it: if you think this workflow uploads your app to a server, check which steps it has: it only runs tests. How to fix it: in this guide the focus is test CI —running your suite on every change—. "CD", automatic deployment to production, is another stage that's mentioned as a concept but that we don't set up here. Green means "your tests passed on a clean machine", which is a great deal, but it doesn't mean "it's in production".

Exercises

Exercise 1 — Translate into the three questions. A teammate describes in words what they want from their pipeline: "That every time I push code, a new Ubuntu computer starts up, sets up Python, installs the project's libraries, and runs the test suite." Without writing YAML yet, map each part of that sentence to one of a workflow's three questions (when, where, what) and say which one includes several things.

See solution
  • When: "every time I push code" → the trigger, the on: (a push). That's lesson 3.
  • Where: "a new Ubuntu computer"runs-on: ubuntu-latest. The clean machine, the runner.
  • What: "sets up Python, installs the project's libraries, and runs the test suite" → the steps, and this is the one that includes several things: set up Python (setup-python, lesson 4), install dependencies (pip install, lesson 5), and run the suite (pytest, lesson 6). One is missing that the teammate didn't mention but is indispensable: bringing the code to the runner (checkout), because the new machine doesn't have it.

The lesson here is that the what is almost always a list of steps in order, not a single action, and that one of those steps —bringing the code— is so obvious it's forgotten, even though without it there's nothing to test.

Exercise 2 — Automatic watering or memory. For each of these three situations, say whether it describes the "remembering to water" problem (running tests by hand) or the "automatic watering" solution (CI), and why: (a) "on the Friday before the long weekend no one ran the tests and by Monday the bug was already in everyone's branch"; (b) "each push gets marked with a green check or a red X without anyone doing anything"; (c) "I know perfectly how to run the suite, the problem is I forget exactly when I'm in a rush".

See solution
  • (a) is the memory problem: the system depended on someone remembering, and exactly when there was the most hurry and distraction (a Friday before a holiday), no one did. Like the withered plant, there was also no alarm until it was too late (Monday).
  • (b) is the automatic watering working: the check/X appears "without anyone doing anything" because the pipeline triggers on its own on every push. That "without anyone doing anything" is exactly what CI buys.
  • (c) is the purest description of the memory problem, and the underlying reason CI exists: the knowledge isn't missing ("I know how to run the suite"), what fails is the discipline of always running it, and it fails worse under pressure. CI doesn't teach you to test; it takes the burden of remembering off your shoulders.

The thread: CI doesn't make you better at testing, it frees you from depending on your memory to do it. Just like the timer doesn't water better, it waters always.

Exercise 3 — Real or "this is how it would look". According to the module's honesty rule, classify each of these as "executed for real locally" or "content that gets explained / this is how it would look": (a) the 11 passed in 0.03s block you'll see in lesson 6; (b) a screenshot of the Actions-tab log on GitHub with green checks per step; (c) the assert 5625 == 6000 diff of a broken test; (d) the tests.yml file with its steps.

See solution
  • (a) the 11 passed in 0.03s: executed for real locally. All pytest output in this guide comes from running the real suite with pytest 9.1.1.
  • (b) the Actions-tab log with checks: this is how it would look / content. We don't spin up a GitHub runner; we show you the format of the log as honest illustration.
  • (c) the assert 5625 == 6000 diff: executed for real locally. It's pytest's real output when a Reservo test fails (we provoked it by breaking the code on purpose and running pytest for real).
  • (d) the tests.yml file: content that gets explained. It's a file we write and take apart, not something we "execute" in the sense of spinning up a CI.

The rule, in one line: the pytest output is real; the workflow and its log are explained content. And it works because CI does to your suite the same thing your machine does, so the local output is the one CI would see.

Summary and next step

In this lesson you closed the gap between module 1's idea —"CI runs your tests on every change"— and the concrete file that fulfills it. A workflow is a text file that tells GitHub when to run something (the trigger), where to run it (a clean machine, the runner), and what to run (the steps: bring the code, set up Python, install dependencies, run pytest). You saw the whole tests.yml at a glance and learned to read it as those three questions, and you understood that the last step —run: pytest— is exactly the same command you run in your terminal, on the same suite: CI doesn't test differently, it tests on a machine it prepared to be like yours.

You also fixed the honesty rule that governs the whole module: the pytest run is executed for real locally (which is why we can paste you real outputs), while the workflow is content that gets explained and the CI log is shown as "this is how it would look". And you placed GitHub Actions as the canonical platform —with GitLab CI and others as conceptual equivalents of the same pattern—.

Before moving on you should be able to: define a workflow in one sentence; name the three questions it answers and which YAML key corresponds to each; explain why the runner starts empty and what that implies; and distinguish which part of this module is executed for real and which part is honest illustration.

What's next is to stop looking at the picture and take it apart. In lesson 2 you're going to open that tests.yml and understand every line: where the file lives, how YAML works as a format (the indentation that does matter), and the four keys that structure everything —name, on, jobs, and inside a job runs-on and steps—. By the end of that lesson, the file will stop being fourteen mysterious lines and become a recipe you can read and modify.

Resources

  • Understanding GitHub Actions — the official introduction to the concepts we named here: workflow, event, job, step, runner. The best starting point for the complete panorama.
  • pytest official documentation — home page — the reference for the tool CI is going to run. The same suite you run locally is the one the pipeline executes; this is the manual for the part that does get executed for real.
  • Continuous integration (GitHub Actions documentation) — what continuous integration is and how GitHub Actions implements it, with the CI/CD distinction we mentioned in the common mistakes. Useful for reinforcing "green isn't production".