Module 3: Reproducing A Ci Failure Locally

5. A clean venv that replicates CI's

Description

You already know what differs between CI and your machine (the environment gap, lesson 3) and why dependencies diverge (ranges vs pins, lesson 4). This lesson gives you the tool that turns that knowledge into action: the clean virtual environment, a freshly created venv inside your machine where only what you install on purpose exists —a replica of CI's empty kitchen, set up on your own disk—. By the end you'll know how to create one with python -m venv, activate it, install in it the exact versions CI used with pip install -r requirements.txt, make your project importable inside, and run the suite there to reproduce the red at will.

The idea is simple and powerful: instead of fighting your machine's sediment —all the old libraries and accumulated variables that make it "too kind"—, you sidestep it entirely by creating a new, empty corner. That corner starts from scratch, just like the CI runner on every run, so if you install in it exactly what CI installed, you get its same kitchen. And with the same kitchen, the same code produces the same result —including the red you couldn't see—.

Connection to the module. Lesson 4 gave you the pin (the exact versions); this one gives you the clean place to apply it. It's the bridge between "I know which version CI used" and "I reproduced its failure": the clean venv is where the pin becomes a real reproduction. It prepares lesson 6 (the differences a clean venv doesn't cover: variables, tz, files) and lesson 7 (the complete method, of which the clean venv is the central step). On Reservo's requirements.txt and its pytz failure.

The analogy: the test kitchen

A chef who wants to reproduce a customer's problem ("your dish made me sick") doesn't try it in their usual kitchen, full of their ingredients and quirks. They set up a test kitchen: an empty table, and on it they put only what the customer said they used —this brand of oil, this flour, this method—. Nothing more. If on that controlled table the dish also comes out wrong, they reproduced the problem and can study it. If they'd cooked in their usual kitchen, any "house" ingredient —a spice they always add without thinking— could mask the problem or create a new one, and they'd never know what caused what.

The clean venv is that test kitchen. Your usual machine has too much: old pytz installed for another project, variables exported months ago, global packages you don't even remember. Any of them could be covering or falsifying the failure you're chasing. A new venv is an empty table inside your machine: it only has what you put on purpose. You put exactly what CI installed, you run, and you observe. You control the variables one by one because you started from scratch, not from an accumulated jumble.

Said directly:

A clean virtual environment is an empty test kitchen inside your machine: it starts from scratch, like the CI runner, and only contains what you install on purpose. Installing CI's exact versions in it reproduces CI's kitchen —and with it, its result, including the failure—.

What a virtual environment is and why it starts from scratch

A virtual environment (venv) is a folder that contains a copy (or link) of the Python interpreter and its own site-packages —the place where libraries are installed—, isolated from the system Python and from other projects. When you "activate" it, your terminal uses that Python and that site-packages: pip install installs there, import looks there. It's a sealed compartment. Installing something in a venv doesn't touch the rest of your machine, and —the key for us— a freshly created venv is almost empty: it doesn't inherit the libraries you have installed globally or in other venvs.

That last part is what makes it the perfect tool for reproducing. A new venv doesn't have your sedimented pytz 2022.1. It has nothing, except pip to be able to install. It's a blank page. When you install CI's requirements.txt in it, you get exactly the dependency tree CI got —no more (there's no sediment left over) and no less (you install everything declared)—. The kitchen ends up identical in the layer that breaks the most: the libraries one.

The base commands, which you'll repeat your whole life:

python3.14 -m venv fresh-venv          # create the venv (a folder called fresh-venv)
source fresh-venv/bin/activate         # activate it (on macOS/Linux; on Windows: fresh-venv\Scripts\activate)
pip install -r requirements.txt        # install the declared dependencies
pytest                                  # run the suite in this clean environment
deactivate                              # exit the venv when you're done

A couple of details that matter. python3.14 -m venv fixes which version of Python the venv will have: if you want to replicate CI's Python 3.14, create the venv with a python3.14, not with any python. (Replicating the exact interpreter version is a topic for lesson 7 and module 4, but it starts here: the venv inherits the version of the Python you create it with.) And pip install -r requirements.txt is the same command your CI workflow's install step runs —you're imitating it literally—.

What a freshly created venv contains (nothing, almost)

It's worth seeing how empty a new venv is, because that emptiness is exactly its value. Freshly created, before installing anything, a venv contains only pip.

What to expect. You create the venv and ask what's installed:

$ python3.14 -m venv fresh-venv
$ fresh-venv/bin/python -m pip list
Package Version
------- -------
pip     25.2

That's all: pip 25.2 and nothing more. No pytz, no pytest, none of the libraries you have scattered around your machine. Confirm it by asking directly whether it sees your sedimented pytz:

$ fresh-venv/bin/python -c "import importlib.util as u; print('sees pytz?', u.find_spec('pytz') is not None)"
sees pytz? False

False: the clean venv doesn't see the pytz you have installed globally. It's isolated. That's the difference from running bare python on your machine, which would see all your sediment. Starting from this False —from this blank page— is what guarantees that, when you install CI's requirements.txt, you'll have only what CI has, without contamination.

Compare this mentally with the CI runner: its ephemeral virtual machine starts equally empty, and the pip install step puts exactly what's declared on it. Your clean venv is the local version of that same ephemeral machine. The only difference is that yours lives in a folder you can delete (rm -rf fresh-venv) and recreate whenever you want.

Making your project importable

There's a step people forget that produces a baffling ModuleNotFoundError: No module named 'reservo': in a clean venv, your third-party dependencies (pytz) are installed with pip, but your own code (reservo/) isn't installed —it's just a folder on your disk—. For from reservo.localtime import local_start_hour to work inside the venv, Python has to be able to find the reservo/ folder.

There are two ways, and it's worth knowing both:

  1. Running pytest from the project root. If you stand in the folder that contains reservo/ and run pytest from there, pytest adds the root to the import path, and reservo is found. It's the simplest for a small project, and it's what this module's examples assume. (Whether you run from the root or from a subfolder is, additionally, lesson 2's suspect #7 —the working directory—; always running from the root rules it out.)

  2. Installing the project in editable mode. If the project has a pyproject.toml (or setup.py), pip install -e . installs it "linked": the venv knows where reservo/ lives and imports it like any library, run from wherever you run. It's what serious projects do, and often what CI does. The -e (editable) means it keeps pointing at your folder, so you see your changes without reinstalling.

To reproduce Reservo's failure, option 1 is enough —running pytest from the root—, but if your CI does pip install -e ., replicate it: it's part of matching the kitchen. The module's general rule applies here too: do the same CI does, including how it makes your code available to the tests.

Worked example: reproduce the red in a clean venv

Let's pull it all together to reproduce Reservo's failure. You know (from reading the CI log, lesson 2) that CI runs Python 3.14.0 and installed pytz 2026.3.post1. The plan: create a clean venv with Python 3.14, pin pytz to that exact version, install, and run the same command.

First, the requirements.txt pinned to what CI used (instead of the project's ambiguous >=):

# requirements-repro.txt — pinned to what CI installed
pytz==2026.3.post1

Then, the clean venv and the install:

$ python3.14 -m venv repro-venv
$ repro-venv/bin/python -m pip install -r requirements-repro.txt pytest==9.1.1
$ repro-venv/bin/python -c "import pytz; print('pytz:', pytz.__version__)"
pytz: 2026.3.post1

The kitchen ended up like CI's in the dependency layer: pytz 2026.3.post1, Python 3.14.0, pytest 9.1.1. Now run the same command CI runs, from the project root:

What to expect.

$ repro-venv/bin/python -m pytest test_localtime.py -q
F                                                                        [100%]
=================================== FAILURES ===================================
____________________ test_summer_booking_starts_at_16_local ____________________

>       assert local_start_hour(a_booking(), "America/Mexico_City") == 16
E       AssertionError: assert 15 == 16
E        +  where 15 = local_start_hour(Booking(id='bk-1', ...), 'America/Mexico_City')

test_localtime.py:21: AssertionError
=========================== short test summary info ============================
FAILED test_localtime.py::test_summer_booking_starts_at_16_local - assert 15 == 16
1 failed in 0.03s

Red. You reproduced CI's failure on your machine. That AssertionError: assert 15 == 16 is the same one you saw in the CI log and that didn't appear in your usual environment. It's no longer a ghost: you have it live, in your terminal, whenever you want. This is the milestone that unlocks everything else —now you can diagnose why (sibling guide), or decide that the expected number aged and correct it (mini-project)—.

To close the loop and demonstrate that the clean venv is deterministic —that it reproduces any result, not just the red—, set up another venv pinned to the old version (2022.1, your usual machine's) and run the same:

$ python3.14 -m venv old-venv
$ old-venv/bin/python -m pip install pytz==2022.1 pytest==9.1.1
$ old-venv/bin/python -c "import pytz; print('pytz:', pytz.__version__)"
pytz: 2022.1
$ old-venv/bin/python -m pytest test_localtime.py -q
1 passed in 0.02s

Green. With pytz 2022.1 pinned, the same test passes —it reproduces the dev's kitchen—. Two clean venvs, same machine, same code, same command: one red and one green, and the only difference between them is the version number you pinned. That's total control over the dependency layer: you can reproduce at will CI's environment (red) or the dev's (green), and compare. The clean venv turned a mysterious divergence into a switch you turn on and off.

Deep dive: what the clean venv covers and what it doesn't

The clean venv is the central tool of reproduction, but it's honest to mark its limits, because it doesn't close the whole environment gap —only some layers—.

What it does cover:

  • The dependency layer: isolated from the sediment, it installs exactly what's declared. This is the one that breaks most, and the one the venv solves at the root.
  • The interpreter layer: if you create the venv with the correct Python version, you replicate that layer (partially; module 4's matrix systematizes it).
  • Much of the working directory layer: by forcing you to run from the root or to install the project, it orders how imports resolve.

What it doesn't cover on its own:

  • The environment variables layer: a venv doesn't erase your shell's variables. If the failure depends on RESERVO_TAX_PERCENT, the clean venv will still see it if it's exported. They have to be managed separately (lesson 6).
  • The operating system layer: a venv on your Mac is still macOS; it doesn't become CI's Linux. For OS differences you need containers or the matrix (beyond this module, except as a mention).
  • The system time zone layer (the TZ variable): the venv doesn't change it; you control it with variables (lesson 6). The tz data it does control, because it travels in pytz, which is a dependency.
  • The file system layer: a venv doesn't erase your disk's uncommitted files; if your test reads one only you have, it'll keep reading it.

That's why the clean venv is necessary but not always sufficient: it closes the dependency layer (the most common) in one stroke, but the invisible layers —variables, system tz, files— need their own attention. Lesson 6 deals precisely with those, and lesson 7 pulls them all together into a method where the clean venv is the central step but not the only one. Knowing what it covers and what it doesn't saves you the mistake of creating a perfect venv and being baffled because the failure still doesn't reproduce: if it's a variable, the venv alone wasn't going to catch it.

Common mistakes

Reproducing in your usual environment instead of in a clean venv. What happens: you install CI's pytz version on top of your global Python, with all your sediment present, and the result is confusing —sometimes it reproduces, sometimes it doesn't, and you don't know what else influences it—. Why it happens: creating a venv feels like an unnecessary extra step. How to spot it: if you ran pip install without having activated (or pointed at) a new venv, you touched your global environment. How to fix it: always reproduce in a freshly created venv. The venv's emptiness is what isolates the variable you're chasing; in your global environment, a hundred other things can interfere.

Creating the venv with the wrong Python version. What happens: you reproduce with a venv created with python3.12 when CI uses 3.14 (or vice versa), and either you don't reproduce the failure, or you reproduce a different one. Why it happens: you create the venv with "whatever python is handy" without checking the version. How to spot it: if python --version inside the venv doesn't match the CI log's header, you have the interpreter layer misaligned. How to fix it: create the venv with the executable of CI's exact version (python3.14 -m venv ...), and verify with python --version inside. The venv inherits the version of the Python you created it with.

Forgetting that your own code doesn't install itself. What happens: you set up a clean venv, install pytz and pytest, run from another folder, and get ModuleNotFoundError: No module named 'reservo', and you think you broke something. Why it happens: it's easy to assume that if pytz imports, reservo does too, but reservo is your code, not an installed library. How to spot it: the error mentions your package (reservo), not a third-party library. How to fix it: run pytest from the project root (where reservo/ lives), or install the project with pip install -e . if it has pyproject.toml. Replicate how CI makes your code available to the tests.

Exercises

Exercise 1 — The correct venv. The CI log starts with platform linux -- Python 3.14.0, pytest-9.1.1 and installed pytz==2026.3.post1. Write the sequence of commands that would set up a clean venv to reproduce its environment (create, install, verify versions), assuming you run the tests from the project root.

See solution
# 1. Create the venv with the SAME Python version as CI (3.14)
python3.14 -m venv repro-venv

# 2. Install exactly what CI installed (pinned pytz + pytest)
repro-venv/bin/python -m pip install pytz==2026.3.post1 pytest==9.1.1

# 3. Verify that the versions match the CI log
repro-venv/bin/python --version                       # -> Python 3.14.0
repro-venv/bin/python -c "import pytz; print(pytz.__version__)"   # -> 2026.3.post1

# 4. Run the same command, from the project root
repro-venv/bin/python -m pytest test_localtime.py -q

Notes: it's created with python3.14 to match the interpreter layer; pytz is pinned to the log's exact version; it's verified before running (so you don't reproduce with a different kitchen without noticing); and it's run from the root so reservo is importable. The only layer this venv doesn't match automatically is the OS (you're still on your machine, not on linux), but for a pytz failure that doesn't matter.

Exercise 2 — The venv that doesn't reproduce. You set up a clean venv with pytz==2026.3.post1 and Python 3.14, ran the suite, and the local-time test... passed. The failure didn't reproduce. Give at least two hypotheses of which layer might be misaligned, and how you'd verify it.

See solution

If with CI's pytz the failure doesn't reproduce, some other layer differs. Hypotheses:

  1. The pytz version isn't really CI's. Maybe the CI log said another version, or the venv resolved to a different one. Verify with pytz.__version__ inside the venv and compare it character by character with the CI log.
  2. The system time zone (TZ) differs. The local-time calculation doesn't depend only on pytz, but also on how the datetime is built. If your shell has TZ=America/Mexico_City and the runner another, or if the test depends on the system tz, the result changes. Verify with echo $TZ and printenv TZ, and try forcing the same one (lesson 6).
  3. A related environment variable. Some variable the code reads that you have and CI doesn't (or vice versa). Compare env between your shell and what the workflow defines.
  4. The Python version doesn't match exactly. 3.14.0 vs 3.14.1 could (rarely) matter. Verify python --version against the log's header.

The underlying lesson: the clean venv closes the dependency layer, but if the failure lives in another layer (variables, system tz), the venv alone doesn't catch it. That's where lesson 6 comes in.

Exercise 3 — Venv determinism. Explain, with the example of the two venvs (one with pytz==2022.1, another with pytz==2026.3.post1), what it means for the clean venv to be "deterministic" and why that's exactly what you need to reproduce a failure.

See solution

"Deterministic" means the result depends only on what you put on purpose in the venv, and not on hidden sediment or chance. With the venv pinned to pytz==2022.1, the suite gives green always; with the one pinned to pytz==2026.3.post1, it gives red always. The color doesn't dance between runs: it's a pure function of the version you installed. You can set up the venv, destroy it (rm -rf), and set it up identical as many times as you want, and get the same result.

Why that's exactly what you need to reproduce: a failure you want to fix has to happen at will, not every now and then. If the venv were non-deterministic (depended on your sediment), you'd sometimes reproduce the failure and sometimes not, and you'd never know whether your fix worked or whether it simply "came up green" this time. The venv's determinism turns the failure into a switch —you turn the red on by installing CI's version, you turn it off by installing the old one— and only with a reliable switch can you work: you change the code, run, and the color tells you the truth without noise.

Summary and next step

In this lesson you set up the central tool of reproduction: the clean venv, an empty test kitchen inside your machine that starts from scratch like the CI runner. You saw with real output how empty it starts (pip list shows only pip 25.2; it doesn't see your sedimented pytz), you learned to create it with python3.14 -m venv (inheriting the correct Python version), to install CI's exact versions with pip install -r requirements.txt, and to make your project importable (run from the root or pip install -e .). And you reproduced Reservo's failure: with pytz==2026.3.post1 pinned, the same AssertionError: assert 15 == 16 from CI appeared in your terminal —the ghost, made flesh—.

You closed the loop by showing the venv's determinism: a venv with the old version gives green always, one with the new gives red always; the color is a pure function of what you installed, a switch you turn on and off. And you marked the honest limits: the venv closes the dependency layer (the most common) in one stroke, but it doesn't erase on its own the invisible layers —variables, system tz, uncommitted files—.

Before moving on you should be able to: create a venv with a specific Python version; explain why a new venv doesn't see your sediment; make your project importable inside; reproduce a failure by installing CI's exact version; and say which layers the venv doesn't cover.

What's next is hunting exactly those layers the venv doesn't cover. Lesson 6 goes after the hidden differences —environment variables, system time zone, test order, files that only exist on your disk—: how to detect them by comparing the two environments and how to replicate them so the reproduction is complete when the failure wasn't (only) about a dependency.

Resources