2. Why the shape of the suite matters

Overview

By the end of this lesson you will be able to name, one by one, the concrete costs of a suite without structure —not as a vague complaint of "it's messy", but as specific problems that are felt day to day and that the folder structure solves—. In the previous lesson you saw the flat Reservo suite and its contrast with the layered suite. Here we are going to stay in the "before" and examine it under a magnifying glass: what exactly you cannot do when twelve tests live piled in a single directory, and why that cost is not a fixed nuisance you pay once, but a debt that grows with every file you add.

This matters because structure is one of those investments that seem unnecessary until the day they are urgent, and by then it costs triple to do them. With ten tests, the flat directory works: you open them all, run them all, done. The problem does not appear all at once; it accumulates. One day you want to run just the fast tests while developing —so as not to wait for the slow ones every time— and you discover you have no way to ask the folder for that. Another day you look for the test that covers the refund and you have to open five files to find it. Another day someone new arrives, opens tests/ expecting to understand the system, and just sees a list of names. Each of those days is cheap on its own; added up over a year and multiplied across the whole team, they are the difference between a suite that helps and one that gets in the way. This lesson makes that debt visible before you pay it.

Connection to the module: this lesson is the diagnosis that justifies everything that follows. It names three costs of the flat directory —you cannot run in parts, you cannot find where a test lives, you cannot see the shape of the system— and each one anticipates a solution of the module. Running in parts is lesson 5 (physical layers you run separately). Finding where a test lives is lesson 3 (grouping by layer or by feature) and lesson 7 (structure as a map). Seeing the shape of the system is lesson 7 (structure as documentation). Here we only diagnose; the treatment comes later. And remember the boundary: we are still talking about folders, not markers —those are module 4—.

The toolbox dumped in a bucket

Think of it this way. A carpenter has their tools. They can store them in two ways. The first: all in a bucket. Hammers, screwdrivers, drill bits, nails, the tape measure, the sandpaper —all in there, jumbled—. When the bucket has five things, finding the hammer is trivial: you reach in and there it is. The bucket works. But the workshop grows, new tools arrive, and one day the bucket has eighty things. Now finding the small Phillips screwdriver means dumping half the bucket on the floor. And if you were asked to "bring me just the drill bits", you would have to take everything out and separate by hand, because in the bucket there is no "drill bits section": there is a jumble.

The second way: a box with compartments and a wall board with silhouettes. The hammers on their hook, the drill bits in their drawer, the screwdrivers ordered by size. Finding the small Phillips screwdriver is going to its place. "Bring me just the drill bits" is opening a drawer. And when a new tool arrives, there is an obvious place where it goes —or if there is not, that absence tells you that you need a new compartment, which is also useful information—. The wall board, moreover, tells anyone who walks into the workshop what tools there are and how this carpenter works, without them having to explain anything.

Both store the same tools. The difference is what it costs to use them: find one, take out a subset, and understand the workshop at a glance. The bucket scales terribly —each new tool worsens the search for all the others—; the box with compartments scales well —each new tool goes to its place and does not get in the way of the others—. A test suite in a flat directory is the bucket. This module builds the box with compartments. And this lesson measures, with the Reservo suite, exactly how much the bucket costs.

The three costs of the flat directory

Let us go back to the flat Reservo suite: the twelve tests of lesson 1, piled in a single tests/ directory. We are going to examine three costs, from the most operational to the most human.

Cost 1: you cannot run just a subset

This is the one that bites first, because you pay it every time you develop. While you work on the pricing code, you want to run only the pure-logic tests —fast, isolated— over and over, without dragging along in each run the integration tests that assemble the complete BookingService. In a layered suite, that is one command: pytest tests/unit. In the flat bucket, there is no "the fast stuff" folder, so the only way to run a subset is to name the files one by one:

python3 -m pytest tests/test_overlaps.py tests/test_pricing.py tests/test_refund.py

What to expect. On my machine (Python 3.14.0, pytest 9.1.1):

.......                                                                  [100%]
7 passed in 0.01s

It works —it runs the 7 fast tests—, but look at what you had to type: the three file names, by hand. And here is the problem that does not show in the green result: that list is fragile. Tomorrow you add tests/test_availability.py, another fast test. If you forget to add it to the list —and you will forget—, your "run the fast stuff" command silently stops running it. There is no error, no warning: that test simply stops executing in your development cycle, and you find out when you have already broken something. The flat folder has no way to say "all the fast ones"; it only has loose files that you have to remember to enumerate. Compare it with the layered version, where the same subset is a folder:

python3 -m pytest tests/unit

.......                                                                  [100%]
7 passed in 0.01s

Same result, 7 tests, but now tests/unit is the subset: any new file you put there enters automatically, and none that should not enter sneaks in. The folder does the work you did by hand and badly in the bucket.

Cost 2: you cannot find where a test lives

The second cost you pay every time you look for something. Suppose a coworker reports that the 36-hour refund returns the wrong amount. You want to open the test that covers that case. In the flat bucket, which of the five files is it? test_pricing.py sounds like prices, not refunds... or is the refund there because "it's also money"? Or in test_cancel_flow.py, because cancelling triggers the refund? Or is there a test_refund.py? In a flat directory of five files you still solve it by opening a couple. But the flat suite of a real project does not have five files: it has forty, with names that accumulated without a plan —test_pricing.py, test_pricing2.py, test_new_pricing.py, test_pricing_fixed.py—, and finding the right test is the expedition to the shed.

Structure solves this by giving each test a predictable place. If the suite is by feature, the refund lives in tests/pricing/ (or tests/refunds/); you go there and it is there. If it is by layer, you know refund_cents is pure logic, so it lives in tests/unit/, and there you look for it. In both cases, the shape of the suite tells you where to look before opening a single file. The bucket tells you nothing: each search starts from scratch.

Cost 3: you cannot see the shape of the system

The third cost is the quietest and the most expensive in the long run. Someone new arrives to the team —or it is you yourself, coming back to the project six months later—. You open tests/ to understand how Reservo works, because the tests are the best living documentation there is: they say what the system does and what rules it obeys. In the flat bucket, what you see is this:

tests/
├── conftest.py
├── test_booking_flow.py
├── test_cancel_flow.py
├── test_overlaps.py
├── test_pricing.py
└── test_refund.py

A list of names. It tells you there is something about bookings, something about prices, something about cancellations —the names help—, but it does not tell you how they relate, which are the logical core and which the machinery that ties them, or where to start reading. The shape is flat, so it communicates no hierarchy or layers: everything looks equally important and of the same kind. Compare it with what the newcomer sees in the layered suite:

tests/
├── conftest.py
├── integration/
│   ├── conftest.py
│   ├── test_booking_flow.py
│   └── test_cancel_flow.py
└── unit/
    ├── conftest.py
    ├── test_overlaps.py
    ├── test_pricing.py
    └── test_refund.py

This shape teaches. In ten seconds, without opening a file, the newcomer learns: Reservo is tested in two layers; there is a core of pure logic (unit/: overlap, pricing, refund) and a layer of pieces that collaborate (integration/: the booking and cancellation flows). The structure told them the architecture of the system. That is the hidden cost of the bucket: it is not just uncomfortable for you today, it is that it teaches no one anything, and a suite that does not teach wastes the best documentation a project can have.

Worked example: the list versus the tree, measured

The three costs have a common root: the flat directory has no hierarchy, and without hierarchy there are no subsets, no predictable places, no shape that teaches. We can see it directly with --collect-only, which draws the hierarchy —or its absence—. First, the bucket:

python3 -m pytest tests --collect-only -q

What to expect, on the flat suite (I use -q to see just the list, without the indented tree):

tests/test_booking_flow.py::test_booking_a_room_charges_the_pro_price
tests/test_booking_flow.py::test_room_is_unavailable_after_it_is_booked
tests/test_booking_flow.py::test_double_booking_the_same_slot_is_rejected
tests/test_cancel_flow.py::test_cancelling_72h_ahead_refunds_in_full
tests/test_cancel_flow.py::test_cancelling_frees_the_room
tests/test_overlaps.py::test_touching_intervals_do_not_overlap
tests/test_overlaps.py::test_nested_interval_overlaps
tests/test_pricing.py::test_basic_member_pays_hourly_rate_times_hours
tests/test_pricing.py::test_pro_member_gets_twenty_percent_off
tests/test_refund.py::test_full_refund_at_72h
tests/test_refund.py::test_half_refund_at_36h
tests/test_refund.py::test_no_refund_at_12h

12 tests collected in 0.00s

Twelve lines, all with the same tests/ prefix. There is no grouping: the integration tests (test_booking_flow, test_cancel_flow) and the unit ones (test_overlaps, test_pricing, test_refund) are interleaved without distinction. To "see the fast ones" you would have to read the names one by one and classify in your head. Now the indented tree of the same flat suite, which makes the lack of hierarchy even more evident:

python3 -m pytest tests --collect-only

<Dir tests>
  <Module test_booking_flow.py>
    <Function test_booking_a_room_charges_the_pro_price>
    <Function test_room_is_unavailable_after_it_is_booked>
    <Function test_double_booking_the_same_slot_is_rejected>
  <Module test_cancel_flow.py>
    <Function test_cancelling_72h_ahead_refunds_in_full>
    <Function test_cancelling_frees_the_room>
  <Module test_overlaps.py>
    <Function test_touching_intervals_do_not_overlap>
    <Function test_nested_interval_overlaps>
  <Module test_pricing.py>
    <Function test_basic_member_pays_hourly_rate_times_hours>
    <Function test_pro_member_gets_twenty_percent_off>
  <Module test_refund.py>
    <Function test_full_refund_at_72h>
    <Function test_half_refund_at_36h>
    <Function test_no_refund_at_12h>

The tree has exactly two levels: <Dir tests> and its five <Module>. It is a flat fan. There is no intermediate <Dir> that groups, because on disk there are no subfolders. The depth of the tree is the measure of the organization: two levels is a bucket; three or more is a box with compartments. The whole module is about gaining that third level with criterion.

Why the cost grows (and it is not linear)

A detail worth understanding well, because it is what makes structure urgent: the cost of the bucket is not fixed, and it does not even grow evenly. It grows faster than the suite.

Think about the cost of "finding a test". With 5 files, in the worst case you open 5. With 40 files, in the worst case you open 40. But moreover, with more files there are more similar names, more accumulated duplication (test_pricing2.py), and more chance that the test you seek is in a counterintuitive place. Each new file not only adds its own weight: it worsens the search for all the others, because it enlarges the jumble they all live in. It is the bucket: tool number eighty is not only hard to find itself; it makes it harder to find the other seventy-nine.

Structure breaks that curve. In a layered suite, adding a refund test to tests/unit/test_refund.py does not affect at all your ability to find the integration tests: they live in another branch of the tree, in another drawer. Each compartment bounds the jumble to its own content. That is why the investment in structure, which at ten tests seems excessive, is exactly what keeps the suite navigable when it reaches a thousand: it does not eliminate the growth, but it makes it local. A new file weighs only its own, not everyone's.

Common mistakes

Waiting for it to hurt to structure (of timing). What happens: the team leaves the suite flat "because it's still manageable" and plans to reorganize "when it's needed". Why it happens: with few tests the bucket really works, and reorganizing feels like work with no immediate reward. How to detect it: the moment when it "is needed" is exactly the moment when reorganizing costs the most —forty jumbled files, with mixed fixtures and tacit dependencies—, so if you wait for the pain, you pay for the structure in its most expensive version. How to fix it: the practical rule is to structure early and cheap —two folders, unit/ and integration/, from when the suite grows past a handful of files—; it is trivial to do with ten tests and a nightmare with two hundred.

Confusing "the tests pass" with "the suite is fine" (of criterion). What happens: someone looks at 12 passed and concludes there is nothing to improve. Why it happens: green is the strong and satisfying signal as always, and it is easy to believe it is the only one that matters. How to detect it: the three costs of this lesson —not running in parts, not finding a test, not seeing the shape— coexist perfectly with 12 passed; the flat bucket passes all its tests. Green measures whether the tests are correct, not whether the suite is navigable. How to fix it: add to your criterion of "healthy suite" the questions of this lesson: can I run just a part? do I quickly find where something lives? does the shape teach the system to someone new? If any is "no", there is work to do even if everything is green.

Solving the subset with file names by hand (of method). What happens: instead of creating folders, someone memorizes or saves in an alias the list of "fast" files and passes it by hand to pytest. Why it happens: it seems faster than reorganizing, and it works the first day. How to detect it: as soon as you add a new fast file and forget to add it to the list, your "run the fast ones" stops running it without warning —a test that exists but that your development cycle no longer executes—. How to fix it: let the folder define the subset. pytest tests/unit automatically includes everything you put there; there is no list to maintain or file to forget. The structure does the work the by-hand list does badly.

Exercises

Exercise 1 — Name the cost. For each situation, say which of the three costs of the flat directory you are paying —(1) not running subsets, (2) not finding a test, (3) not seeing the shape— and how the structure would cure it. (a) You are debugging the price computation and wait eight seconds on each run because the integration tests run together with the pricing ones. (b) A new colleague asks "where do I start to understand this?" and you answer "open the files and read". (c) A refund bug is reported and you open three files before hitting the right test.

See solution

• (a) Cost 1: not running subsets. You are paying the time of the integration tests on each run because you cannot isolate the pricing ones. The cure: a physical layer (tests/unit/) that you run alone with pytest tests/unit, leaving the integration out of the fast cycle.
• (b) Cost 3: not seeing the shape. The newcomer cannot learn the system from the structure because the structure is flat and teaches nothing; that is why you send them to read files. The cure: a layered (or by-feature) shape that, with --collect-only or a simple ls, shows them the architecture before opening code.
• (c) Cost 2: not finding a test. The refund has no predictable place, so the search is an expedition through several files. The cure: an obvious place for each test —tests/unit/test_refund.py if you organize by layer, tests/pricing/ if by feature— so you know where to look without opening anything.

The rule you are applying: each everyday pain of the flat bucket corresponds to one of the three costs, and each cost has a concrete structural cure. Diagnosing the cost well tells you which part of the structure you need.

Exercise 2 — Predict the fragility. In the flat suite, you run your fast tests like this: pytest tests/test_overlaps.py tests/test_pricing.py tests/test_refund.py. Tomorrow you add tests/test_availability.py, another fast pure-logic test, but forget to update your command. What happens exactly? Does pytest warn you? Now describe what would have happened if the suite were layered and you had put the file in tests/unit/.

See solution

With the by-hand list: your command pytest tests/test_overlaps.py tests/test_pricing.py tests/test_refund.py keeps running 7 tests —the usual ones—, completely ignoring test_availability.py. Pytest does not warn you: there is no error, no warning. You named three files, pytest runs those three, and does exactly what you asked. The new test exists on disk, but your development cycle stopped executing it, and you do not find out until you break availability and no red test tells you —because the one that covered it never ran—. It is a dangerous silence: the absence of coverage generates no signal.

With the layered suite and the file in tests/unit/: pytest tests/unit automatically discovers test_availability.py, because you asked "run the unit folder", not "run these files". Without touching your command, the fast subset goes from 7 tests to 9. The folder is the subset, so any file that falls inside enters on its own. There is no list to maintain or possible forgetting. That is the underlying difference: naming files is enumerating; a folder is a rule ("everything that is here"), and rules do not forget the new things.

Exercise 3 — Measure the depth. The text says that "the depth of the --collect-only tree is the measure of the organization": two levels is a bucket, three or more is a box with compartments. Count the depth (how many levels of <Dir>/<Module>/<Function>) in the flat tree and in the layered tree of lesson 1. Then answer: if you organized Reservo by feature with tests/pricing/ and tests/booking/, what depth would it have? Would it also be a box with compartments?

See solution

In the flat tree, the levels are: <Dir tests> → <Module ...> → <Function ...>. Three kinds of node, but a single folder level (tests). As a directory hierarchy it is flat: there is no folder inside tests. It is the bucket.

In the layered tree, the levels are: <Dir tests> → <Dir unit>/<Dir integration> → <Module ...> → <Function ...>. There are two folder levels (tests and then unit/integration). That second directory level is the compartment; it is what turns the bucket into a box.

If you organized by feature with tests/pricing/ and tests/booking/, you would have the same directory depth as the layered version: tests/ → pricing//booking/ → modules → functions. It would also be a box with compartments —equally navigable—, only that the compartments would be labeled by domain area (pricing, bookings) instead of by layer (fast, slow). Both gain the grouping level the bucket lacks; what changes is the criterion of the label. Which label is better —layer or feature— is exactly the decision of lesson 3.

Summary and next step

In this lesson you diagnosed, by name, the three costs of the flat directory. Cost 1: you cannot run a subset without naming files by hand, and that list is fragile —a new file you forget to add stops running silently—. Cost 2: you cannot find where a test lives, because the bucket gives nothing a predictable place. Cost 3: you cannot see the shape of the system, because a flat structure does not teach architecture to whoever arrives new. You saw the common root of the three —the lack of hierarchy, measurable as the depth of the --collect-only tree— and why the cost grows faster than the suite: each new file worsens everyone's jumble, whereas in a suite with compartments each file weighs only its own.

The analogy that holds it: the toolbox bucket versus the box with compartments. Both store the same; the difference is what it costs to find a tool, take out a subset and understand the workshop at a glance.

Before moving on you should be able to: name the three costs of the flat directory and give an example of each; explain why a by-hand list of files is more fragile than a folder; and measure the organization of a suite by the depth of its directory tree.

What comes next is the first big design decision: with what criterion you group. In lesson 3 you will see the two big schemes —organize by layer (tests/unit/, tests/integration/), which groups by how the test is, or by feature (tests/pricing/, tests/booking/), which groups by what it is about—, the trade-off of each executed on Reservo, and why many suites end up combining the two. It is choosing the label of the compartments this lesson demonstrated you need.

Resources

• pytest — How to invoke pytest — the reference for how to ask pytest for a file, several files or a folder; the basis of the subsets we ran here by hand and that the structure turns into a single command.
• pytest — Good Integration Practices — the official project-structure conventions; the standard against which this lesson's "flat bucket" is measured.
• pytest — Collection of test files and directories — the reference for the options that govern what pytest collects and from where; useful to understand that the --collect-only tree reflects exactly your directory tree (we open it in depth in lesson 6).