1. Module introduction: from measuring to judging

Overview

The previous four modules taught you to measure. You know how to write a k6 script and launch virtual users (module 2); you know what p95 latency, throughput, and error rate mean, and you know how to actually compute them (module 3); you know how to shape the load with profiles and stages so it resembles reality (module 4). At the end of all that you have numbers. But a number decides nothing on its own. "The p95 was 246 ms" isn't a conclusion: it's a datum waiting for someone to judge it. And as long as that judgment depends on a human looking at a chart and saying "hmm, seems fine to me," performance isn't protected —because humans get distracted, are in a hurry before a launch, and approve things they shouldn't—.

This module closes that gap. Here the load test stops informing and starts to judge. The tool that makes it possible is called a threshold: a rule on a metric —"the p95 must be below 200 ms"— that turns a measurement into a binary verdict: pass or fail. And that verdict has teeth: when a threshold fails, the test exits with an error code that a continuous integration (CI) pipeline understands as "I failed," and that stops the deploy without anyone having to approve anything by hand. That's a performance quality gate: an automatic gate that only opens if the app handles the load with the latency and reliability you promised.

Connection to the module: this lesson is the map. It explains the conceptual leap —from measuring to judging—, presents the piece that makes it possible (the threshold and the exit code), declares the /quote_cpu endpoint this module adds to the canonical API to be able to see a p95 cross a limit for real, and fixes the boundaries with the rest of the guide. It reuses everything before: the script and the VUs (M2), the metrics you'll put under a limit (M3), the profiles that generate the load (M4). What comes after —the correctness check()s and the scenarios (M6), and the analysis and CI pipeline in depth (M7)— uses this gate as a foundation. Here we build the verdict; M7 installs it in the whole pipeline.

The factory traffic light

Imagine a bottling line. At the end of the belt, each bottle passes through a station with a scale and a camera. The station doesn't hand you a report with each bottle's exact weight for you to read bottle by bottle —there'd be thousands per hour, no one would look at them—. It does something much more useful: it has a rule ("the bottle must weigh between 498 and 502 grams and be well capped") and, with that rule, it turns on a light. Green: the bottle continues to packing. Red: a mechanical arm pulls it off the belt. The station turned a continuous measurement (the weight, in grams) into a binary decision (pass / no pass), and that decision acts on its own: it removes the bad bottle without a supervisor having to watch.

A threshold is exactly that rule, and the exit code is exactly that mechanical arm. The scale is your p95 measurement; the rule is p(95) < 200; the green or red light is the pass/fail verdict; and the arm that removes the bottle is the CI pipeline that, on seeing the error code, blocks the deploy. Without the station, someone would have to weigh bottles by hand and trust their eye —slow, expensive, and fallible—. With the station, quality protects itself, at the speed of the belt. This module teaches you to build that station for your API's performance.

What a quality gate is (and why performance needs one)

A quality gate is an automatic check a code change must pass before advancing toward production. You already know several from other test families, even if you didn't call them that:

• The unit tests that run in CI are a gate: if one fails, the build goes red and the merge is blocked.
• A coverage gate is another: if test coverage drops below 80%, the build fails (you'll see it in depth in lesson 6, because it's the closest relative of what we do here).
• The linter is a style gate: if the code doesn't meet the formatting rules, it doesn't pass.

They all share the same anatomy: they measure something, compare it against a limit, and emit a binary verdict the pipeline respects. Performance was left off that list for a silly reason: measuring it required launching load, and launching load seemed like "something you do now and then, by hand, before a big launch." k6's thresholds correct that. They turn the load test into just another gate —one that can be run on every change— and so performance goes from being an intermittent, subjective concern to an objective, continuous condition, just like correctness or coverage.

The consequence is profound: a performance regression becomes as hard to merge as a broken test. If your change makes the p95 rise from 180 ms to 260 ms and your gate demands p(95) < 200, the build goes red and the team finds out before deploying, not after users complain. That's what this module teaches you to set up.

The endpoint this module declares: /quote_cpu

There's a practical problem for teaching thresholds against Reservo: the canonical API is very fast. Running on localhost, POST /quote responds in a few milliseconds even with dozens of concurrent clients. That's great for the real app, but terrible for seeing a limit fail: if the p95 always comes out at 8 ms, any reasonable limit (p(95) < 200) always passes, and you'd never see the red light. We need a target whose p95 really rises under load, to be able to see it cross the limit.

As the guide's design established, a module that needs extra behavior adds and declares it. Module 3 declared a slow endpoint (/quote_slow) with fixed latency. This module declares a different and more interesting one for our purpose: /quote_cpu, which does the same as /quote (receives {room, tier, hours}, returns {price_cents}, with the same anchor numbers 7500 and 6000) but before responding does real CPU work: a loop that sums squares. The key is why that degrades under load.

Python has a GIL (Global Interpreter Lock): a lock that allows only one thread to execute Python bytecode at a time. Waiting work (sleeping, waiting on the network) releases the GIL, but pure CPU work —like our sum loop— holds it. Consequence: when 4 concurrent requests arrive at /quote_cpu, they barely compete (there's plenty of CPU and little overlap). But when 120 arrive at once, they all want the GIL for their loop, and the interpreter serializes them: each request waits its turn behind the others. That wait is real, it accumulates, and the p95 spikes from tens of milliseconds to hundreds. It's a faithful model of a CPU-bound backend (one that does heavy computation per request) saturating under traffic —exactly the kind of degradation a load test exists to catch—.

This is the addition to the canonical server. The rest of the server (the one from module 1's lesson 6) doesn't change; only the burn_cpu function, the /quote_cpu branch in do_POST, and the CPU_WORK constant are added:

# --- ADDED by module 5 to Reservo's canonical server ---

# How much CPU work /quote_cpu does per request (loop iterations).
# Serialized by the GIL: under high concurrency, each request waits its turn.
CPU_WORK = 60000


def burn_cpu(iterations):
    """Real CPU work (not sleep): a loop the GIL serializes between threads."""
    total = 0
    for i in range(iterations):
        total += i * i
    return total


# ...inside do_POST, after validating and BEFORE computing the price:
#     if self.path == "/quote_cpu":
#         burn_cpu(CPU_WORK)          # CPU work serialized by the GIL
#     cents = price_cents(room, tier, hours)
#
#     if self.path in ("/quote", "/quote_cpu"):
#         self._send_json(200, {"price_cents": cents})

Notice the honesty of the design: /quote_cpu doesn't fake being slow with a fixed sleep. It does real work whose latency depends on the load —cheap when there's low concurrency, expensive when there's a lot—. That's why the same endpoint, with the same limit, passes under light load and fails under heavy load. That load dependence is precisely what makes it a good target for teaching a threshold's pass/fail: the degradation comes from the load, which is the variable a load test manipulates.

The environment rule (again, because it matters)

This guide has an honesty rule that governs every lesson, and in this module it's especially relevant because the verdict —the pass/fail— is the heart of the matter:

• What is actually run and cited is Python. The Reservo API (with /quote_cpu), the load generator, the p95/error/checks computation and —the star of this module— the evaluate_thresholds function with its real exit code (sys.exit), run for real against localhost and their output is pasted as-is. When you see a block with the command python3.14 ... and a GATE: PASS or GATE: FAIL, that happened.
• k6 goes as labeled content. k6 isn't installed in this environment (it's a Go binary with its own JavaScript runtime; node doesn't run it). Its options.thresholds blocks and its summary are content, faithful to k6's official documentation, never a fabricated output presented as executed. When you see a k6 JavaScript block or its summary, it will be labeled as content.

This separation isn't an annoying limitation: it's what makes the learning solid. You see the real mechanism of pass/fail with your own metrics in Python (a broken threshold → sys.exit(1) → the shell receives $? = 1), and you see the industrial form of that same mechanism in k6 (a broken threshold → k6 run exits with 99 → CI fails). They're the same idea at two scales, and understanding Python's is understanding k6's from the inside.

This module's boundaries

So you know what's this module's and what's not:

• The k6 script and the VUs (how the default function is written, how virtual users are launched) are module 2's. Here we take them as known: a threshold is declared on a script that already exists.
• The metrics (what the p95 is, how it's computed, what the error rate is) are module 3's. Here we don't re-explain them; we put them under a limit. A threshold doesn't invent a new metric: it puts a rule on one you already know how to measure.
• The load profiles (stages, ramp-up, executors) are module 4's. Here we generate the load with the usual Python generator; the threshold judges the result, whatever the profile that produced it.
• The correctness check()s and the scenarios are module 6's. Watch a nuance: in this module the threshold checks: ['rate>0.99'] appears, which uses the checks rate —but check() as a tool for verifying correctness under load is developed in depth in M6—. Here we treat it as just another metric to put a limit on.
• The analysis in depth and the complete CI pipeline (the .github/workflows/load.yml, exporting results, detecting regressions) are module 7's. Here we build the verdict (the exit code); M7 installs it in the whole pipeline and shows the YAML.

In one sentence: this module is about the threshold and the pass/fail. How a metric becomes a rule, how that rule becomes an exit code, and how that exit code becomes a gate that approves or rejects.

What the destination looks like (an executed preview)

So the map isn't only words, here's the end of the road, actually executed. It's the Python gate evaluating /quote_cpu's real metrics under two loads. First, light load (4 concurrent clients): the three rules pass, the gate opens, the exit code is 0.

What to expect — with low concurrency the p95 stays in milliseconds, well below the 200 ms limit; all green:

$ python3.14 threshold_gate.py http://127.0.0.1:PORT /quote_cpu 400 4 200
# /quote_cpu  |  400 requests, concurrency 4
THRESHOLD                              MEASURED           RESULT
----------------------------------------------------------------------
http_req_duration: p(95) < 200ms       p(95) = 9.74ms     PASS
http_req_failed:   rate < 1.00%        rate  = 0.00%      PASS
checks:            rate > 99.00%       rate  = 100.00%    PASS
----------------------------------------------------------------------
GATE: PASS  (exit code 0)

And now the same rule, the same endpoint, but with heavy load (120 concurrent clients): the GIL serializes the CPU work, the p95 spikes over the limit, and the gate closes with exit code 1.

What to expect — with 120 concurrent the p95 crosses 200 ms; only that rule fails, and that's enough for the whole gate to fail:

$ python3.14 threshold_gate.py http://127.0.0.1:PORT /quote_cpu 2000 120 200
# /quote_cpu  |  2000 requests, concurrency 120
THRESHOLD                              MEASURED           RESULT
----------------------------------------------------------------------
http_req_duration: p(95) < 200ms       p(95) = 246.96ms   FAIL
http_req_failed:   rate < 1.00%        rate  = 0.00%      PASS
checks:            rate > 99.00%       rate  = 100.00%    PASS
----------------------------------------------------------------------
GATE: FAIL  (exit code 1)

That's the whole module in two runs: the same app, the same rule, and a verdict that changes from green to red according to the load —automatic, binary, with an exit code a pipeline respects—. (The exact numbers vary a bit in each run, because they depend on how the operating system distributes the CPU; what doesn't vary is the story: light passes, heavy fails.) The rest of the lessons take this apart piece by piece.

Common mistakes

Believing that measuring is the same as gating. What happens: a team runs load tests, saves the p95 charts in a pretty dashboard, and believes it "has performance covered." But nobody looks at the dashboard with discipline, and a regression sneaks in anyway. Why it happens: having the number gets confused with acting on the number. How to detect it: if your load test can't fail a build, it's not a gate, it's a report. How to fix it: add thresholds and connect them to the exit code, so the verdict acts on its own (this whole module).

Thinking k6 ran here. What happens: someone sees an options.thresholds block or a k6 summary and cites it as "what this guide measured." Why it happens: the k6 content looks very real. How to detect it: k6 isn't installed; the executed numbers always come with a python3.14 ... command and a GATE: PASS/FAIL. How to fix it: remember the rule —Python is run and cited; k6 is labeled content, faithful to the docs—.

Assuming /quote_cpu is an unrealistic trick. What happens: someone objects that degrading on purpose with a CPU loop is "cheating." Why it happens: they don't see it's a faithful model of a real backend. How to detect it: if you think production endpoints never get slow under load, you haven't seen enough incidents. How to fix it: understand that /quote_cpu reproduces, small and controlled, exactly what happens to a CPU-bound service when it gets more traffic than its CPU can handle —the degradation the gate must catch—.

Exercises

Exercise 1 — Recognize the gate pattern. The following four checks are quality gates from different families. For each, identify: (i) the metric it measures, (ii) the limit, (iii) what happens if it fails. (a) pytest --cov-fail-under=80. (b) A linter that rejects lines longer than 100 characters. (c) http_req_duration: ['p(95)<200'] in k6. (d) The factory station that weighs bottles.

See solution

• (a) Metric: test coverage (%). Limit: 80%. If it fails (coverage < 80%): the command exits with code ≠ 0 and the build goes red.
• (b) Metric: each line's length (characters). Limit: 100. If it fails (there are longer lines): the linter exits with code ≠ 0 and the CI step fails.
• (c) Metric: p95 latency (http_req_duration). Limit: 200 ms. If it fails (p95 ≥ 200): k6 run exits with code 99 and the CI step fails.
• (d) Metric: the bottle's weight (grams). Limit: 498–502 g. If it fails: the light goes red and the arm removes the bottle.

All four have the same anatomy: measure → compare with a limit → binary verdict that acts on its own. That's the module's unifying idea.

Exercise 2 — Predict the verdict. The threshold is p(95) < 200. For each measured run, say whether the gate passes or fails, and why. (a) p95 = 9.74 ms. (b) p95 = 246.96 ms. (c) p95 = 199.9 ms. (d) p95 = 200.0 ms.

See solution

• (a) PASSES. 9.74 < 200. Light load; plenty of margin.
• (b) FAILS. 246.96 ≥ 200. Heavy load; the p95 crossed the limit.
• (c) PASSES. 199.9 < 200. By a hair, but it passes: the rule is strictly less-than.
• (d) FAILS. 200.0 isn't less than 200. The threshold p(95)<200 demands strictly less; exactly 200.0 doesn't meet it. (It's a good reminder that the operator matters: < isn't <=.)

Exercise 3 — Explain the GIL in /quote_cpu. In two or three sentences, explain why /quote_cpu responds fast with 4 concurrent clients but slow with 120, mentioning the GIL. Why is that load dependence exactly what we need to teach thresholds?

See solution

/quote_cpu does pure CPU work (a sum loop) per request, and Python's GIL allows only one thread to execute bytecode at a time. With 4 concurrent clients there's little overlap and they barely compete for the GIL, so each request finishes fast (p95 of milliseconds). With 120 clients, they all want the GIL for their loop at once and the interpreter serializes them: each request waits its turn behind many others, that wait accumulates, and the p95 spikes to hundreds of milliseconds. That load dependence is ideal for teaching thresholds because it makes the same endpoint with the same limit pass under light load and fail under heavy load —the pass/fail as a function of the load, which is exactly what a threshold judges—.

Summary and next step

This module makes the leap from measuring to judging. Until now a load test produced numbers for a human to interpret; from here on, it produces a binary verdict —pass or fail— that acts on its own. The piece that makes it possible is the threshold: a rule on a metric (p(95) < 200) that, when broken, makes the test exit with an error code that a CI pipeline understands as "I failed" and uses to block the deploy. That's a performance quality gate, and it has the same anatomy as a coverage gate or a broken test: measure, compare with a limit, emit a verdict the pipeline respects.

You saw the factory-station analogy (scale = measurement, rule = threshold, red light = verdict, mechanical arm = exit code), the reason performance deserved its own gate, and the endpoint this module declares —/quote_cpu, with CPU work serialized by the GIL— to be able to see a p95 cross the limit for real. And you saw the destination executed: the same gate, green under light load (p95 = 9.74 ms) and red under heavy load (p95 = 246.96 ms), with its real exit code.

Before moving on you should be able to: explain how measuring differs from gating; name the common anatomy of all quality gates; and explain why /quote_cpu degrades under load and why that makes it a good target. What comes next, in lesson 2, is the precise definition of a threshold and of the pass/fail, with the first executable version of the Python mirror: a function that takes your real latencies and returns a verdict. We start building the station.

Resources

• k6 — Thresholds — the official reference that governs the whole module: what thresholds are, how they're declared, and why they make the test fail. The source of all the k6 content here.
• Google SRE Book — Service Level Objectives — the foundation of why a limit is put on a metric and how it's chosen (we develop it in lesson 5). SLI, SLO, SLA, and error budget.
• sys.exit — Python documentation — the mechanism the Python gate returns its exit code with (0 = pass, ≠ 0 = fail), the executable mirror of k6's exit code. We use it in depth in lesson 4.
• k6 — Python's GIL doesn't apply to k6 (context) — a contrast note: k6 runs the VUs in true parallel (it's Go); the GIL that degrades /quote_cpu is a feature of our Python server, the target, not the load generator. Useful for not confusing where the serialization lives.