Module 5: Thresholds Pass Fail And Slos

5. SLO/SLA: choosing the threshold with judgment

Next →

Overview

All the machinery of the previous modules is precise: you measure the p95 rigorously, compare it with a limit, a verdict comes out, the verdict becomes an exit code that gates the deploy. But there's a number at the center of all that machinery we haven't justified: the 200 of p(95) < 200. Why 200 and not 150, or 500, or 1000? This lesson answers that question, and the answer is uncomfortable for an engineer: the limit's value isn't a technical decision, it's a business and user decision. A perfect gate with an arbitrary limit protects nothing —either it rejects healthy builds, or it lets slow apps through—. You'll learn the vocabulary the industry uses to choose these numbers with judgment —SLI, SLO, SLA and the error budget—, where a limit that actually means something comes from, and why a too-loose limit makes the gate useless while a too-strict one makes it unstable. And you'll see it executed in the most compelling way possible: the same load, with the same measured p95, passing against one limit and failing against another —living proof that the threshold is a choice, not a datum nature hands over—.

Connection to the module: lessons 2–4 gave you the threshold's machinery (the rule, the verdict, the exit code) treating the limit's value as an input. This lesson fills that gap: how that value is chosen. It's the hinge between the technical and the human side of the module. What comes next (lesson 6) shows that this same gate is a sibling of coverage gates; lesson 7 puts different limits on different parts of the system —which only makes sense if you know how to choose each one—. Here you learn to set the number.

The speed limit isn't set by the car

Why is the limit on a highway 120 km/h and not 90, or 160? The engineer who designed the car didn't decide it —the engine can do 200—. Someone decided it looking at something else: how long a car takes to brake, how serious a crash is at each speed, how many extra people die per each extra 10 km/h. The number comes from the acceptable harm to people, not the machine's capacity. A limit of 300 km/h would be useless (it'd never be broken, it protects no one); one of 40 would be absurd (everyone would violate it, and it'd slow the economy without gaining proportional safety). The good limit lives at the point where the cost to the user starts to be unacceptable.

A performance limit is identical. The 200 of p(95) < 200 doesn't come from what your server can do, but from what your user tolerates before the experience degrades. Usability research has reference points: around 100 ms an interaction feels instantaneous; up to ~1 second the user keeps the thread of what they were doing; past that, they start to lose attention and get frustrated. For a quote the user waits for on screen, a p95 of 200 ms means "19 of every 20 users feel the app is snappy." That's the reasoning behind the number: the limit is set by the harm to the user, not the engine.

The vocabulary: SLI, SLO, SLA

The discipline that formalizes this is site reliability engineering (SRE, from Google), and it has three terms worth not confusing, because each lives in a different layer:

  • SLI — Service Level Indicator. What you measure. It's the raw metric: "the p95 latency of POST /quote," "the percentage of requests with status < 400." An SLI is a measurable fact. Modules 3 and 4 were entirely about producing SLIs.
  • SLO — Service Level Objective. What you commit to meeting. It's the SLI plus a limit and a target: "the p95 of /quote will be below 200 ms 99% of the time." The SLO is the internal goal the team commits to maintaining. Your gate's threshold is an SLO made executable.
  • SLA — Service Level Agreement. What you promise a customer contractually, with consequences. It's an SLO written into a commercial contract, with penalties if broken: "if availability drops below 99.9% monthly, we refund you 10%." The SLA is usually looser than the internal SLO, on purpose: the team sets itself a stricter goal (SLO) to have margin before breaking the contractual promise (SLA).

The chain is: you measure an SLI, you commit to an SLO (the SLI + limit), and maybe you promise a customer an SLA (a contractual SLO). Your threshold —p(95) < 200— is the executable form of an SLO. When the gate passes, you're meeting your service objective; when it fails, you're breaking it. That's why choosing the limit is choosing your SLO, and choosing your SLO is a business act: it defines what you promise, implicitly or explicitly, to your users.

The error budget: why the limit isn't "always"

Notice a detail of the SLO above: "the p95 will be below 200 ms 99% of the time." It doesn't say "always." That acknowledgment that perfection isn't the goal is formalized in the error budget: if your goal is to meet it 99% of the time, you have a 1% budget to fail without breaking the promise. That 1% isn't a failure to avoid at all costs: it's a resource you can spend —on deploying fast, on experimenting, on taking risks—. A team that never spends its error budget probably set a too-loose SLO and is being excessively conservative; one that exhausts it all the time has a real reliability problem.

For a performance gate, the error budget has a practical consequence: don't make the limit so strict that it spends the budget on noise. If your real p95 hovers around 190 ms with natural variation, a limit of p(95) < 200 will fail now and then out of pure sampling luck —an unstable (flaky) gate, which fails healthy builds and teaches the team to ignore the red—. The error budget reminds you that the limit must leave room for normal variation: it's set where it separates "this is a real regression" from "this is noise," not up against the typical value.

The living proof: the same p95, two verdicts

Here's the demonstration that fixes the whole idea. We're going to measure /quote_cpu under the same heavy load (2000 requests, 120 concurrent) twice, changing only the limit. The measured p95 will be almost the same in both (the load is the same); the only thing that changes is the rule we compare it against. First, a strict limit of 200 ms:

What to expect — with a p95 of ~243 ms, the 200 ms limit breaks: FAIL, exit code 1:

$ python3.14 threshold_gate.py http://127.0.0.1:PORT /quote_cpu 2000 120 200
# /quote_cpu  |  2000 requests, concurrency 120
THRESHOLD                              MEASURED           RESULT
----------------------------------------------------------------------
http_req_duration: p(95) < 200ms       p(95) = 242.86ms   FAIL
http_req_failed:   rate < 1.00%        rate  = 0.00%      PASS
checks:            rate > 99.00%       rate  = 100.00%    PASS
----------------------------------------------------------------------
GATE: FAIL  (exit code 1)

Now the same load, the same endpoint, a practically identical p95 —but with a loose limit of 500 ms:

What to expect — the p95 (~243 ms) is now below 500 ms: PASS, exit code 0. Nothing changed in the app; the rule changed:

$ python3.14 threshold_gate.py http://127.0.0.1:PORT /quote_cpu 2000 120 500
# /quote_cpu  |  2000 requests, concurrency 120
THRESHOLD                              MEASURED           RESULT
----------------------------------------------------------------------
http_req_duration: p(95) < 500ms       p(95) = 243.43ms   PASS
http_req_failed:   rate < 1.00%        rate  = 0.00%      PASS
checks:            rate > 99.00%       rate  = 100.00%    PASS
----------------------------------------------------------------------
GATE: PASS  (exit code 0)

Read it slowly, because it's the whole argument of the lesson in two runs. The app behaved the same both times (p95 of 242.86 and 243.43 ms —the same latency, with the normal sampling noise). The verdict went from FAIL to PASS. The only thing that decided the result was the limit's number, which we chose. This demonstrates, in an indisputable way, that the limit isn't a datum the measurement hands over: it's a decision you make before measuring, that determines what counts as "good enough." The measurement is objective (243 ms is 243 ms); the approval is a choice (is 243 ms acceptable? depends on your SLO).

Hence the responsibility: if you choose 500 ms because "that way it always passes," your gate protects nothing —you'd let through an app that makes 1 in every 20 users wait a quarter second—. If you choose 200 ms because your user needs snappiness, the gate catches that degradation. The same mechanism, with two limits, is either a guardian or theater. The difference is entirely the judgment with which you set the number.

How to choose the number, in practice

So, where do you get the concrete limit for your API? A sensible procedure:

  1. Start from the user and the business. What is this operation for whoever uses it? A synchronous interaction (the user waits looking at the screen) calls for a low p95 (hundreds of ms); a background job (a report generated and sent by email) tolerates seconds. The usability reference points (100 ms = instantaneous, 1 s = keeps attention) are your anchor.
  2. Look at your baseline. Run the test with no limit and observe the typical p95 under realistic load. The limit must be above that healthy baseline with margin for variation (to not be flaky), but below the point where the user would start to suffer (so it means something).
  3. Choose the percentile according to whom you protect. p(95) protects 19 of every 20; p(99) also the extreme tail. Critical operations (payments, login) usually gate the p99; the rest, the p95.
  4. Leave error budget. Don't stick the limit to the typical value. Set it where it separates regression from noise.
  5. Be stricter in the internal SLO than in the SLA. If you promise a customer 500 ms contractually, gate your CI at 400: that way you find out before breaking the promise.

The result is a number you can defend: "we gate the p95 of /quote at 200 ms because it's a synchronous interaction, our healthy baseline is ~120 ms, and past 200 ms the user perceives slowness." That's a limit with judgment. "We set it at 200 because it's a round number" isn't.

Common mistakes

Choosing the limit from the server's capacity, not from the user. What happens: someone measures that the app "holds up to 800 ms without falling over" and sets the limit at 800. Why it happens: "what the machine can do" gets confused with "what the user tolerates." How to detect it: if you justify your limit by talking about the server and not the user, it's badly anchored. How to fix it: the limit comes from the harm to the user (SLO), not the engine's limit —just as the speed limit isn't set by the car—.

A limit so loose it never fails. What happens: to avoid annoying red builds, someone sets p(95) < 5000 and the gate always passes. Why it happens: it's optimized for not being bothered, not for protecting the user. How to detect it: if your gate has never failed even with a real regression, it's theater. How to fix it: set the limit where it really separates good from bad; a gate that can't fail isn't a gate.

A limit so close to the typical value it's flaky. What happens: the healthy p95 hovers around 190 ms and the limit is 200; the gate fails one in every three times from noise, and the team learns to re-run until it passes (or to ignore the red). Why it happens: no error budget was left for normal variation. How to detect it: intermittent failures with no code changes. How to fix it: raise the limit until it separates regression from noise (e.g. 250–300 ms if the healthy one is 190 with variation), or stabilize the measurement with more samples. A flaky gate gets ignored, and an ignored gate doesn't protect.

Exercises

Exercise 1 — Classify SLI/SLO/SLA. Label each statement as SLI, SLO, or SLA. (a) "The p95 of /quote was 243 ms in the last run." (b) "We commit to keeping the p95 of /quote below 200 ms 99% of the time." (c) "If monthly availability drops below 99.9%, the customer receives a 10% credit."

See solution
  • (a) SLI. It's the raw measured metric, a fact. "What you measure."
  • (b) SLO. It's the SLI + limit + target, the internal goal the team commits to meeting. "What you propose." (It's what your threshold makes executable.)
  • (c) SLA. It's an SLO in a commercial contract, with a penalty. "What you promise a customer, with consequences."

The chain: you measure the SLI, commit to the SLO, promise the SLA (usually looser than the SLO, to have margin).

Exercise 2 — The same p95, two verdicts. A measured p95 of 243 ms failed against a limit of 200 and passed against one of 500. (a) What changed between the two runs: the app, the measurement, or the rule? (b) What does that demonstrate about the nature of the limit? (c) If your user perceives slowness past 200 ms, which of the two gates should you use, and what does it tell you about the other?

See solution
  • (a) Only the rule (the limit: 200 vs 500). The app behaved the same and the measurement was practically the same (242.86 vs 243.43 ms, normal noise).
  • (b) That the limit is a decision, not a datum. The measurement is objective; the approval is a choice. The same measured fact is "approved" or "rejected" according to the number you chose.
  • (c) The 200 ms one, because it protects what matters to your user (snappiness under 200 ms). The 500 ms one would be theater: it would let through an app that makes 1 in every 20 users wait 243 ms, a degradation your own user judgment considers unacceptable.

Exercise 3 — Defend a limit. For a PDF report generation operation the user requests and receives by email minutes later (doesn't wait on screen), propose a p95 limit and one or two sentences defending it from the user/business. Compare it with /quote's.

See solution

A much looser limit, for example p(95) < 5000 ms (5 s) or even more, and probably on the p95 and not the p99. Defense: "it's an asynchronous job the user doesn't wait for on screen —they receive it by email minutes later—, so a few seconds of latency don't harm their experience; gating this operation at hundreds of milliseconds would waste effort and make the gate flaky with no benefit to the user." Contrast with /quote: /quote is synchronous (the user waits looking), so its limit is strict (200 ms) because the harm to the user starts soon; the report is asynchronous, so it tolerates seconds. Same gate mechanism, opposite limits, because the harm to the user is different —exactly the judgment lesson 7 leverages to set a per-scenario SLO—.

Summary and next step

The limit's value —the 200 of p(95) < 200— isn't a technical decision but a business and user one: it comes from the harm the user tolerates, not what the server can do, just as the speed limit is set by the harm of a crash, not the car's engine. The vocabulary for choosing it with judgment is SRE's: the SLI (what you measure, the raw metric), the SLO (the SLI + limit + target, the internal goal —what your threshold makes executable—), and the SLA (a contractual SLO with a penalty, usually looser). The error budget acknowledges that the goal isn't "always" but a percentage, and reminds you not to stick the limit to the typical value —that makes the gate flaky—.

You saw it unarguably: the same load and the same measured p95 (~243 ms) failed against a limit of 200 and passed against one of 500. Nothing changed in the app; the rule we chose changed. That's the proof that the limit is a choice that determines what counts as "good enough" —and that the same mechanism is a guardian or theater according to the judgment with which you set the number—.

Before moving on you should be able to: distinguish SLI, SLO, and SLA; explain why the limit comes from the user and not the server; and diagnose a useless gate (too loose) or flaky one (too strict). What comes next, in lesson 6, is stepping back and seeing that this performance gate isn't unique: it's the same pattern as the coverage gates of the Testing ecosystem —a dimension of quality turned binary that gates the pipeline—. We'll link with the testing guide to see the whole family of quality gates this one belongs to.

Resources

← Previous4. The nonzero exit code that fails CINext →6. The performance gate, like coverage gates