4. The thresholds tied to the SLO

Overview

Lesson 3's profile produced a p95 of 245 ms under stress. But a number isn't a verdict: is 245 ms good or bad? The answer isn't given by the technique, it's given by the business, in the form of an SLO (Service Level Objective) —the performance promise Reservo makes to its users—. In this lesson we put on the capstone its thresholds: http_req_duration: ['p(95)<200'], http_req_failed: ['rate<0.01'], and checks: ['rate>0.99'], each tied to a concrete SLO. We see where each number comes from (why 200 ms and not 150 or 500), why the threshold is evaluated over the whole run as k6 does, and how the same measured p95 passes or fails depending on the SLO you choose —the proof that the threshold is a business decision, not a technical detail—. And we confirm the exit code that makes it a gate: 1 in the Python gate, 99 in k6.

Connection to the module: this is the capstone's third piece —the verdict that turns lesson 3's metrics into a pass/fail—. It fully reuses module 5: what a threshold is, how it's written in k6, the exit code ≠ 0 that fails CI, and how the threshold is chosen from the SLO/SLA. Here we don't re-explain the exit code mechanism; we use it to tie the capstone's thresholds to Reservo's SLOs. Lesson 5 will read the metrics in depth; lesson 6 will run the complete gate (evaluate + exit with a code + export). Here we focus on the thresholds and their origin.

The finish line the business defines, not the runner

Imagine a race. A runner crosses the finish in 3 hours 45 minutes. Is it a good time? The question has no answer until someone defines the cutoff line. If it's an amateur club's marathon, 3:45 qualifies easily. If it's the qualification for the Olympic Games, not even close. The same time, two verdicts, depending on the mark the competition decided to require. The cutoff line isn't set by the runner looking at their watch; it's set by the organization according to what the race means. And once set, it's binary: you crossed before it or you didn't.

A threshold is that cutoff line, and the SLO is who defines it. Your test measures p(95) = 218 ms —that's the runner's time, a fact—. If Reservo's SLO says "95% of quotes respond in under 200 ms," that p95 fails (it crossed late). If the SLO were 300 ms, the same p95 would pass. The threshold number doesn't come from the test or from your technical intuition; it comes from what the business promised the user. That's why a threshold without an SLO behind it is a finish line painted at random: it passes or fails builds without anyone knowing why. Choosing the threshold well —tying it to a real promise— is what makes the gate protect something.

The thresholds in k6 (content)

Here are the capstone's thresholds, added to lesson 3's options. Remember: labeled content, faithful to k6's documentation, not run here.

// CONTENT (not run here): k6 is not installed.
// Reference: grafana.com/docs/k6 (options → thresholds).
export const options = {
  stages: [ /* smoke -> load -> stress -> ramp-down, lesson 3 */ ],

  thresholds: {
    // LATENCY: 95% of requests under 200 ms (the latency SLO).
    http_req_duration: ['p(95)<200'],
    // AVAILABILITY: less than 1% of requests fail (the error SLO).
    http_req_failed: ['rate<0.01'],
    // CORRECTNESS: more than 99% of checks pass (the responses are correct under load).
    checks: ['rate>0.99'],
  },
};

Each threshold is a rule metric: [expression], and if any breaks, k6 run exits with a code ≠ 0 and the build goes red. The three cover the three questions a load test answers:

• http_req_duration: ['p(95)<200'] — is it fast? The latency p95 (the percentile that represents the tail user, M3) must stay under 200 ms. It's the latency SLO: the speed promise.
• http_req_failed: ['rate<0.01'] — is it available? Less than 1% of requests may fail. It's the availability SLO: the promise that the system responds. Note that latency and availability are independent —the system can be slow (broken p95) but available (0% error), as we saw in lesson 3—.
• checks: ['rate>0.99'] — is it correct? More than 99% of the check()s (status 200, correct price, confirmed booking) must pass. It's the correctness SLO: the promise that the responses are good, not just fast. A fast 200 with a wrong price breaks this threshold even if the other two pass.

Where each number comes from (the SLO)

The three numbers —200 ms, 1%, 99%— aren't arbitrary; each translates a business promise. Here's how each is reasoned for Reservo (M5):

• 200 ms p95. UX research says that below ~100 ms a response feels instantaneous, and up to ~200-300 ms the user perceives it as fluid; beyond that, they start to notice the wait. Reservo promises that quoting feels agile, so it sets the latency SLO at p(95) < 200 ms: 95% of users see the price in under a fifth of a second. The p95 (not the average) is chosen because the average hides the tail users, who are the ones who get frustrated (M3).
• 1% of errors. A typical availability SLO for a web service is expressed in "nines": 99% available = up to 1% error tolerated. For a booking flow that's already generous (a 1% of failed quotes is a lot); many services aim for 99.9%. The capstone uses rate < 0.01 as a reasonable starting line, knowing the business could demand more.
• 99% of checks. Under load, we accept that a minimal fraction of responses may come out wrong (a rare race condition, a timeout), but we require that almost all be correct: rate > 0.99. If more than 1% of the quotes returns a wrong price, something is broken in the logic under concurrency, and that must fail the build.

The underlying lesson: the threshold is a business decision disguised as a technical number. Changing 200 to 300 isn't a configuration adjustment; it's changing what you promise the user. That's why the SLO is agreed with the business, not invented by whoever writes the test.

Why over the whole run (as k6)

An important detail the capstone inherits from M4: k6's thresholds are evaluated over all the requests of the complete run (smoke + load + stress + ramp-down aggregated), not over a single stage. The p(95)<200 looks at the p95 of all the requests together. Since the lower-load phases (smoke, load) contribute fast requests, the aggregate p95 comes out a bit lower than the p95 of the peak alone —but if the stress is bad enough, the aggregate still crosses the threshold—. The Python generator does exactly this: it measures the p95 per stage (to read the shape, lesson 3) and the p95 of the whole run, and evaluates the threshold over the latter, to be faithful to how k6 decides.

The same p95, two verdicts (executed)

To prove that the threshold is what rules, let's separate measuring from judging. The run already wrote its metrics to a results.json (that's lesson 6); here a small evaluator reads them and judges them against an SLO. It's the same pattern from M5: the verdict is a function of the metrics and the threshold.

# check_thresholds.py — reads a results.json and judges it against the SLO (M5).
# Usage: python3.14 check_thresholds.py <results.json> [p95_limit_ms]
import json, sys

report = json.load(open(sys.argv[1]))
P95_LIMIT = float(sys.argv[2]) if len(sys.argv) > 2 else report["slo"]["p95_ms"]
agg = report["aggregate"]
rows = [
    ("http_req_duration: p(95)", agg["p95"] < P95_LIMIT),
    ("http_req_failed:   rate",  agg["error_rate"] < report["slo"]["error_rate"]),
    ("checks:            rate",  agg["checks_rate"] > report["slo"]["checks_rate"]),
]
# ...prints the table and exits with 0 (all PASS) or 1 (any FAIL).
sys.exit(0 if all(ok for _, ok in rows) else 1)

First, the healthy run (/quote) against the 200 ms SLO:

What to expect — the aggregate p95 (21.72 ms) is well below 200; the three thresholds pass and it exits with code 0:

$ python3.14 check_thresholds.py green.json
SLO: p(95) < 200ms  ·  error < 1%  ·  checks > 99%
run: /quote  (80082 requests)
------------------------------------------------------------
http_req_duration: p(95)       21.72ms  < 200ms   PASS
http_req_failed:   rate          0.00%  < 1%      PASS
checks:            rate        100.00%  > 99%     PASS
------------------------------------------------------------
VERDICT: PASS  (exit code 0)
$ echo $?
0

Now the degraded run (/quote_cpu) against the same 200 ms SLO:

What to expect — the aggregate p95 (218.51 ms) crosses the threshold; the latency threshold fails and it exits with code 1:

$ python3.14 check_thresholds.py red.json
SLO: p(95) < 200ms  ·  error < 1%  ·  checks > 99%
run: /quote_cpu  (9000 requests)
------------------------------------------------------------
http_req_duration: p(95)      218.51ms  < 200ms   FAIL
http_req_failed:   rate          0.00%  < 1%      PASS
checks:            rate        100.00%  > 99%     PASS
------------------------------------------------------------
VERDICT: FAIL  (exit code 1)
$ echo $?
1

And now the proof that the threshold is what rules: the same metrics from the degraded run, but against a more lenient SLO of 300 ms:

What to expect — with the cutoff line at 300 ms, the 218.51 p95 now passes; the same number, opposite verdict, only because the promise changed:

$ python3.14 check_thresholds.py red.json 300
SLO: p(95) < 300ms  ·  error < 1%  ·  checks > 99%
run: /quote_cpu  (9000 requests)
------------------------------------------------------------
http_req_duration: p(95)      218.51ms  < 300ms   PASS
http_req_failed:   rate          0.00%  < 1%      PASS
checks:            rate        100.00%  > 99%     PASS
------------------------------------------------------------
VERDICT: PASS  (exit code 0)
$ echo $?
0

The same measured p95 (218.51 ms) fails against a 200 ms SLO and passes against a 300 ms one. The test didn't change, the metrics didn't change —what changed is the cutoff line, and with it the verdict and the exit code—. That demonstrates the threshold is a business decision: choosing 200 instead of 300 is choosing to promise a fifth of a second instead of almost a third. Setting it at random is painting the finish wherever it lands.

The exit code that makes it a gate

The verdict only stops a deploy if it's translated to an exit code (M5). The Python gate exits with 1 when a threshold fails (you saw it above: $? = 1 in the red run). k6 does the same with a detail: when one or more thresholds fail, k6 run exits with code 99 (ThresholdsHaveFailed), reserved for "the test ran fine but didn't meet the thresholds." For the pipeline the difference between 1 and 99 doesn't matter: both are ≠ 0, so the step fails and the deploy is blocked. A broken performance threshold fails the build exactly like a broken unit test: both exit with code ≠ 0.

Common mistakes

Choosing the threshold at random (or copying it from a tutorial). What happens: p(95)<500 is set because "it sounded reasonable" or because it was in an example, without asking what the business promises. Why it happens: it's easier to invent a number than to agree on an SLO. How to detect it: if no one can say why 500 and not 200, the threshold is arbitrary. How to fix it: tie each threshold to a concrete SLO —a latency, availability, or correctness promise the business makes to the user—. A gate with an arbitrary threshold protects something arbitrary; one tied to the SLO protects the real promise.

Setting only the latency threshold and forgetting error and checks. What happens: http_req_duration is watched and a fast run is taken as good, even if 5% of requests fail or return broken prices. Why it happens: latency is the star metric and steals the attention. How to detect it: if your thresholds only have one rule, you cover only one of the three questions. How to fix it: put the three —latency, availability, and correctness—. A fast system that fails or lies isn't a good system; the three thresholds together are the complete verdict.

Confusing the peak's p95 with the p95 the threshold evaluates. What happens: it's seen that the stress gave p95 = 245 ms and it's expected that the threshold reports 245, but k6 (and the gate) report the aggregate (218). Why it happens: it's forgotten that the threshold looks at the whole run, not a single stage. How to detect it: if your "threshold's" p95 doesn't match the peak's, it's because the aggregate includes the fast phases. How to fix it: remember the threshold evaluates all requests together; the per-stage p95 is for reading the shape, the aggregate is for the verdict. Both are real; they measure different things (M4).

Exercises

Exercise 1 — Pass or fail? For a run with aggregate p95 = 180 ms, error rate = 0.4%, and checks = 99.6%, say whether each threshold passes and what the global verdict is, against the SLO p(95)<200, rate<0.01, checks>0.99.

Exercise 2 — The same number, two businesses. A run measures p95 = 250 ms. (a) Does it pass against an SLO of p(95)<200? (b) Against one of p(95)<300? (c) What would you tell the team that wants to raise the SLO to 300 just so the test stops failing?

Exercise 3 — Write the thresholds of a stricter SLO. Reservo tightens its promise: 99% of quotes must respond in under 150 ms, and availability rises to 99.9%. Write the corresponding k6 thresholds (keep the checks one at 99%).

thresholds: {
  http_req_duration: ['p(99)<150'],  // now the p99 (not the p95) under 150 ms
  http_req_failed: ['rate<0.001'],   // 99.9% available = < 0.1% error
  checks: ['rate>0.99'],
}

Summary and next step

In this lesson you put on the capstone its verdict: the thresholds tied to the SLO. You wrote the three in k6 (content) —http_req_duration: ['p(95)<200'] (latency), http_req_failed: ['rate<0.01'] (availability), checks: ['rate>0.99'] (correctness)— and saw where each number comes from: not from the technique or the intuition, but from a business promise (the SLO). You confirmed the threshold is evaluated over the whole run (like k6), and —the underlying lesson— that the same 218.51 ms p95 fails against a 200 ms SLO and passes against a 300 ms one: the threshold is a business decision disguised as a number, and the cutoff line is defined by the promise, not the runner. And you tied the verdict to its exit code: 1 in the Python gate, 99 in k6, both ≠ 0.

You fully reused module 5 (threshold, exit code, choosing the threshold from the SLO). Before moving on you should be able to: write the three thresholds of an SLO; explain why the threshold is a business decision; and say why it's evaluated over the whole run. What comes next, in lesson 5, is to read in depth the metrics the run produces (p50/p95/p99, RPS, error rate) and the equivalent k6 summary —the instruments the threshold judges—.

Resources

• k6 — Thresholds — the official reference for how thresholds are written (p(95)<200, rate<0.01, checks>0.99) and that a broken threshold makes k6 run exit with code ≠ 0. The source of this lesson's k6 content.
• Google SRE Book — Service Level Objectives — the chapter that explains what an SLO is and why the threshold is derived from a business promise, not the technique; the foundation of "where each number comes from."
• Google SRE Workbook — Implementing SLOs — how the latency, availability, and correctness objectives are chosen in practice; the criterion behind the 200 ms, the 1%, and the 99%.
• k6 — Error codes (code 99) — the source code where k6 defines ThresholdsHaveFailed = 99, the exit code it returns when a threshold fails. The source of the "99".