7. `abortOnFail` and per-scenario thresholds

Overview

The short format you've used so far —http_req_duration: ["p(95)<200"]— covers most cases, but leaves two real needs unmet. The first: when an app is already clearly broken ten seconds into a thirty-second test, continuing to hammer it is waste —you'd want to cut early—. The second: not all parts of a system deserve the same limit —a synchronous quote needs a strict p95, but a heavy-reports endpoint tolerates more—, so you'd want a different SLO for each part. This lesson covers the two k6 tools that solve them: abortOnFail (with delayAbortEval), from the long format, to abort as soon as a critical limit breaks; and thresholds by tag / by scenario, to give each part of the system its own rule. Both are k6 content (labeled), and both have their executable mirror in Python: a multi-target gate that evaluates several endpoints, each with its own limit, and that actually aborts when the critical one breaks —with its real exit code—.

Connection to the module: lesson 5 taught you to choose a limit with judgment; this one teaches you to apply different limits to different parts, which is the natural consequence of each part having a different SLO. abortOnFail connects with the load profiles (module 4): aborting early avoids spending a long test on an already-failed system. It's the module's last technique lesson before the mini-project (lesson 8), which brings everything together. The CI pipeline where these gates live in depth is module 7.

The referee who stops the fight

In boxing there's a rule that protects the fighter: the TKO (technical knockout). If a boxer is taking a beating without defending themselves, the referee doesn't wait for the last round's bell —they stop the fight right there—. There's no point in letting twelve rounds continue when the result is already obvious and continuing only does harm. The referee cuts early because the verdict is already clear and prolonging it is pure waste (and risk).

abortOnFail is that referee. A thirty-minute load test against a system that already blew up at two minutes provides no new information —it only consumes resources, lengthens the pipeline, and keeps punishing a service we already know is failing—. With abortOnFail, k6 stops the test as soon as a critical limit is evaluated as broken, like the referee who stops the fight. The lesson's second tool —the per-scenario thresholds— is like having different rules according to the fight's category: a featherweight and a heavyweight aren't judged by the same criterion, and /quote and /quote_cpu aren't gated with the same limit.

abortOnFail: cutting early

abortOnFail is a property of the long format of thresholds (the one you saw in passing in lesson 3). Instead of a rule as a string, you write an object:

// CONTENT (not run here): abortOnFail and delayAbortEval.
// See grafana.com/docs/k6/latest/using-k6/thresholds/
export const options = {
  vus: 50,
  duration: "5m",   // a long test...
  thresholds: {
    http_req_duration: [
      {
        threshold: "p(95)<200",   // the usual rule
        abortOnFail: true,        // ...but aborts as soon as it breaks
        delayAbortEval: "10s",    // after waiting 10s to gather samples
      },
    ],
  },
};

Three properties:

• threshold — the rule, just like in the short format ("p(95)<200").
• abortOnFail: true — if during the test that limit is evaluated as broken, k6 aborts the whole test at that moment, without waiting for the 5 minutes to finish. It saves time and resources when the app already failed.
• delayAbortEval: "10s" — wait this long before starting to evaluate the limit for aborting. It's important: at the start, with few samples, a p95 can look ugly from pure noise (recall from module 3 that percentiles need lots of data to be stable). delayAbortEval gives the test a few seconds to gather samples before deciding to cut, avoiding aborts from a misleading initial spike.

An operational detail worth knowing (from k6's documentation): when k6 runs in its cloud mode, thresholds are evaluated every 60 seconds, so an abortOnFail can take up to a minute to fire. Locally it's more immediate. In both cases, the idea is the same as the TKO: don't prolong a test whose verdict is already red.

Thresholds by tag and by scenario

The second tool solves that not all of the system deserves the same limit. k6 lets you put a threshold on a sub-metric filtered by a tag, with the syntax metric{tag:value}. The most common case is distinguishing traffic types:

// CONTENT (not run here): thresholds by tag. See grafana.com/docs/k6
thresholds: {
  // Requests tagged as API: strict p95.
  "http_req_duration{type:api}": ["p(95)<200"],
  // Those tagged as static content: looser p95.
  "http_req_duration{type:staticContent}": ["p(95)<500"],
},

For that to work, in the script you tag each request with that tag:

// CONTENT (not run here): tagging a request for the by-tag threshold.
http.post(url, payload, { headers, tags: { type: "api" } });

k6 also adds system tags automatically, and one of them is scenario —the name of the scenario that ran the request (scenarios are k6's way of running several load profiles at once; their complete anatomy is module 6's)—. That lets you have thresholds by scenario without tagging by hand:

// CONTENT (not run here): threshold by scenario (system tag 'scenario').
thresholds: {
  // The 'quotes' scenario (interactive traffic) has a strict SLO.
  "http_req_duration{scenario:quotes}": ["p(95)<200"],
  // The 'reports' scenario (heavy jobs) has a loose SLO.
  "http_req_duration{scenario:reports}": ["p(95)<3000"],
},

The idea is lesson 5's put into practice: each part of the system has its own SLO —anchored in its own harm to the user— and therefore its own limit. A single gate with one limit for everything would be either too strict for the heavy report, or too loose for the interactive quote. Per-scenario thresholds give each part the rule that suits it.

The executable mirror: a multi-target gate

Let's build the Python equivalent, which does run. gate_multi.py evaluates several targets, each with its own p95 limit, and supports an abort_on_fail flag per target: if a critical target breaks its limit, it cuts immediately without continuing to measure the rest.

def main():
    # Each target has ITS OWN limit: /quote is interactive (strict SLO),
    # /quote_cpu is a heavy endpoint (looser SLO). Per-scenario limits.
    targets = [
        ("/quote",     600, 20,  200, False),  # strict SLO for the fast one
        ("/quote_cpu", 600, 20,  400, True),   # loose SLO, but critical: aborts
    ]

    all_pass = True
    for path, total, concurrency, p95_limit, abort_on_fail in targets:
        latencies, error_rate = measure(path, total, concurrency)
        p95 = q(latencies, 95)
        passed = p95 < p95_limit
        status = "PASS" if passed else "FAIL"
        flag = "  [abortOnFail]" if abort_on_fail else ""
        print(f"{path:<12} p(95)={p95:7.2f}ms  limit<{p95_limit}ms  -> {status}{flag}")
        if not passed:
            all_pass = False
            if abort_on_fail:
                print(f"  ABORT: critical limit of {path} broken -> cutting the test now")
                sys.exit(1)     # the mirror of abortOnFail: cut early

    sys.exit(0 if all_pass else 1)

Each target is a tuple (path, total, concurrency, p95_limit, abort_on_fail). Notice that /quote carries a 200 ms limit (strict, it's interactive) and /quote_cpu carries 400 ms (loose, it's heavy) —per-scenario limits, each according to its SLO—. And /quote_cpu carries abort_on_fail=True: if it breaks its limit, the sys.exit(1) cuts right there, without continuing. It's the exact mirror of k6's abortOnFail.

Running it: all pass (light load)

With light load (20 concurrent on both), the two endpoints meet their respective limit. Real output:

What to expect — /quote well below its strict 200 ms limit; /quote_cpu well below its loose 400 ms limit; the gate passes with code 0:

$ python3.14 gate_multi.py http://127.0.0.1:PORT
/quote       p(95)=   7.23ms  limit<200ms  -> PASS
/quote_cpu   p(95)=  41.21ms  limit<400ms  -> PASS  [abortOnFail]
$ echo $?
0

Note that /quote_cpu has a higher p95 (41 ms) than /quote (7 ms) —it does more work—, but passes anyway, because its limit is looser on purpose. With a single 200 ms limit for both it would also have passed here; the difference shows when the heavy one degrades.

Running it: the critical one breaks and aborts

Now we raise /quote_cpu's load to 200 concurrent (heavy load). Its p95 breaks the critical 400 ms limit, and since it carries abortOnFail, the gate cuts right there. Real output:

What to expect — /quote passes; /quote_cpu breaks its critical limit and the gate aborts immediately with code 1, without continuing:

$ python3.14 gate_multi_abort.py http://127.0.0.1:PORT
/quote       p(95)=   6.78ms  limit<200ms  -> PASS
/quote_cpu   p(95)= 506.79ms  limit<400ms  -> FAIL  [abortOnFail]
  ABORT: critical limit of /quote_cpu broken -> cutting the test now
$ echo $?
1

There's the referee stopping the fight: /quote passed its strict limit (6.78 ms < 200), but /quote_cpu broke its own (506.79 ms ≥ 400), and since it was critical (abortOnFail), the gate exited with code 1 without evaluating anything else. In a truly long test, that early cut saves minutes of hammering a system that already failed. And notice lesson 5's lesson embodied here: the two endpoints were judged, each with its own limit —200 for the interactive, 400 for the heavy—, not with a single rule that would be unfair to one of the two.

Common mistakes

Aborting without delayAbortEval and cutting from an initial spike. What happens: abortOnFail: true is set with no delay, and the test aborts at second 1 because the p95 with three samples looked ugly. Why it happens: percentiles with little data are unstable (module 3), and abortOnFail evaluates them from the start. How to detect it: aborts that always happen in the first few seconds, with no real degradation. How to fix it: add delayAbortEval (e.g. "10s") to give it time to gather samples before deciding to cut.

A single limit for parts with different SLOs. What happens: the whole system is gated with p(95)<200, and the heavy-reports endpoint always fails even though its latency is perfectly acceptable for an asynchronous job. Why it happens: it's ignored that each part has its own harm to the user (lesson 5). How to detect it: an endpoint that "always fails the gate" but whose latency no one considers a real problem. How to fix it: set thresholds by tag/scenario —{scenario:reports} with a loose limit, {scenario:quotes} with a strict limit—, each anchored in its own SLO.

Tagging wrong so the by-tag threshold applies to nothing. What happens: "http_req_duration{type:api}": ["p(95)<200"] is written but no request carries tags: { type: "api" }, so the threshold is evaluated over zero samples and gives a misleading result (or doesn't apply). Why it happens: the by-tag threshold and the request's tagging are out of sync. How to detect it: a by-tag threshold that never fails even with the app broken, or that k6 reports with no data. How to fix it: make sure the tags: in http.post(...) uses exactly the same name and value as the {tag:value} of the threshold. (System tags like scenario don't need this: k6 adds them on its own.)

Exercises

Exercise 1 — Write the long format with abort. Write the k6 threshold (long format) for: the p95 below 300 ms, aborting the test if it breaks, after waiting 15 seconds to gather samples.

thresholds: {
  http_req_duration: [
    {
      threshold: "p(95)<300",
      abortOnFail: true,
      delayAbortEval: "15s",
    },
  ],
},

Exercise 2 — An SLO per scenario. You have two scenarios: checkout (the user pays and waits on screen) and nightly_export (a data dump that runs at dawn, no one waits). Write the per-scenario thresholds, choosing a defensible limit for each, and justify each choice in one sentence.

thresholds: {
  // Checkout: interactive and critical, the user waits looking -> strict SLO.
  "http_req_duration{scenario:checkout}": ["p(95)<300"],
  // Nightly export: async, no one waits -> very loose SLO.
  "http_req_duration{scenario:nightly_export}": ["p(95)<10000"],
},

Exercise 3 — Why did it abort? In the gate_multi_abort.py run, /quote showed PASS and /quote_cpu showed FAIL [abortOnFail] with exit code 1. (a) Why did the gate exit with 1 if /quote passed? (b) What would have happened if /quote_cpu did not have abort_on_fail? (c) What does the abort save in a long test?

Summary and next step

Two refinements on the basic gate. abortOnFail (long format: { threshold, abortOnFail, delayAbortEval }) makes k6 cut the test as soon as a critical limit breaks —the referee who stops the fight—, with delayAbortEval to wait for samples to gather and not abort from a misleading initial spike. The thresholds by tag/scenario ("http_req_duration{scenario:quotes}": ["p(95)<200"]) give each part of the system its own limit, anchored in its own SLO —lesson 5 applied piece by piece—. You built it as an executable in gate_multi.py: two endpoints, /quote with a strict limit (200 ms) and /quote_cpu with a loose limit (400 ms) and abortOnFail; under light load both pass (exit 0), and when /quote_cpu degrades under heavy load it breaks its critical limit and the gate actually aborts (exit 1), without continuing.

Before moving on you should be able to: write a long-format threshold with abortOnFail and delayAbortEval; write per-scenario thresholds with defensible limits for each part; and explain what aborting early saves. What comes next, in lesson 8, is the mini-project: you bring together the whole module —bring up Reservo with /quote_cpu, measure under two loads, apply evaluate_thresholds (green with the light, red with the heavy, with its exit code) and write the equivalent k6 thresholds block as content—. The performance gate, end to end, with your own hands.

Resources

• k6 — Thresholds (long format, abortOnFail, delayAbortEval) — the official reference for the long format and aborting on failure, including the note that in cloud it's evaluated every 60 s. The source of the abortOnFail content.
• k6 — Thresholds on sub-metrics with tags — the metric{tag:value} syntax for thresholds by tag and by scenario. The source of the by-part-of-the-system limits.
• k6 — Tags and groups (includes the system tag scenario) — how k6 tags requests, including system tags like scenario that enable per-scenario thresholds without tagging by hand.
• sys.exit — Python documentation — the mechanism the multi-target gate aborts with (sys.exit(1)) as soon as a critical limit breaks; the executable mirror of abortOnFail.