Module 2: Slis Slos And The Error Budget

6. Hands-on: burn rate — the consumption rate, not just the balance

Description

Lesson 4 measured how much error budget was left at the end of 30 days: 4.23 minutes, a positive but tight balance. That figure, though, hides something only visible by looking at the month day by day: there were moments when that budget got spent much faster than the final balance suggests, and moments when it was barely touched. A positive balance at month's close doesn't distinguish between "spent evenly, no surprises" and "nearly exhausted in two days, and the rest of the month compensated by pure luck of low traffic." This lesson extends the calculator with the piece that makes that distinction possible: burn rate, the rate at which a system consumes its error budget, measured against the rate the SLO allows.

Connection to the module

This lesson doesn't change lessons 4 and 5's dataset or SLO — it's still TRAFFIC_30_DAYS against 99.9% monthly. What changes is the question: instead of "how much budget is left at the end?", burn rate asks "how fast is it being spent, right now, in this recent window?" The answer, run over the same 30 days, is going to show something neither lesson 4 nor 5 revealed: exactly when, during the month, the budget was at real risk of running out — information this guide's Module 4 later turns into a real Prometheus/Alertmanager alert.

Picking the analogy back up: the mobile data plan, with numbers

Lesson 1 of this module introduced the idea: a 20 GB monthly data plan doesn't tell you if you're on track just from the cumulative balance — it tells you if you're on track from the rate you're spending it at. An error budget works the same way. The 99.9% monthly SLO allows, in total, 43.2 minutes of downtime a month — that's the "20 GB" of this analogy. But a 30-day month doesn't spend that budget at a guaranteed even pace: it can be spent uniformly (a little each day), or it can be spent almost entirely in a couple of bad days, with the rest of the month barely touching it. Burn rate is, literally, this analogy's "GB spent today" meter, applied to the error budget: how many times faster than allowed the budget is being consumed, in a specific time window.

The formula, quoted from the source

"Burn rate is how fast, relative to the SLO, the service consumes the error budget."

"With an SLO of 99.9% over a time window of 30 days, a constant 0.1% error rate uses exactly all of the error budget: a burn rate of 1."

— Google SRE Workbook — Alerting on SLOs

Translated into a concrete formula: burn rate is the observed error rate, divided by the error rate the SLO allows.

   burn_rate = observed_error_rate / allowed_error_rate
   allowed_error_rate = 1 - SLO

A burn rate of 1 means "exactly at the rate the SLO allows" — sustained all month, it would exhaust the budget right at the end, neither before nor after. A burn rate of 2 means double that speed: sustained, it would exhaust the full monthly budget in half the time, 15 days. A burn rate of 14.4 — a figure you're going to see again in a moment — means that, sustained, an entire month's budget would be exhausted in just over 50 hours.

The multi-window, multi-burn-rate pattern, and why it exists

Measuring burn rate with a single time window has a practical problem the source itself explains:

"We can enhance the multi-burn-rate alerts [...] to notify us only when we're still actively burning through the budget—thereby reducing the number of false positives. To do this, we need to add another parameter: a shorter window to check if the error budget is still being consumed as we trigger the alert."

— Google SRE Workbook — Alerting on SLOs

The idea, in one sentence: a long window on its own detects elevated consumption, but it can stay "lit up" long after the problem is already resolved — the average over several days still looks bad even though today is already healthy. A short window, evaluated alongside the long one, confirms whether the elevated consumption is still active right now. Only when both signal elevated consumption at the same time does the alert make sense. Google SRE publishes a reference table with three severity levels, each with its own window pair and its own burn rate threshold:

Severity	Long window	Short window	Burn rate	% of budget it consumes
Page (urgent)	1 hour	5 minutes	14.4x	2%
Page (urgent)	6 hours	30 minutes	6x	5%
Ticket (not urgent)	3 days	6 hours	1x	10%

— Google SRE Workbook — Alerting on SLOs, Table 5-8.

This guide has an honest limitation worth stating before moving on: this lesson's dataset has daily granularity — a single pair of numbers per day — not hourly or per-minute. The table's two "Page" rows need fine-grained telemetry (a reading every few minutes) this fixed dataset, by design, doesn't have — that granularity arrives only in Module 3, with real CloudWatch/Prometheus data, and it's exactly what Module 4's scripts/burn_rate_evaluator.py is going to implement with the literal hour and minute windows. What this lesson can reproduce exactly, because its long window is already at a day-scale, is the Ticket row: 3-day long window, 1x burn rate threshold. That row's original short window is 6 hours; with daily data, this lesson adapts that short window to 1 day — the finest interval the dataset allows — keeping the original mechanism's two-window logic completely intact.

Extending the calculator

Add these functions to scripts/error_budget_calculator.py, after print_report:

# --- New in this lesson: burn rate ---
# burn_rate = observed error rate / error rate allowed by the SLO

def window_slice(dataset, end_day, length_days):
    return [row for row in dataset if end_day - length_days < row[0] <= end_day]


def burn_rate_of(window, slo=SLO):
    if not window:
        return 0.0
    valid = sum(row[1] for row in window)
    good = sum(row[2] for row in window)
    error_rate = 1 - (good / valid)
    allowed_error_rate = 1 - slo
    return error_rate / allowed_error_rate


# Table 5-8 from the Google SRE workbook, "Ticket" row: 3-day long window,
# burn rate >= 1x, 10% of budget. The original short window is 6 hours;
# this dataset is daily, so the short window is adapted to 1 day.
TICKET_LONG_DAYS = 3
TICKET_SHORT_DAYS = 1
TICKET_THRESHOLD = 1.0


def ticket_tier_fires(dataset, end_day, slo=SLO):
    long_window = window_slice(dataset, end_day, TICKET_LONG_DAYS)
    short_window = window_slice(dataset, end_day, TICKET_SHORT_DAYS)
    long_br = burn_rate_of(long_window, slo)
    short_br = burn_rate_of(short_window, slo)
    fires = long_br >= TICKET_THRESHOLD and short_br >= TICKET_THRESHOLD
    return fires, long_br, short_br

And replace the if __name__ == "__main__": block so that, besides lesson 4's report, it prints the burn rate analysis:

if __name__ == "__main__":
    report = budget_report(TRAFFIC_30_DAYS)
    print_report(report, "process-shipment-manifest -- 30 days of traffic")

    print()
    print("--- Daily burn rate (days with errors only) ---")
    for row in TRAFFIC_30_DAYS:
        br = burn_rate_of([row])
        if br > 0:
            print(f"Day {row[0]:>2}: valid events={row[1]:>3}  good events={row[2]:>3}  burn rate={br:.2f}x")

    print()
    print(f"--- Ticket tier (long window {TICKET_LONG_DAYS}d / short window {TICKET_SHORT_DAYS}d, threshold {TICKET_THRESHOLD}x) ---")
    for end_day in range(3, 31):
        fires, long_br, short_br = ticket_tier_fires(TRAFFIC_30_DAYS, end_day)
        if fires or long_br >= 0.3:
            marca = "FIRES" if fires else "does not fire"
            print(f"Day {end_day:>2}: long window={long_br:.2f}x  short window={short_br:.2f}x  -> {marca}")

python3 scripts/error_budget_calculator.py

What to expect (literal — run it twice and it produces, digit for digit, the same result; that's how you confirm its determinism):

--- process-shipment-manifest -- 30 days of traffic ---
Valid events:            12195
Good events:             12184
Measured SLI:            99.9098%
Target SLO:              99.9%
Budget (monthly):        43.2 minutes
Consumed this period:    38.97 minutes (90.2% of budget)
Remaining budget:        4.23 minutes

--- Daily burn rate (days with errors only) ---
Day  4: valid events=438  good events=437  burn rate=2.28x
Day 17: valid events=452  good events=446  burn rate=13.27x
Day 18: valid events=438  good events=435  burn rate=6.85x
Day 23: valid events=445  good events=444  burn rate=2.25x

--- Ticket tier (long window 3d / short window 1d, threshold 1.0x) ---
Day  4: long window=0.75x  short window=2.28x  -> does not fire
Day  5: long window=0.74x  short window=0.00x  -> does not fire
Day  6: long window=0.83x  short window=0.00x  -> does not fire
Day 17: long window=4.52x  short window=13.27x  -> FIRES
Day 18: long window=6.74x  short window=6.85x  -> FIRES
Day 19: long window=6.67x  short window=0.00x  -> does not fire
Day 20: long window=2.48x  short window=0.00x  -> does not fire
Day 23: long window=0.85x  short window=2.25x  -> does not fire
Day 24: long window=0.75x  short window=0.00x  -> does not fire
Day 25: long window=0.75x  short window=0.00x  -> does not fire

Reading the result: what the final balance would never have shown

Three readings of this result deserve attention, from simplest to most revealing:

1. Day 17, on its own, nearly crosses the "Page" threshold. A burn rate of 13.27x on a single day is less than one point away from the 14.4x threshold Google uses for its most urgent alert (1-hour window). This dataset has no hourly granularity to confirm whether, within that day, there was a specific hour that did cross that threshold — exactly the kind of detail only real telemetry, from Module 3, can reveal — but the daily figure already makes clear day 17 wasn't just any bad day: it was, on Google SRE's scale, a day that came within a hair of the highest severity.

2. The Ticket tier fires exactly on days 17 and 18 — no other day of the month. Out of the dataset's 30 days, only two simultaneously meet the threshold on both the long window and the short window. This is information neither lesson 4 nor 5 gave: not just "the month was tight" (4.23 minutes of margin), but exactly when that tightness happened — directly actionable information for a real team, which would know, without guessing, which two days to check the logs on first.

3. Day 19 is this lesson's most important result. The long window (days 17-19) still shows 6.67x — at first glance, it looks like the problem is still active — but the short window (day 19 alone) shows 0.00x: zero errors that day. The two-window rule, exactly as the Google SRE quote explains, keeps the alert from staying "lit up" on day 19 just because the average of the last three days still carries the weight of days 17 and 18. The system had already recovered by day 19; a single-window alert (long only) would have kept firing that day, generating noise about a problem that was already over — exactly the "false positive" this lesson's quote names as the reason the two-window design exists.

Common mistakes

Confusing the 90.2% of budget consumed (lesson 4, the month's cumulative figure) with burn rate (this lesson, a point-in-time rate) as if they were the same number. What happens: someone uses "90.2%" and "13.27x" in the same sentence as if they measured the same thing with different notation. How to spot it: if you can't explain why a month with 90.2% of budget consumed by the end had, on one specific day, a burn rate of more than 13 times the allowed rate. How to fix it: they're two different questions about the same dataset — "how much in total?" (90.2%, accumulated over the whole month) and "how fast right now?" (13.27x, measured over a single day's window). A high burn rate for a few days can perfectly coexist with a cumulative consumption that, in the end, is still positive — that's exactly what happened in this dataset.

Assuming the Ticket tier should have also fired on day 19, because "the problem was serious" (deliberately ignoring the short window). What happens: someone, seeing day 19 had long window=6.67x, argues the alert should have stayed active that day. How to spot it: if your reasoning about when an alert should fire uses only the long window, ignoring the short one. How to fix it: that's exactly the trap the two-window design avoids — without the short window, any multi-day average stays "contaminated" by an already-resolved incident for days after it ends, generating alerts for a problem that no longer exists. Day 19's short window (0.00x) is the evidence the system had already recovered; ignoring it would reintroduce the false-positive problem this design exists to solve.

Treating this lesson's result as this guide's "real alert" (jumping ahead of Module 4). What happens: someone assumes ticket_tier_fires is already the production alert Andes Cargo is going to use. How to spot it: if you expect this code to connect to Alertmanager or fire a real notification. How to fix it: this function runs over a fixed dataset, in a script a human runs manually — it's a demonstration of the multi-window pattern's logic, not the alerting infrastructure itself. Module 4 builds scripts/burn_rate_evaluator.py, a separate artifact, that does evaluate this logic continuously over real metrics and connects it to Alertmanager and a real CloudWatch alarm — with the literal hour and minute windows this daily dataset can't reproduce.

Exercises

Exercise 1 — Calculate day 4's burn rate by hand, and verify it matches the script's result. Day 4 had 438 valid events and 437 good ones.

See solution

Day 4's error rate: (438 − 437) / 438 = 1/438 ≈ 0.2283%. The error rate allowed by the 99.9% SLO: 1 − 0.999 = 0.1%. Burn rate = 0.2283% / 0.1% ≈ 2.28x — matches the script's output exactly (burn rate=2.28x). This day, with a single bad event, already doubles the allowed consumption rate — confirmation that even a single error, on a day of moderate traffic, can push the burn rate above 1x without needing a major incident.

Exercise 2 — Explain, without running anything, why day 20 appears in the script's output (long window=2.48x) even though day 20 itself had no bad events at all. Use window_slice's definition.

See solution

window_slice(dataset, end_day=20, length_days=3) selects the days where 20 - 3 < day <= 20, that is, days 18, 19, and 20 — not just day 20 alone. Day 18 still has 3 bad events (part of the days 17-18 incident), so, even though days 19 and 20 are completely clean, the long window ending on day 20 still includes day 18's "weight" in its calculation. This is, again, the same reason the short window exists: a 3-day window keeps showing elevated activity for several days after the real problem ended, precisely because it averages over a range that still contains the incident — only the short window, evaluated alongside it, confirms the activity is no longer active.

Exercise 3 — Design, in prose (with no code), how you'd adapt Table 5-8's two "Page" rows (14.4x/1h/5min and 6x/6h/30min) if this lesson had hourly data instead of daily. What would change in window_slice and in the threshold constants?

See solution

With hourly data, window_slice would work identically — it's still a generic function that filters a range of "time units" around an endpoint — only the unit would shift from days to hours: window_slice(dataset, end_hour, length_hours). The constants would change to PAGE_FAST_LONG_HOURS = 1, PAGE_FAST_SHORT_MINUTES = 5 (which, on an hourly dataset, would need to be approximated, or would require per-minute data to be literal), PAGE_FAST_THRESHOLD = 14.4, and analogously for the 6-hour/30-minute row with a 6x threshold. ticket_tier_fires's logic — both windows must cross the threshold at the same time — would be reused unchanged for the two new rows, only changing which window function and which threshold they receive. This is, in essence, exactly the generalization this guide's Module 4 builds in scripts/burn_rate_evaluator.py, with real CloudWatch/Prometheus data that does have minute-level granularity.

Summary and next step

In this lesson you extended scripts/error_budget_calculator.py with burn rate — the observed error rate, divided by the rate the SLO allows — quoted from sre.google/workbook/alerting-on-slos/. Over the same 30-day dataset, you found day 17 reached a burn rate of 13.27x — a step away from Google's 14.4x "Page" threshold — that the two-window pattern (Ticket tier, 3 days/1 day) fired exactly on days 17 and 18, and that same pattern correctly did not fire on day 19, even though the long window was still elevated, because the short window confirmed the system had already recovered. You verified the script stays deterministic after the extension.

Before moving on you should be able to: explain the difference between error budget (cumulative balance) and burn rate (point-in-time rate) without using the word "budget" twice; calculate a single day's burn rate by hand from its valid and good events; and explain, using day 19's example, why a single-window alert generates false positives the two-window pattern avoids.

Lesson 7 runs this same calculator — with no code changes at all — over a second dataset, hand-designed to represent a genuinely bad week for process-shipment-manifest. The tool is already complete; what's left is practicing reading what it says.

Resources

Google SRE Workbook — Alerting on SLOs — the exact source for burn rate's definition, Table 5-8, and the justification for the two-window pattern.
This same repository, Module 2, lesson 1 (01-module-introduction-2.md) — the mobile data plan analogy this lesson develops with real numbers.
This same repository, Module 2, lesson 4 (04-hands-on-the-error-budget-calculator.md) — the dataset and base functions this lesson extends without modifying.

← Previous5. Choosing an SLO that means something Next →7. Hands-on: applying the calculator to a real Andes Cargo scenario