Module 1: What Is Sre And Reliability As A Feature

8. Project: Andes Cargo's reliability charter

Next →

Description

Seven lessons gave you the vocabulary (SRE, error budget, SLI/SLO/SLA, toil, on-call, postmortem), a real risk inventory, and this guide's first executed calculation — 33.3x the monthly budget, in the Claude Code incident. This module's final project gathers all of that into this guide's first formal document: an ADR (Architecture Decision Record), the same format THREAT-MODEL.md/RISK-MAP.md already used in cloud-security-and-guardrails-guide and COST-PROFILE.md/COST-MAP.md already used in finops-and-cost-guardrails-guide. The result is RELIABILITY-CHARTER.md: what "reliable" means for this specific system, and the complete map of the 8 modules this guide is going to move through to build it.

Connection to the module

This is Module 1's close, and the document that literally governs the sequence of lessons you're going to move through from M2 to M8 — the same role COST-MAP.md plays in finops-and-cost-guardrails-guide. Unlike that document, whose seven rows were independent drivers that could be resolved in any order, every row of RELIABILITY-CHARTER.md depends directly on the one before it: Module 2's SLO is what Module 3 measures, what Module 4 alerts on, what Module 6 measures the real incident against. You're going to come back to this document at every module's project, to mark a row as resolved.

Step 1 — Why this document doesn't pick an SLO yet

The strongest temptation when writing this document is to put in a real number — "Andes Cargo's SLO is 99.9%" — because it feels like the right way to close a module about reliability. This document resists that temptation on purpose: picking an SLO without the four golden signals (Module 2, lesson 2) or a real dataset (Module 2, lesson 4) would be exactly the most common mistake you already named in lesson 3 of this module — "picking the highest possible SLO because it sounds better" — just this time dressed up as "picking a reasonable SLO because it sounds responsible." RELIABILITY-CHARTER.md sets the philosophy — how this system's reliability is going to be decided — and leaves the real number, with its evidence, for Module 2.

Step 2 — The complete document

At the root of andes-cargo-infra/, create RELIABILITY-CHARTER.md:

# RELIABILITY-CHARTER.md — What "Reliable" Means for Andes Cargo

**Status:** Accepted · **Date:** this module's close · **Supersedes:** none
**Governs:** Modules 1 through 8 of `sre-and-incident-response-guide`
**Source:** Module 1, lessons 2-7 (SRE vocabulary, the error-budget math on the Claude Code incident,
the SRE-lens inventory of `andes-cargo-infra/`)

## Context

Six guides built and hardened `andes-cargo-infra/`: four core services (`aws-core-services-guide`),
a production-grade Lambda with concurrency limits and retries (`aws-serverless-and-containers-guide`),
the whole stack declared in Terraform (`terraform-and-iac-guide`), a CI/CD pipeline running two green
gates (`cicd-and-gitops-on-aws-guide`, `cloud-security-and-guardrails-guide`, `finops-and-cost-
guardrails-guide`). None of the three questions those gates answer — is this secure, does this deploy
cleanly, does this fit the budget — measures whether the system stays up in production. Module 1 of
this guide established, with a real incident and real arithmetic, why that gap matters: a hypothetical
99.9% monthly SLO gives `process-shipment-manifest` 43.2 minutes of allowed downtime a month: a single
24-hour recovery, the kind DataTalks.Club lived through in February 2026, burns 33.3x that budget in
one event. No SLA exists for Andes Cargo (an internal tracking system, no paying external customer),
so the number that matters here is an SLO, not an SLA, and it does not exist yet.

## Decision

**"Reliable" means measured, not promised — the same distinction Module 1, lesson 2 drew from Ben
Treynor's definition of SRE.** For Andes Cargo specifically, this charter commits to five concrete
positions, ahead of Module 2's real numbers:

1. **No SLO is chosen in this document.** Picking a number without the four golden signals (Module 2,
   lesson 2) or a real dataset (Module 2, lesson 4) would be exactly the "100% because it sounds
   responsible" mistake Module 1, lesson 3 named as this guide's most common error. The SLO is decided
   with evidence, in Module 2, not guessed here to make this charter feel more complete.
2. **`process-shipment-manifest` is not a payments system or a life-support system.** It is an
   asynchronous manifest processor for a cargo-tracking product. This charter explicitly rejects
   chasing 99.99%+ reliability for it — the cost of that pursuit (Module 1, lesson 3: each additional
   nine can cost 100x more than the last) is not justified by what a few extra minutes of delayed
   processing would cost the business.
3. **Single-region (`us-east-1`) is accepted, with the risk named, not hidden.** Module 1, lesson 4
   found no cross-region failover anywhere in `andes-cargo-infra/`; lesson 5 named the `us-east-1`
   outage of October 2025 as the concrete shape that risk takes. This guide's $0, single-region scope
   (declared in its own design) means multi-region DR is out of scope — this charter records that as a
   known, accepted gap, not an oversight discovered later.
4. **An error budget is a measurement and negotiation tool, never a preventive control.** Module 1,
   lesson 5 drew this line explicitly against the Claude Code incident: `conftest` and `prevent_destroy`
   (built in `cloud-security-and-guardrails-guide`) are this system's preventive layer. Nothing in this
   guide replaces them; this guide's tools quantify damage and inform decisions after the fact.
5. **Every incident this guide operates gets a blameless postmortem.** Following Module 1, lesson 7's
   citation of Google SRE's postmortem culture: identifying contributing causes with full precision,
   never indicting an individual or team for acting in good faith with the information they had.

## The map

| # | Module | Central question | Deliverable | Status |
|--:|---|---|---|---|
| 1 | What Is SRE and Reliability as a Feature | What does "reliable" mean, measurably, for this system? | `RELIABILITY-CHARTER.md` (this document) | In progress |
| 2 | SLIs, SLOs, and the Error Budget | What SLI and SLO make sense for `process-shipment-manifest`, with evidence? | `SLO.md`, `scripts/error_budget_calculator.py` | Open |
| 3 | Observability as SLI Input | Where does the SLI's real data come from, and how is it read? | `observability/docker-compose.yml`, `observability/instrument_manifest_flow.py` | Open |
| 4 | Alerting on Error Budget Burn Rate | When does an alert fire, and why that threshold and not a static one? | `scripts/burn_rate_evaluator.py`, `observability.tf` | Open |
| 5 | The Incident Lifecycle | Who does what, at what severity, on what on-call schedule? | `INCIDENT-RESPONSE-PLAN.md`, `oncall/schedule.py` | Open |
| 6 | Operating the Claude Code Incident | How does a real incident run through the machine this guide built? | `incidents/2026-02-26-claude-code-destroy/TIMELINE.md` | Open |
| 7 | Blameless Postmortems and Runbooks | What did the system teach us, and what do we do differently next time? | `.../POSTMORTEM.md`, `runbooks/manifest-processor-error-rate.md` | Open |
| 8 | Capstone: The Andes Cargo Reliability Package | Does the whole machine work end-to-end, on an incident nobody saw coming? | Full repository, portfolio-ready | Open |

## Consequences

Every row's `Status` column moves from `Open` to `Resolved` — noting the closing lesson — as each
module completes. Unlike `COST-MAP.md` in `finops-and-cost-guardrails-guide`, this charter's rows are
not independent drivers that could theoretically resolve out of order: each module's deliverable is a
direct input to the next (the SLO of M2 is what M3 measures, what M4 alerts on, what M6 measures the
incident against), so this map is read top to bottom, not reordered.

## Alternatives considered

**Pick a placeholder SLO now (e.g., 99.9%) so this document feels complete on day one.** Rejected:
Module 1, lesson 3 already showed why an SLO chosen without evidence is this guide's most common
mistake. A placeholder number, even labeled as such, tends to calcify into the real one by the time
Module 2 arrives — better an honest `Open` in this table than a number nobody actually chose with
data.

**Commit to a multi-region reliability target, matching what a real production system would need.**
Rejected: explicitly out of this guide's declared $0, single-region scope. Naming the gap honestly
(Decision, point 3) serves a learner better than a target this guide cannot build toward.

**Treat the security gate and cost gate as sufficient evidence of reliability, and skip this charter.**
Rejected: this is the exact premise Module 1 spent seven lessons dismantling — green and secure and
affordable measured three different questions, none of them "does this stay up, and for how long can
it not."

Step 3 — Verifying the document

wc -l RELIABILITY-CHARTER.md
grep -c '^| [1-8] ' RELIABILITY-CHARTER.md

What to expect (literal — you wrote the content, the shape is deterministic):

86
8

Eighty-six lines, eight numbered rows in the table — one per module, none missing, none duplicated. Row 1 is the only one with Status: In progress, exactly as it should be: this document is, itself, the deliverable of the row it describes.

How to read this document, six months from now

Just like its counterpart in finops-and-cost-guardrails-guide, this ADR's real test isn't its shape on the day it's written — it's whether it's still useful when nobody remembers the full context from memory. Facing this document, a new reader should be able to answer, without asking anyone: what does "reliable" mean here? (the Decision section, five concrete positions, none an invented number); what's left to build, and in what order? (the table, read top to bottom, never reordered); why isn't there an SLO yet? (Alternatives considered, first entry); and what risk was accepted on purpose, instead of being ignored by accident? (Decision, point 3 — a single region, with the reason named). If any of those four questions requires rereading an entire lesson, the document didn't do its job.

Module 1's close

With RELIABILITY-CHARTER.md written, this module delivers exactly what lesson 1 promised: not a calculator built — that arrives in Module 2 — but the complete vocabulary, a real risk inventory, this guide's first real number (33.3x), and the document that honestly governs what's missing and in what order. You enter Module 2 with a reliability philosophy already decided and a traceable map, instead of a general feeling of "we should be more reliable."

Common mistakes

Marking row 1 of the table as Resolved when finishing this lesson (an expectation mismatch about the document itself). What happens: someone, after finishing writing RELIABILITY-CHARTER.md, marks its own row as resolved. How to spot it: if row 1's Status says Resolved instead of In progress. How to fix it: this document never fully finishes "resolving" itself — it stays in force, governing the rest of the guide, until Module 8 closes the capstone. In progress is the honest state while the charter itself remains the active reference for every remaining module.

Adding a numeric SLO to this document "so it isn't left incomplete" (the same mistake from lesson 3, reappearing here). What happens: someone, uncomfortable that a reliability document has no SLO number at all, adds one anyway, citing this very lesson as if it already allowed it. How to spot it: if your version of RELIABILITY-CHARTER.md has an availability percentage anywhere. How to fix it: this document's Alternatives considered section explains, with the same logic as lesson 3 of this module, why an SLO with no evidence is worse than no SLO at all — reread that section before adding any number. Module 2 is, literally, the place designed for that decision.

Treating the absence of an SLA as an oversight, instead of a deliberate decision (incomplete reading). What happens: someone notices this document mentions no SLA at all and assumes it's a missing piece that will be added later. How to spot it: if you look, in later modules, for where Andes Cargo's SLA gets defined. How to fix it: this document's Context is explicit — Andes Cargo is an internal system, with no external customer paying for an availability contract with financial consequences. An SLA isn't a piece this guide has pending; it's, correctly, a piece this specific system doesn't need, exactly the distinction lesson 7 of this module established between SLO and SLA.

Exercises

Exercise 1 — Update RELIABILITY-CHARTER.md by hand, predicting Module 2's close. Without having done Module 2 of this guide yet, write how row 2 of the table would look once that module finishes, including the Status column and a brief note on what evidence would back it up.

See solution

A reasonable update: Status: Resolved (M2.8), with a note like "SLI defined as successful invocations over valid invocations for process-shipment-manifest; SLO chosen and justified against the four golden signals (M2.2) and a real 30-day sample dataset (M2.4); SLO.md committed with the number and its reasoning; scripts/error_budget_calculator.py running against that dataset, producing a literal error-budget figure in minutes." The exercise's point isn't guessing the SLO's exact number — that depends on the real criteria applied in Module 2 — but practicing that every Status update comes with what concrete evidence backs it up, never just the word "Resolved."

Exercise 2 — Defend Decision position 3 against a real objection. A technical interviewer, reviewing this document, asks: "why accept the single-region risk instead of at least documenting a migration plan to multi-region?" How would you respond, using the document itself as reference?

See solution

A complete answer: "The document doesn't ignore the risk — it names it explicitly, with the us-east-1 outage of October 2025 as the concrete case that risk represents. The decision not to build even a migration plan isn't negligence: it's honesty about this guide's scope, declared since its design, of $0 and a single region. A multi-region migration plan that's never going to run, written only so this document looks more complete, would be worse than the current honesty — it would promise something this guide has no intention of building. The right alternative, if Andes Cargo ever really needed to solve this risk, is a different guide or project, with its own engineering budget, not an extra section in this document." Recognizing the risk with precision, without faking a mitigation that doesn't exist, is exactly the same discipline this entire guide has already practiced with every representative case.

Exercise 3 — Explain why this map's rows can't be resolved out of order, unlike COST-MAP.md. A colleague, familiar with finops-and-cost-guardrails-guide, asks why you couldn't, for example, build the runbook first (Module 7) and define the SLO afterward (Module 2), since in COST-MAP.md the drivers' order was a pedagogical sequencing decision, not a hard technical dependency.

See solution

Because, unlike COST-MAP.md's seven cost drivers — which were, in practice, fairly independent of each other — every deliverable in this guide is a direct input to the next: a runbook (Module 7) describes what to do when a burn rate alert fires, but that alert (Module 4) can't exist without real observability data (Module 3), which in turn has no SLI/SLO to measure itself against without Module 2. Trying to write Module 7's runbook before a real SLO exists would mean writing instructions for responding to an alert that, technically, can't yet fire on anything measurable. This document's Consequences section is explicit about this difference: this map's rows are read top to bottom, they're not reordered.

Summary and next step

In this module's final project you wrote RELIABILITY-CHARTER.md, this guide's first portfolio document: five concrete positions on what "reliable" means for Andes Cargo (measured, not promised; not chasing unneeded nines; a single region with the risk named; error budget as measurement, not a preventive control; every incident with a blameless postmortem), and the complete map of the 8 modules, with their central question and their deliverable, read in strict dependency order. You verified the document with the same deterministic discipline as this whole ecosystem: 86 lines, 8 rows, none resolved yet, honestly marked.

Before closing this module you should be able to: recite the Decision section's five positions without looking at the document; explain why no row in this map can be resolved out of order; and defend, against a real objection, why this document doesn't pick an SLO yet.

With this, Module 1 of sre-and-incident-response-guide is complete: SRE vocabulary learned and cited from the official source, a real risk inventory over andes-cargo-infra/, this guide's first executed number (33.3x the monthly budget), and a portfolio document defensible against any technical question. Module 2 opens this map's first row: the real error budget calculator, running for the first time against a committed dataset.

Resources

  1. AWS Prescriptive Guidance — Architecture decision records — AWS's official reference for the ADR format RELIABILITY-CHARTER.md follows.
  2. This module, lessons 2 through 7 — the direct source for every position in this document's Decision section.
  3. finops-and-cost-guardrails-guide, Module 1, lesson 8 (08-project-andes-cargos-cost-profile-and-roadmap.md) — the sibling lesson whose ADR format this lesson adapts to the reliability domain.
  4. cloud-security-and-guardrails-guide, Module 1, lesson 8 — the same risk-map/roadmap pattern, applied for the first time in this ecosystem.
← Previous7. The vocabulary you'll use all guide longNext →1. Introduction: from vocabulary to math