Project: Ship the Production-Ready Agent

Description

This is the entire guide's last lesson: eight modules, sixty-four lessons, and a single agent that kept earning, layer by layer, the operation it needs to survive real traffic. There's no new engineering to learn — there are four artifacts to confirm, generated end to end in a single run, and a final challenge combining the four disciplines in a way no previous lesson showed exactly like this.

This lesson's deliverable is a run checklist — every discipline confirmed with real code, not from memory — demonstrating this guide's complete operations layer keeps DISEÑO.md's promise: observe (M2), measure (M3/M4), gate (M5), and harden + version (M6/M7), all wrapping run_reservo_agent without touching a single line of its logic. The final challenge — with its complete solution in a collapsible — asks you to operate a new incident: a candidate version ready to promote, on the exact same day book_room fails again, with a shorter outage than Lesson 6's.

Connection to the module

This lesson builds nothing new — it verifies what this module's Lessons 1-7 already built, and puts it to the test one last time against a scenario combining pieces from M2, M3/M4, M6, and M7 in an order no previous lesson showed together.

Analogy: closing the books, after a complete year of service

This module's introduction's restaurant survived a complete year of real traffic, with all five operations disciplines running at once. Before closing that year's books, someone does the complete closing: confirms every log matches every invoice, that the menu's version file is up to date, that the backup generator is still calibrated. It isn't that anyone doubts the year went well — that's already been seen, layer by layer, that it did. It's a serious operation's final discipline: closing with evidence, not with an impression. This lesson is that closing: every discipline this capstone promised, confirmed one last time with code that runs and produces four verifiable artifacts.

"Done" checklist, run for real

An operations capstone is "done" when every item on this list can be confirmed with code, not with a code reading. Run this complete script, end to end, in your working directory:

import json
from dataclasses import asdict

import run_logger as rl
import cost_calculator as cc
import latency_model as lm
import harness as hn
from tool_circuit_breaker import CircuitBreaker, CircuitOpenError, call_with_breaker
from prompt_registry import PROMPT_REGISTRY
from rollout import rollout_decision, rollback

checklist = {}

# 1. OBSERVAR (M2): RUN_LOG.jsonl existe, con trace_id deterministas.
events = [json.loads(line) for line in open("RUN_LOG.jsonl", encoding="utf-8")]
checklist["observar"] = len(events) > 0 and all("trace_id" in e for e in events)

# 2. MEDIR (M3/M4): el pricing fijo y TOOL_LATENCY_MS siguen calibrados.
checklist["medir"] = (
    cc.estimate_cost_cents(64, 56) == 0
    and lm.TOOL_LATENCY_MS == {"list_rooms": 40, "get_quote": 25, "book_room": 120, "cancel_booking": 90}
)

# 3. GATEAR (M5): el gate corre contra el CASE_SET y produce un veredicto.
gate_report = hn.run_regression_gate(hn.CASE_SET)
checklist["gatear"] = gate_report.passed

# 4. ENDURECER (M6): el circuit breaker abre en el umbral configurado.
probe_breaker = CircuitBreaker("book_room", failure_threshold=3, cooldown_calls=2)
for _ in range(3):
    probe_breaker.on_failure()
checklist["endurecer"] = probe_breaker.state == "OPEN"

# 5. VERSIONAR (M7): el registro tiene v1/v2, y el rollback vuelve a v1.
checklist["versionar"] = (
    set(PROMPT_REGISTRY.keys()) >= {"v1", "v2"}
    and rollback("v2", "v1", PROMPT_REGISTRY) == "v1"
)

for disciplina, ok in checklist.items():
    print(f"{disciplina:12} {'✅ OK' if ok else '❌ FALTA'}")

print()
print("capa de operación completa:", "✅ TODAS LAS DISCIPLINAS CONFIRMADAS" if all(checklist.values()) else "❌ REVISAR")

What to expect:

observar     ✅ OK
medir        ✅ OK
gatear       ✅ OK
endurecer    ✅ OK
versionar    ✅ OK

capa de operación completa: ✅ TODAS LAS DISCIPLINAS CONFIRMADAS

Every line of this checklist calls a function you already ran, with real output, at some point in this module or in M2-M7. That's, precisely, what makes this capstone trustworthy: it isn't the first time anything it confirms has run.

The four artifacts, generated end to end

1. RUN_LOG.jsonl (M2, Lesson 3) — already written to disk

print("RUN_LOG.jsonl:", len(events), "eventos,", len({e['trace_id'] for e in events}), "trace_ids distintos")

RUN_LOG.jsonl: 28 eventos, 4 trace_ids distintos

2. The metrics summary (M3/M4, Lesson 4)

from metrics_summary import build_run_metrics, aggregate_metrics

# Reconstruido a partir de los mismos tres runs completados de la Lección 3.
metrics = [
    build_run_metrics("run-8487582448eb", "Reserva Focus pro 3h para Ana", history_ana),
    build_run_metrics("run-c720132bf969", "Reserva Boardroom pro 1h para Sofia", history_sofia),
    build_run_metrics("run-8d26276b0d45", "Cancela la reserva 999", history_cancel),
]
batch = aggregate_metrics(metrics)
print(f"costo total: {batch.total_cost_cents}c | p50={batch.p50_latency_ms}ms | p95={batch.p95_latency_ms}ms")

costo total: 0c | p50=185ms | p95=185ms

3. regression_report.json (M5, Lesson 5) — already written to disk

raw = open("regression_report.json", encoding="utf-8").read()
print("regression_report.json:", len(raw), "bytes, passed =", json.loads(raw)["passed"])

regression_report.json: 1700 bytes, passed = True

4. AGENT_CHANGELOG.md — the fourth artifact, written in this lesson

With the three previous disciplines already confirmed, this last artifact documents, in human-readable text, the complete versioning history Lesson 5 ran: v2 proposed, NO-GO, rollback to v1.

changelog = f"""# AGENT_CHANGELOG.md -- agente de Reservo

## v1 -- ACTIVA (hash {PROMPT_REGISTRY['v1'].prompt_hash})
Versión original, segura. Pasa el gate de regresión completo: 5/5 (M5, el
agente completo) y 5/5 (M7, mismo CASE_SET, sin overrides).

## v2 -- RECHAZADA, NO-GO (hash {PROMPT_REGISTRY['v2'].prompt_hash})
Propuesta: más proactiva, completa una reserva directamente cuando ya tiene
toda la información, sin pasar por get_quote primero.

Resultado del gate (Módulo 7, Lección 5 de este capstone, mismo CASE_SET de
M5 con overrides): 4/5 -- FAIL en quote_focus_pro_3h ("Cuanto cuesta Focus
pro 3h?"). v2 reservó en vez de cotizar, creando una reserva real para Ana
sin que lo pidiera de forma explícita.

Decisión: NO-GO (rollout_decision, regla dura -- nunca romper un caso que
v1 ya pasaba). Rollback ejecutado: version activa vuelve a v1.

## Incidente de resiliencia -- book_room (Módulo 6, Lección 6 de este capstone)
book_room cayó de forma sostenida durante un lote de 7 usuarios (9 llamadas
reales fallidas antes de recuperarse). El CircuitBreaker abrió tras 3 fallos
consecutivos, rechazó 2 llamadas sin tocar la tool, y cerró de nuevo tras
una sonda exitosa en HALF_OPEN. 11 llamadas reales contra 21 sin breaker.
"""

with open("AGENT_CHANGELOG.md", "w", encoding="utf-8") as fh:
    fh.write(changelog)

print("AGENT_CHANGELOG.md escrito:", len(changelog), "caracteres")

AGENT_CHANGELOG.md escrito: 1017 caracteres

The four artifacts — RUN_LOG.jsonl, the metrics summary, regression_report.json, AGENT_CHANGELOG.md — are this capstone's complete deliverable: plain, parseable or readable text, surviving the process that generated it. Anyone can open them, without running a single line of Python again, and rebuild exactly what happened, what it cost, what got gated, and what got decided.

The final challenge: a new incident, with a twist no previous lesson showed

The checklist confirms the operations layer works over the scenarios you already saw. This lesson's challenge asks you to operate a new incident, with your own hands, before looking at the solution: the team has a v3 prompt version ready — the one that fixed v2's regression in Lesson 5's Exercise 3 — and wants to promote it exactly the same day book_room fails again. This time the outage is shorter: only 4 real calls down, not 9. Before promoting v3, confirm with the gate that the promotion is safe, and confirm with the circuit breaker whether this shorter outage even manages to open the breaker.

This challenge combines, in an order no previous lesson in this module showed together: M7's GO/NO-GO decision (Lesson 5), M6's CircuitBreaker state machine (Lesson 6) with a new parameter (OUTAGE_CALLS=4 instead of 9), and updating AGENT_CHANGELOG.md with both findings.

Before looking at the solution: calculate by hand how many real calls it would take a failure_threshold=3 breaker to reach opening, knowing every user triggers up to 3 internal attempts via retry_with_backoff before giving up.

from prompt_registry import hash_prompt, AgentVersion

# La misma v3 que la Lección 5 (Ejercicio 3) ya registró -- el hash real de
# M7 (Lección 8, mini-proyecto), reusado sin recalcular el texto a mano.
SYSTEM_PROMPT_V3 = (
    "Eres el asistente de reservas de Reservo, un sistema de coworking. "
    "Ayudas a los usuarios a consultar salas, cotizar precios, reservar y "
    "cancelar reservas. Usa siempre las tools disponibles para cotizar y "
    "reservar -- nunca inventes un precio de memoria. Cuando el usuario "
    "pregunta un precio, usa get_quote y NO reserves, incluso si podrias "
    "inferir todos los datos necesarios para reservar. Usa book_room "
    "unicamente cuando el usuario pide reservar de forma explicita."
)
v3_hash = hash_prompt(SYSTEM_PROMPT_V3)
PROMPT_REGISTRY["v3"] = AgentVersion(
    version_id="v3", prompt_text=SYSTEM_PROMPT_V3, prompt_hash=v3_hash,
    tools_version="tools-v1", model="claude-sonnet-5",
    note="Corrige la regresion de v2 en quote_focus_pro_3h.",
)

VERSION_OVERRIDES["v3"] = {}  # v3 vuelve a decidir get_quote en quote_focus_pro_3h, igual que v1
report_v1 = hn.run_regression_gate(hn.CASE_SET, overrides=VERSION_OVERRIDES["v1"])
report_v3 = hn.run_regression_gate(hn.CASE_SET, overrides=VERSION_OVERRIDES["v3"])
decision_v3, broken_v3 = rollout_decision(report_v1, report_v3)
print(f"v3: hash={v3_hash}")
print("GATE v3:", "PASS" if report_v3.passed else "FAIL",
      f"({sum(c.passed for c in report_v3.cases)}/{len(report_v3.cases)})")
print(f"rollout_decision(v1, v3) -> {decision_v3}, casos_rotos={broken_v3}")

v3: hash=c5c4c4631f7e
GATE v3: PASS (5/5)
rollout_decision(v1, v3) -> GO, casos_rotos=[]

_state_short = {"count": 0}
OUTAGE_CALLS_SHORT = 4

def flaky_book_room_short(room, tier, hours, member):
    _state_short["count"] += 1
    if _state_short["count"] <= OUTAGE_CALLS_SHORT:
        raise ConnectionError(f"timeout de red simulado (llamada real #{_state_short['count']})")
    return {"booking_id": 1, "confirmed": True, "price_cents": 6000}

breaker_short = CircuitBreaker("book_room", failure_threshold=3, cooldown_calls=2)
for run_n in range(1, 5):
    before = breaker_short.state
    try:
        result = call_with_breaker(breaker_short, flaky_book_room_short, room="Focus", tier="pro",
                                    hours=3, member=f"user{run_n}", max_retries=3, base_delay_ms=100)
        print(f"run {run_n}: antes={before}  OK -> {result}  failure_count={breaker_short.failure_count}")
    except CircuitOpenError as exc:
        print(f"run {run_n}: antes={before}  RECHAZADO -> {exc}")
    except ConnectionError as exc:
        print(f"run {run_n}: antes={before}  FALLO -> {exc}  failure_count={breaker_short.failure_count}")

print()
print("llamadas reales totales:", _state_short["count"], "| estado final del breaker:", breaker_short.state)

run 1: antes=CLOSED  FALLO -> timeout de red simulado (llamada real #3)  failure_count=1
run 2: antes=CLOSED  OK -> {'booking_id': 1, 'confirmed': True, 'price_cents': 6000}  failure_count=0
run 3: antes=CLOSED  OK -> {'booking_id': 1, 'confirmed': True, 'price_cents': 6000}  failure_count=0
run 4: antes=CLOSED  OK -> {'booking_id': 1, 'confirmed': True, 'price_cents': 6000}  failure_count=0

llamadas reales totales: 7 | estado final del breaker: CLOSED

changelog_v2 = changelog + f"""
## v3 -- PROMOVIDA (hash {v3_hash})
Corrige la regresión de v2: la instrucción de proactividad queda acotada
exactamente al caso que la justificaba (completar una reserva ya pedida
explícitamente), sin generalizarla a "cualquier pregunta con suficiente
información". Gate: 5/5 PASS. rollout_decision(v1, v3) -> GO.

## Incidente de resiliencia #2 -- book_room, apagón corto
OUTAGE_CALLS=4 (mas corto que el incidente anterior). El breaker NUNCA
abrió: el apagón se resolvió dentro del presupuesto de reintentos de
retry_with_backoff de los primeros dos usuarios (7 llamadas reales, 0
rechazos). Confirma que el backoff acotado (Modulo 6, Leccion 3) ya cubre
apagones cortos -- el breaker existe para los que duran mas que eso.
"""
with open("AGENT_CHANGELOG.md", "w", encoding="utf-8") as fh:
    fh.write(changelog_v2)
print("AGENT_CHANGELOG.md actualizado:", len(changelog_v2), "caracteres")

AGENT_CHANGELOG.md actualizado: 1667 caracteres

The complete journey, in a table

Before closing, it's worth seeing this guide's eight modules side by side, with the exact piece each one contributed to the operations layer you just verified:

Eight modules, sixty-four lessons, and no mechanism repeated twice: every piece got built once, ran with real output, and this capstone reused it exactly as it stood.

Common mistakes

1. Thinking this checklist "certifies" the agent for real production. It certifies, precisely, the four disciplines this guide promised: observability, measurement, a form gate, and resilience + versioning over the tool-calls layer. It doesn't certify semantic quality, infrastructure, security against manipulation, or optimized cost — this module's Lesson 7 traced those boundaries precisely.

2. Confusing the final challenge's GO with a guarantee v3 is never going to fail in real production. GO means, precisely, "it didn't break any CASE_SET case v1 already passed" — a necessary condition, never a sufficient one, for a real deployment with zero risk.

3. Generalizing the challenge's Step 2 finding ("the breaker never opened") to "the breaker is never useful." The opposite — Lesson 6's example (OUTAGE_CALLS=9) showed exactly the case where it does open and does save real calls. This lesson's challenge shows the other extreme, just as real: a short outage bounded backoff already resolves on its own. Both lessons, together, are the complete picture.

4. Writing AGENT_CHANGELOG.md once and never updating it again. This lesson's file gets overwritten twice — once after Lesson 5's rollback, again after the final challenge — because a changelog not reflecting the most recent state is just as untrustworthy as having none at all.

5. Thinking "I finished the guide" means never having to go back to M2-M7 again. This capstone reused every piece without rebuilding it — but a real system, with changing traffic, is going to need to adjust thresholds (failure_threshold, cost_threshold_cents), add cases to the CASE_SET, or review pricing when it changes. M2-M7's eight lessons remain the reference to go back to, not a closed chapter.

Summary and next step

• We confirmed, with a run checklist, the operations layer's five pieces: observe (M2), measure (M3/M4), gate (M5), harden (M6), and version (M7) — each one with code that runs, not a reading from memory.
• We generated the capstone's four final artifacts: RUN_LOG.jsonl (28 events), the metrics summary (total cost 0c, latency p50/p95), regression_report.json (PASS), and AGENT_CHANGELOG.md, documenting this agent's complete versioning and resilience history.
• We solved a challenge combining M6 and M7 in a new way: v3 promoted with GO, and a short book_room outage that — unlike Module 6 — never manages to open the CircuitBreaker, because it resolves within retry_with_backoff's retry budget — proof this guide's two resilience layers each cover a different range of failure duration.
• We closed this module's eight-lesson map, and with it, the entire guide: agents-in-production-guide, eight modules, sixty-four lessons, a single agent operated end to end.

Next step, outside this guide: when the operated Reservo agent needs infrastructure and incident response at scale → sre-and-incident-response-guide. When it needs semantic judgment on its responses' quality → evaluation-frameworks-guide. When the cost M3 measures genuinely needs to come down → cost-optimization-caching-guide. When it needs generic resilience in depth, beyond the tool-calls layer → resilience-and-reliability-patterns-guide. When it's going to serve real users → agent-security-and-sandboxing-guide, without exception.

Additional resources

1. Anthropic — Tool use (function calling) overview — The complete protocol this guide instrumented, measured, gated, and hardened, end to end.
2. Anthropic — Building effective agents — The discipline of operating, with judgment, a system that already works — this entire guide's premise.
3. Python 3.14 — What's New — The exact version all of this guide's operations engineering ran on, with no external dependency at all.
4. Python — dataclasses, logging, hashlib, statistics, enum — The complete standard library every artifact in this guide got built on: RunEvent/CostReport/LatencyReport (dataclasses), traced_run (logging), trace_id/prompt_hash (hashlib), percentile (statistics), CircuitBreaker (states as strings, with no mandatory need for enum.Enum).