Module 8: Project The Reservo Agent In Production

Module Introduction: the Capstone — the Reservo Agent in Production

Next →

Description

Seven modules left you with four disciplines built, tested, and run separately: observing a complete run with structured logging and a trace_id (Module 2); measuring how much it cost and how long it took (Modules 3 and 4); gating — deciding, with a deterministic criterion, whether the agent still behaves as expected (Module 5); and hardening + versioning — surviving failures that persist across runs and deciding with evidence whether a new version is safe (Modules 6 and 7). Every module built its piece on top of the Reservo agent agent-fundamentals-and-tool-calling-guide M8 already delivered, without touching a single line of its logic. This last module — the capstone — doesn't add any new discipline. It does the one thing missing: wrap the complete agent with all four at once, put it to work on real traffic with that layer on, and deliver it ready to operate.

By the end of this module you're going to have, in its own directory, the complete operations layer — logger, cost calculator, latency model, regression harness, circuit breaker, version registry — wrapping run_reservo_agent without modifying it, and four real artifacts, written to disk: RUN_LOG.jsonl, a cost-and-latency metrics summary, regression_report.json, and AGENT_CHANGELOG.md.

Connection to the module

This module has, with the rest of this guide, the same relationship agent-fundamentals's capstone had with its seven modules: it doesn't add a new layer, it's the view from above of the layers that already exist, working together. Every lesson that follows picks back up, literally, code you already ran in an earlier module of this guide — traced_run, cost_for_run, total_run_latency_ms, run_regression_gate, CircuitBreaker, PROMPT_REGISTRY — and puts it to work on the same agent, at the same time. The only genuinely new thing is the integration: as you're going to see in Lesson 5, two pieces built separately — M5's regression harness and M7's version registry — solve similar-looking questions with slightly different fixtures, and knowing which to use for which question is, itself, part of operating a real system.

Analogy: the restaurant, a year after opening night

agent-fundamentals M8 closed with Reservo's restaurant opening night: the complete kitchen, the complete protocol, the complete service, running for the first time, serving a real customer start to finish. That night went well. But a restaurant that survives a full year of real service isn't the same one that opened that night — not because the kitchen changed, but because a whole operations layer showed up around the kitchen that opening night never needed.

Now there's a health inspector who reviews, dish by dish, who entered the kitchen and what they did — not because the chef is a suspect, but because when something goes wrong at three in the morning, someone needs to reconstruct exactly what happened, without depending on anyone's memory (Module 2: structured logging and trace_id). There's an accountant who bills every dish with its real ingredient cost, and a stopwatch in the kitchen measuring how long every station takes — not to rush the chef, but to know, with numbers, where the money and time go (Modules 3 and 4: cost and latency). There's a mandatory inspection before any menu change reaches customers — a fixed exam, with the same dishes as always, confirming nothing that already worked broke (Module 5: the regression gate). There's a backup generator that turns itself on, with nobody having to go check, when a kitchen station stops responding — and a chef-substitution protocol, with dated files for every recipe, for when someone proposes changing the menu and it needs deciding, with evidence, whether the change is safe or whether to go back to the usual chef (Modules 6 and 7: resilience and versioning).

None of those five things change what the kitchen knows how to cook. The menu is the same, the recipes are the same, the dishes that come out are the same. What changed is everything around the kitchen that lets it survive, with no surprises, a full year of real customers. This module is that complete layer, put around the same agent, all working at once.

Where we are in the ecosystem

Agents in production — operating the Reservo agent
├── Module 1: Why Operating Is Different From Building
├── Module 2: Structured Logging and Tracing a Run          -> OBSERVE
├── Module 3: Measuring Cost and Tokens per Run               -> MEASURE
├── Module 4: Measuring Latency Honestly                      -> MEASURE
├── Module 5: Regression Evals as a Production Gate           -> GATE
├── Module 6: Failures at Scale -- Backoff and Circuit Breakers -> HARDEN
├── Module 7: Versioning and Safe Rollout                     -> VERSION
└── Module 8: Project -- the Reservo Agent in Production  ← YOU ARE HERE
    → the four disciplines, wrapping the same agent, at once;
      an instrumented + measured run, the gate run, a rollout with
      NO-GO and rollback, a circuit breaker genuinely opening, and
      the four final artifacts, written to disk.

Notice the right-hand column: the seven previous modules aren't seven loose topics — they're four disciplines (observe, measure, gate, harden+version), each built, tested, and run separately. This module doesn't invent a fifth discipline — it brings all four together around the same agent, in the same working directory, producing real evidence that, together, they hold Reservo up against traffic that doesn't always behave like an isolated module's perfect script.

What you're going to build

Across this module's eight lessons you're going to assemble a package of files, picking back up — without changing its underlying logic — what you already built in this guide's Modules 1-7 and in agent-fundamentals M8:

# El agente, tal como agent-fundamentals M8 lo entregó -- SIN TOCAR
reservo_tools.py           -> las 4 funciones reales de Reservo, con su estado.
reservo_contracts.py       -> los 4 contratos + TOOLS + call_tool.
reservo_robust.py          -> dispatch_robust: valida, atrapa, reintenta.
reservo_agent.py           -> run_reservo_agent: el runner completo.

# La capa de operación, construida en M2-M7 de ESTA guía -- SIN TOCAR
observability/run_logger.py        -> M2: RunEvent, ToolCallEvent, traced_run.
observability/cost_calculator.py   -> M3: estimate_cost_cents, CostReport, cost_for_run.
observability/latency_model.py     -> M4: TOOL_LATENCY_MS, total_run_latency_ms, percentile.
regression/harness.py              -> M5: CaseResult, GateReport, run_case, run_regression_gate.
regression/golden_cases.json       -> M5: el CASE_SET fijo, cinco casos.
resilience/tool_circuit_breaker.py -> M6: retry_with_backoff, CircuitBreaker.
ops/versions/prompt_registry.py    -> M7: AgentVersion, hash_prompt, PROMPT_REGISTRY.

# Lo que este módulo agrega -- pura integración, ningún mecanismo nuevo
ops/rollout.py              -> rollout_decision + rollback (M8, ensambla lo que M7 dejó listo).
ops/metrics_summary.py      -> junta cost_for_run + latency_model en UN reporte por lote (M8).
RUN_LOG.jsonl               -> ENTREGABLE: el log estructurado de un lote real, escrito a disco.
regression_report.json      -> ENTREGABLE: el veredicto del gate, escrito a disco.
AGENT_CHANGELOG.md          -> ENTREGABLE: qué versión corre, y por qué, en texto legible.

Nine of the eleven files are pure reuse: code you already ran, with output you already saw, in a previous module. Only ops/rollout.py and ops/metrics_summary.py have genuinely new assembly — and, as you're going to confirm in Lessons 4 and 5, that assembly is minimal: functions that call, in the right order, pieces that already exist.

Worked example: the environment, verified, and the complete layer's manifest

Before instrumenting a single run, this first step confirms the environment is what this guide's DISEÑO.md promised, and that the seven reused files — four from agent-fundamentals, three from this guide's M2-M7 — import with no conflict in a new directory.

import sys
print("Python:", sys.version.split()[0])

# El agente, sin tocar
import reservo_tools as rt
import reservo_contracts as rc
import reservo_robust as rr
import reservo_agent as ra

# La capa de operación, sin tocar
import run_logger as rl
import cost_calculator as cc
import latency_model as lm
import harness as hn
from tool_circuit_breaker import CircuitBreaker, CLOSED, retry_with_backoff
from prompt_registry import PROMPT_REGISTRY, hash_prompt

print()
print("=== Manifiesto de la capa de operación ===")
print("tools de Reservo        :", list(rc.TOOLS.keys()))
print("ancla 1 -- Focus basic 3h:", rt.get_quote("Focus", "basic", 3))
print("ancla 2 -- Focus pro   3h:", rt.get_quote("Focus", "pro", 3))
print("TOOL_LATENCY_MS          :", lm.TOOL_LATENCY_MS)
print("pricing (centavos/1M tok):", cc.INPUT_PRICE_CENTS_PER_MILLION_TOKENS, "/", cc.OUTPUT_PRICE_CENTS_PER_MILLION_TOKENS)
print("CASE_SET del gate (M5)   :", len(hn.CASE_SET), "casos")
print("versiones en el registro :", list(PROMPT_REGISTRY.keys()))

breaker_demo = CircuitBreaker("book_room", failure_threshold=3, cooldown_calls=2)
print("circuit breaker arranca en:", breaker_demo.state, "(", CLOSED, ")")

What to expect:

Python: 3.14.0

=== Manifiesto de la capa de operación ===
tools de Reservo        : ['list_rooms', 'get_quote', 'book_room', 'cancel_booking']
ancla 1 -- Focus basic 3h: {'price_cents': 7500}
ancla 2 -- Focus pro   3h: {'price_cents': 6000}
TOOL_LATENCY_MS          : {'list_rooms': 40, 'get_quote': 25, 'book_room': 120, 'cancel_booking': 90}
pricing (centavos/1M tok): 300 / 1500
CASE_SET del gate (M5)   : 5 casos
versiones en el registro : ['v1', 'v2']
circuit breaker arranca en: CLOSED ( CLOSED )

None of this output is new: 7500/6000 are the same two get_quote anchors accompanying the guide since agent-fundamentals M2; TOOL_LATENCY_MS and the pricing are the same constants fixed in this guide's M4 and M3; PROMPT_REGISTRY has the same two versions M7 built. The manifest discovers nothing — it confirms the ground you're going to assemble the complete layer on is still solid, exactly as agent-fundamentals M8 did before building its own capstone.

How the eight previous modules support this one

M8 (esta guía)   Capstone -- CUATRO disciplinas, envolviendo al mismo agente, a la vez
     ▲
M7   Versionar    PROMPT_REGISTRY, rollout_decision, rollback -- comparar antes de desplegar
     ▲
M6   Endurecer    CircuitBreaker por tool -- memoria de fallos ENTRE runs
     ▲
M5   Gatear       run_regression_gate -- comparación LITERAL, nunca un juez
     ▲
M4   Medir        TOOL_LATENCY_MS, percentiles -- latencia modelada, nunca time.time()
     ▲
M3   Medir        estimate_cost_cents, cost_for_run -- costo estimado, pricing fijo
     ▲
M2   Observar     traced_run, trace_id determinista -- cada paso, según ocurre
     ▲
M1   El puente    run_and_observe -- la primera envoltura, y su límite exacto
     ▲
agent-fundamentals M1-M8   run_reservo_agent -- EL AGENTE, construido, sin tocar

Every layer in this stack depends, with no exception, on the one beneath it — the same discipline as agent-fundamentals. traced_run (M2) wraps dispatch_robust without touching it; cost_for_run and latency_model (M3/M4) read the history run_reservo_agent produces, without changing how it produces it; M5's gate runs over complete runs, using cost_for_run/latency_for_run without rebuilding them; M6's CircuitBreaker wraps a tool's real function, from outside the loop; and M7's registry identifies prompt versions that same loop uses. This capstone doesn't add a ninth layer — it's the run evidence that all eight, together, don't step on each other.

The map of the eight lessons

  • 02 — Assembling the Operations Layer. M2-M7's complete file package, imported together, with a quick check that every piece still works on its own before putting them to work together.
  • 03 — The Instrumented Run. traced_run (M2) wrapping run_reservo_agent, over a real batch of Reservo tasks — a real RUN_LOG.jsonl, written to disk, read back with no Python variable in memory at all.
  • 04 — The Cost and Latency Report. cost_for_run (M3) and the latency model (M4), over the same batch — cost in cents, total latency, p50/p95 over real runs, a single per-batch summary.
  • 05 — The Regression Gate in the Capstone. run_regression_gate (M5) run against the agent exactly as it stands — PASS. Then, the same criteria, applied to comparing a new prompt version (M7) — NO-GO, and the rollback run.
  • 06 — The Resilience Layer. M6's CircuitBreaker, over book_room genuinely failing, integrated with the rest of the operations layer — the real-calls savings, measured with M3's and M4's tools.
  • 07 — What Your Agent Still Needs. The ecosystem close: where to go when this operated agent has to face infrastructure incidents, semantic judgment, cost reduction, or hardening against attacks.
  • 08 — Project: Ship the Production-Ready Agent. The final checklist, the four artifacts generated end to end in a single run, and a challenge that puts the complete layer to the test with a scenario you haven't seen before.

Why integrating the operations layer is different from building each piece

Each of Modules 2 through 7 tested its piece in isolation, with a script designed on purpose for that piece. That's correct and necessary — that's how reliable operations software gets built, one layer at a time. But integrating four separately built disciplines has a silent cost, different from the one you already saw in agent-fundamentals M8 (there, the cost was about data format — json.dumps versus ast.literal_eval). Here, the cost is about questions: M5 built its regression gate — the five-case CASE_SET, run_regression_gate — to confirm today's agent still works; M7, when comparing two versions of a prompt, didn't build any new fixture — it reused exactly that same CASE_SET and that same function, with its overrides parameter, to answer a related but different question: "is this candidate version safe to replace today's with?" This module's Lesson 5 shows both questions, run over the same mechanism, and traces the boundary precisely — the same kind of integration friction no isolated lesson had a reason to run into.

Common mistakes

  1. Believing this module teaches an observability platform. There's no Datadog, LangSmith, or Sentry anywhere in this capstone — as in every previous module of this guide, it's all pure Python over dicts, dataclasses, and JSON Lines files. The patterns (structured logging, trace_id, regression harness, circuit breaker) are identical with any real platform; what changes is where the events get sent, not their shape.

  2. Rewriting any of the nine reused pieces "so they fit together better." This module's value is in the discipline of not rewriting what already works and is already tested. If cost_for_run and the CircuitBreaker seem not to fit perfectly, the right solution is a new assembly function in ops/, never a silent edit to observability/cost_calculator.py or resilience/tool_circuit_breaker.py.

  3. Thinking M7 uses a different CASE_SET than M5's to compare versions. It doesn't — it reuses exactly the same five-case CASE_SET, and the same run_regression_gate, with an overrides substituting the script of the specific case the candidate version would change. They're two real questions, with different purposes — running the CASE_SET with no overrides confirms today's system still works; running it with overrides confirms a candidate version is safe — but the same fixture answers both. Lesson 5 names the difference precisely; confusing them produces a verdict that doesn't answer the question that actually mattered.

  4. Thinking "the gate passed" means "the agent is ready for production forever." A PASS today is evidence about today's behavior, against today's case set — not a permanent guarantee. Every new run of this capstone is a point-in-time confirmation, not an indefinite certificate.

  5. Jumping straight to Lesson 8 without going through 02-07. Lesson 8's four final artifacts make no sense without having seen, separately, that each discipline — observe, measure, gate, harden+version — works before bringing them together into a single run.

Summary and next step

  • This module doesn't add any new discipline: it assembles, around the same Reservo agent, the four M2-M7 built and ran separately — observe, measure, gate, harden + version.
  • You confirmed the environment — Python 3.14.0 — and that the seven reused files (four from agent-fundamentals, three from this guide) import together with no conflict, with the same constants as always: get_quote's anchors (7500/6000), TOOL_LATENCY_MS, the fixed pricing, and PROMPT_REGISTRY's two versions.
  • The map of the eight lessons makes clear what each one picks back up, and where the real integration friction shows up: Lesson 5, where two fixtures from M5 and M7 answer related but different questions.

Next lesson: 02 — Assembling the Operations Layer. We pick M2-M7's seven files back up, side by side, and confirm every piece still works on its own before putting them to operate together over a real run.

Additional resources

  1. Anthropic — Tool use (function calling) overview — The protocol this module's operations layer instruments, measures, and gates end to end.
  2. Anthropic — Building effective agents — Why operating an agent that already works is its own discipline, different from building it.
  3. Python 3.14 — What's New — The exact version all of this guide's engineering runs on.
  4. Python — Modules — How Python resolves the import across the eleven files you're going to assemble in this directory.
← PreviousMini-Project: A Versioned Rollout for ReservoNext →Assembling the Operations Layer