A Tool That Fails Repeatedly

Description

Before building any solution, it's worth seeing the problem with your own eyes, run for real. This lesson doesn't add any resilience mechanism yet — that starts in Lesson 3 — it stops, on purpose, at the exact moment something starts going wrong, and lets the code show you what the Reservo agent, exactly as it stands today, is missing: memory across runs. You're going to simulate a real book_room outage and see, run for real, how three different users — Ana, Luis, and Sofía — arrive one after another while the tool is still down, and how the agent discovers that same outage three times, from scratch, with neither the second nor the third time learning anything from the first.

Connection to the module

This lesson uses, without touching, agent-fundamentals Module 8's run_reservo_agent runner and its Module 7 error handling (dispatch_robust, which catches any real exception from a tool and turns it into a tool_result with is_error: true, without letting the run crash). That already works, and keeps working exactly the same here. What this lesson exposes is what that mechanism doesn't solve: every run is an island. dispatch_robust has no way of knowing that the tool it's about to call already failed twice in the last five minutes, for two other users.

Analogy: the same call to a restaurant, three different nights

Imagine you call a restaurant to book a table, and nobody picks up — the line is down. The next day, a friend of yours, knowing nothing about your failed call, tries the same thing — and nobody picks up either. On the third day, someone else entirely, completely unaware of the two previous calls, dials the same number — and runs into the same silence. Three different people discovered, each on their own, exactly the same fact: that restaurant isn't answering the phone. None of the three found out the other two had already tried and failed. If there were a shared notebook at the restaurant's door — "not answering since Monday" — the second and third person would have saved themselves the entire call.

That's exactly what the Reservo agent lacks in this lesson: there's no shared notebook. Every run_reservo_agent run is like each of those three people — it discovers book_room's outage on its own, pays the full cost of discovering it, and leaves no trail that helps the next run skip that discovery. Lesson 4 builds exactly that notebook — the circuit breaker.

Worked example: three runs, the same downed tool, no memory

A tool that fails sustainedly

We reuse the exact pattern agent-fundamentals M7 (Lesson 6) used with flaky_list_rooms: a test tool, declared just for this lesson, that wraps the real tool and fails a set number of times before recovering. This time the tool is book_room — the write one, the one the user really needs to work — and the outage lasts six real calls, more than any individual user is going to try on their own.

import reservo_tools as rt
import reservo_agent as ra

_book_room_real = rt.book_room
_state = {"count": 0}
OUTAGE_CALLS = 6


def flaky_book_room(room, tier, hours, member):
    """Simula una tool con una dependencia inestable: falla las primeras
    seis llamadas con un error TRANSITORIO. No es parte del contrato
    canónico de Reservo -- se declara para esta lección, igual que
    agent-fundamentals M7 hizo con flaky_list_rooms."""
    _state["count"] += 1
    if _state["count"] <= OUTAGE_CALLS:
        raise ConnectionError(f"timeout de red simulado (intento {_state['count']})")
    return _book_room_real(room, tier, hours, member)


ra.TOOL_FUNCS["book_room"] = flaky_book_room

The last line is the same "wrap without touching" technique you already know from Module 2: we replace, from outside, the TOOL_FUNCS dictionary's "book_room" entry that dispatch_robust uses — without editing a single line of reservo_agent.py. dispatch_robust keeps doing exactly what it always did: it calls the function registered under that name, and if it raises an exception, it catches it and builds a tool_result with is_error: true. It doesn't know, and doesn't need to know, that the function it's calling now is an unstable version.

Three users, three runs, the same outage discovered three times

def make_script(room, tier, hours, member):
    return [
        {"stop_reason": "tool_use", "content": [
            {"type": "tool_use", "id": "toolu_01", "name": "book_room",
             "input": {"room": room, "tier": tier, "hours": hours, "member": member}}]},
        {"stop_reason": "end_turn", "content": [
            {"type": "text", "text": f"No pude confirmar la reserva de {room} para {member} -- el sistema de reservas no respondió. Intenta de nuevo en unos minutos."}]},
    ]


print("=== tres runs independientes, la misma tool caída, SIN memoria entre ellos ===")
users = [("Focus", "pro", 3, "Ana"), ("Studio", "pro", 2, "Luis"), ("Boardroom", "pro", 1, "Sofía")]
for i, (room, tier, hours, member) in enumerate(users, start=1):
    final, history = ra.run_reservo_agent(f"Reserva {room} para {member}", make_script(room, tier, hours, member))
    tool_result = history[2]["content"][0]
    print(f"--- run {i} ({member}) ---")
    print(f"  tool_result: is_error={tool_result['is_error']} content={tool_result['content']!r}")
    print(f"  RESPUESTA: {final['content'][0]['text']}")
print()
print(f"llamadas reales a book_room hasta ahora: {_state['count']} (las 3 fallaron -- el apagón sigue)")

What to expect:

=== tres runs independientes, la misma tool caída, SIN memoria entre ellos ===
--- run 1 (Ana) ---
  tool_result: is_error=True content='ConnectionError: timeout de red simulado (intento 1)'
  RESPUESTA: No pude confirmar la reserva de Focus para Ana -- el sistema de reservas no respondió. Intenta de nuevo en unos minutos.
--- run 2 (Luis) ---
  tool_result: is_error=True content='ConnectionError: timeout de red simulado (intento 2)'
  RESPUESTA: No pude confirmar la reserva de Studio para Luis -- el sistema de reservas no respondió. Intenta de nuevo en unos minutos.
--- run 3 (Sofía) ---
  tool_result: is_error=True content='ConnectionError: timeout de red simulado (intento 3)'
  RESPUESTA: No pude confirmar la reserva de Boardroom para Sofía -- el sistema de reservas no respondió. Intenta de nuevo en unos minutos.

llamadas reales a book_room hasta ahora: 3 (las 3 fallaron -- el apagón sigue)

Each of the three asked for a different room — Focus, Studio, Boardroom — and each one ran into exactly the same fate: a real call to book_room, a ConnectionError, and an honest final response with no real hope that "try again in a few minutes" helps, because nothing in the system knows it needs to wait longer than that, or that three users have already run into the same thing. dispatch_robust did its job perfectly all three times — it caught the error, didn't let the run crash, produced a clear response. What it didn't do, because it isn't designed to, was tell the next run that it already knows the answer.

The cost of not remembering

With just three users the problem is already visible; at real scale — hundreds of users arriving during an outage that lasts minutes, not milliseconds — the cost multiplies exactly in proportion to the number of people unlucky enough to ask for something during the downed window. Every one of those real book_room calls costs the same as it would if the tool were healthy: the wait time until the timeout (which Module 4 would measure in milliseconds), and if book_room were, in a real system, a call to a paid service or an external API with a cost per invocation, every failed attempt would also be money spent with no result at all — exactly what Module 3 would measure in cents.

Notice something subtler: in this lesson, none of the three retries within their own run — each user's script has a single call to book_room. That's on purpose: the problem this lesson raises isn't "how many times do I retry within a run?" — that question was already solved by agent-fundamentals M7, and this module's Lesson 3 is going to scale it up with backoff. The problem is a completely different one: run after run, nobody remembers anything. That's, precisely, the gap Lessons 4 and 5's circuit breaker comes to close.

Common mistakes

1. Thinking the solution is "add more retries within the run." That solves nothing here — the problem isn't that each user has too few chances within their own run, it's that the whole system doesn't remember from one run to the next. Ten retries per run would still discover the same outage, ten times each, for every new user.

2. Confusing "the tool returned is_error" with "the system failed." dispatch_robust worked exactly as it should in all three cases — it caught the real error, never let the run crash uncontrolled, and produced a clear response for the user. The system didn't "fail" in the sense of a bug — what's missing is a completely different layer, which this lesson doesn't build yet.

3. Assuming logging the error is enough to "remember" it. Module 2 already leaves a complete record of every tool_result with is_error: true in RUN_LOG.jsonl — but a log is a history someone has to read later. What's needed here is a decision in the moment, before the next call: is it worth trying, or do we already know it isn't?

Exercises

Exercise 1: Calculate how many real calls get wasted (Easy)

Without running Python: if book_room's outage lasts OUTAGE_CALLS = 6 real calls, and five different users arrive — each with a single attempt at book_room in their script, like in this lesson's example — how many of the five get a ConnectionError? How many real calls got spent in total, with no useful result?

Exercise 2: A sixth user, right at the edge (Medium)

Using this lesson's same flaky_book_room (OUTAGE_CALLS = 6), run a sixth and a seventh user after the worked example's three (Ana, Luis, Sofía already spent real calls 1, 2, and 3). Without running Python first: is the sixth user (real call #4) going to fail or succeed? And the seventh (real call #5)? Then, run it and confirm.

users_extra = [("Focus", "basic", 1, "Marco"), ("Studio", "basic", 1, "Julia")]
for i, (room, tier, hours, member) in enumerate(users_extra, start=4):
    final, history = ra.run_reservo_agent(f"Reserva {room} para {member}", make_script(room, tier, hours, member))
    tool_result = history[2]["content"][0]
    print(f"run {i} ({member}): is_error={tool_result['is_error']}")

run 4 (Marco): is_error=True
run 5 (Julia): is_error=True

Exercise 3: Why isn't checking RUN_LOG.jsonl before every run enough? (Hard)

Module 2 left a RUN_LOG.jsonl with a tool_result event for every call, including the ones with is_error: true. Someone proposes: "before every new run, let's read the complete log and count how many book_room tool_results with is_error: true there were in the last N events — if there are a lot, we don't call the tool." Explain, in one paragraph, what practical problem that proposal has compared to a circuit breaker living in memory inside the process, and why this guide chooses the second option for Lessons 4 and 5.

Summary and next step

• We ran, with flaky_book_room, this module's central problem: three different users — Ana, Luis, Sofía — each with their own run, each discovering from scratch, with a real failed call, that book_room is down.
• We confirmed dispatch_robust (agent-fundamentals M7) works exactly as it should — it catches the real error, never lets the run crash, produces a clear response — and that, by design, doesn't include any memory from one run to the next.
• The cost of not remembering grows in direct proportion to the number of users arriving during the downed window: each one pays the full cost of discovering, on their own, what was already known.
• The next step isn't "more retries within the run" — agent-fundamentals M7 already solved that. It's a memory layer that persists across runs, and before that, a concrete improvement to the retry itself: backoff.

Next lesson: 03 — Retries with Bounded Backoff. Before building memory across runs, we close a piece agent-fundamentals M7 deliberately left pending: how long to wait between one attempt and the next, within the same run.

Additional resources

1. Python — Built-in exceptions (ConnectionError) — The standard exception this lesson, like agent-fundamentals M7, uses to represent a transient failure.
2. Anthropic — Building effective agents — On why a production agentic system needs to anticipate its dependencies' failures, not just handle them one by one once they've already happened.
3. resilience-and-reliability-patterns-guide — The same problem — a downed dependency, discovered over and over with no memory — is that sister guide's starting point for generic distributed systems, with its own case (Mercado).