Agentic retrieval end to end

Description

Through Lesson 03, search_docs was always called on its own, directly from a Python console. This lesson connects it with Module 4's runner — run_agent/dispatch_parallel, reused unchanged from agent-fundamentals-and-tool-calling — and with get_quote, Reservo's canonical pricing tool, inside a single conversation. It's this project's central demonstration: a real question, mixing a document policy with a computed price, resolved by an agent that decides to combine two different tools in the same turn, with a final answer verified by check_grounding — zero numbers without backing.

Connection to the module

This lesson picks up Lesson 03's search_docs without touching a line, and adds Module 4's full agentic runner — dispatch_parallel, run_agent, check_grounding — alongside reservo_tools.py, verbatim from agent-fundamentals-and-tool-calling. It's the project's first lesson where search_docs stops being an isolated function and starts living inside a real agent.

Analogy: the desk serves its first real person

Previous lessons prepared the whole building — documents received, catalog assembled, desk with its sign — but until now no one from outside came in to ask anything for real. This lesson is the moment the first real person walks through the door with a question mixing two needs at once: they want to know a policy (which requires consulting the document catalog) and also how much it would cost to book (which requires a computed rate, not a document). The person at the desk doesn't send them to two different windows — they resolve both parts of the question in the same conversation, and when they answer, they can point exactly to where each fact came from: "this comes from the cancellation policy, and this price I computed myself with the current rate".

Step 1: reservo_tools.py, verbatim from agent-fundamentals-and-tool-calling

# reservo_tools.py
import itertools

ROOM_RATE_CENTS = {"Focus": 2500, "Studio": 4000, "Boardroom": 8000}
BOOKINGS = {}
_booking_ids = itertools.count(1)


def get_quote(room, tier, hours):
    base = ROOM_RATE_CENTS[room] * hours
    price_cents = base if tier == "basic" else base * 80 // 100
    return {"price_cents": price_cents}


def book_room(room, tier, hours, member):
    quote = get_quote(room, tier, hours)
    booking_id = next(_booking_ids)
    BOOKINGS[booking_id] = {
        "booking_id": booking_id, "room": room, "tier": tier,
        "hours": hours, "member": member, "price_cents": quote["price_cents"],
    }
    return {"booking_id": booking_id, "confirmed": True}


def cancel_booking(id):
    if id in BOOKINGS:
        del BOOKINGS[id]
        return {"cancelled": True}
    return {"cancelled": False}

agent-fundamentals-and-tool-calling's three price anchors travel unchanged into this project: Focus 2500¢/h, Studio 4000¢/h, Boardroom 8000¢/h, whole-number pro discount *80//100. None of these rates gets recomputed or touched — this project uses get_quote, it doesn't rewrite it.

Step 2: the agentic runner, reused unchanged

# agent_runner.py
import concurrent.futures
import re

import reservo_tools as rt
from search_docs_tool import search_docs

TOOLS = {
    "get_quote": rt.get_quote,
    "book_room": rt.book_room,
    "cancel_booking": rt.cancel_booking,
    "search_docs": search_docs,
}


def dispatch_parallel(tool_use_blocks, tools):
    with concurrent.futures.ThreadPoolExecutor(max_workers=len(tool_use_blocks)) as pool:
        futures = [pool.submit(tools[b["name"]], **b["input"]) for b in tool_use_blocks]
        results = [f.result() for f in futures]
    return [
        {"type": "tool_result", "tool_use_id": b["id"], "content": str(r)}
        for b, r in zip(tool_use_blocks, results)
    ]


def run_agent(question, model_script, tools, max_iterations=10):
    messages = [{"role": "user", "content": question}]
    for step in range(max_iterations):
        turn = model_script[step]
        messages.append({"role": "assistant", "content": turn["content"]})
        if turn["stop_reason"] != "tool_use":
            return turn, messages
        tool_result_blocks = dispatch_parallel(turn["content"], tools)
        messages.append({"role": "user", "content": tool_result_blocks})
    raise RuntimeError(f"max_iterations reached ({max_iterations})")


def check_grounding(final_text, history):
    claimed = set(re.findall(r"\b\d{3,}\b", final_text))
    seen = set()
    for m in history:
        content = m["content"]
        if isinstance(content, list):
            for block in content:
                if block["type"] == "tool_result":
                    seen |= set(re.findall(r"\b\d{3,}\b", block["content"]))
    return sorted(claimed - seen, key=int)

TOOLS is the registry a real Reservo agent would declare in full: three structured tools (get_quote, book_room, cancel_booking) and one retrieval tool (search_docs), in a single name -> function dictionary. dispatch_parallel runs, in the same turn, every tool_use the model requested at once, with a ThreadPoolExecutor — it's what makes it possible for search_docs and get_quote to run together, in parallel, within a single step of the loop.

Worked example: search_docs + get_quote, in the same turn

The compound question: Reservo's cancellation policy in pro mode, and the price of booking Focus for 3 hours in pro mode. Neither part is covered by a single tool: the first needs search_docs, the second needs get_quote. model_script realistically represents what claude-sonnet-5 would decide — this part is conceptual, not a real call to any API:

from agent_runner import run_agent, check_grounding, TOOLS
import reservo_tools as rt

model_script = [
    # Turn 1: the model decides to combine search_docs with get_quote --
    # both tools run together, via dispatch_parallel.
    {"stop_reason": "tool_use", "content": [
        {"type": "tool_use", "id": "toolu_01A", "name": "search_docs",
         "input": {"query": "What is Reservo's cancellation policy for the pro tier?", "k": 3}},
        {"type": "tool_use", "id": "toolu_01B", "name": "get_quote",
         "input": {"room": "Focus", "tier": "pro", "hours": 3}},
    ]},
    # Turn 2: final answer, citing both already-retrieved facts.
    {"stop_reason": "end_turn", "content": [
        {"type": "text", "text": (
            "Focus has a pro discount: you can cancel free of charge up to 4 hours before "
            "the reservation start time (cancellation-policy). Booking it for 3 hours in "
            "pro mode costs 6000 cents ($60.00)."
        )}]},
]

final, history = run_agent(
    "What is Boardroom's cancellation policy in pro mode, and how much would it cost "
    "to book Focus for 3 hours in pro mode?",
    model_script, TOOLS,
)

for i, m in enumerate(history):
    role, content = m["role"], m["content"]
    if isinstance(content, str):
        print(f"  [{i}] {role:<9} question: {content[:70]!r}...")
        continue
    for block in content:
        if block["type"] == "tool_use":
            print(f"  [{i}] {role:<9} tool_use({block['name']}): {block['input']}")
        elif block["type"] == "tool_result":
            print(f"  [{i}] {role:<9} tool_result: {block['content'][:100]}")
        elif block["type"] == "text":
            print(f"  [{i}] {role:<9} final text: {block['text']!r}")

print("\nFINAL ANSWER:", final["content"][0]["text"])
print("loop iterations:", sum(1 for m in history if m["role"] == "assistant"))
print("numbers without backing (check_grounding):", check_grounding(final["content"][0]["text"], history))
print("\nANCHOR get_quote('Focus','pro',3):", rt.get_quote("Focus", "pro", 3))

What to expect (run):

  [0] user      question: "What is Boardroom's cancellation policy in pro mode, and how much woul"...
  [1] assistant tool_use(search_docs): {'query': "What is Reservo's cancellation policy for the pro tier?", 'k': 3}
  [1] assistant tool_use(get_quote): {'room': 'Focus', 'tier': 'pro', 'hours': 3}
  [2] user      tool_result: [{'chunk_id': 'cancellation-policy-003', 'doc_id': 'cancellation-policy', 'text': 'See `refund-polic
  [2] user      tool_result: {'price_cents': 6000}
  [3] assistant final text: 'Focus has a pro discount: you can cancel free of charge up to 4 hours before the reservation start time (cancellation-policy). Booking it for 3 hours in pro mode costs 6000 cents ($60.00).'

FINAL ANSWER: Focus has a pro discount: you can cancel free of charge up to 4 hours before the reservation start time (cancellation-policy). Booking it for 3 hours in pro mode costs 6000 cents ($60.00).
loop iterations: 2
numbers without backing (check_grounding): []

ANCHOR get_quote('Focus','pro',3): {'price_cents': 6000}

Two loop iterations, not one more, not one less. Turn [1] combines search_docs and get_quote in parallel — exactly the pattern dispatch_parallel enables — and turn [3] closes with an answer citing both retrieved facts. check_grounding confirms the answer's only 3+ digit number (6000) has real backing in some observed tool_result — it comes straight from get_quote — and flags no invented number. The anchor Focus pro 3h = 6000 cents — the same one running through this guide and agent-fundamentals-and-tool-calling since its origin — gets confirmed, once more, with real execution.

Notice something important about search_docs's first tool_result: the chunk that came back (cancellation-policy-003) is, again, the lexical magnet of cross-references this module's Lessons 02-03 already identified — not the chunk with the exact "4 hours" figure. The final answer, however, does correctly mention "4 hours before" — that's conceptual: it represents what claude-sonnet-5 might write reading the full tool_result (not just the first chunk truncated in the printout above) and combining it with the knowledge that Module 4's Lesson 03 already covered how to reformulate a query when the first result is a cross-reference note. A real production runner would build that reformulation explicitly, as Module 4's mini-project did with four turns instead of two — this lesson uses a shorter script on purpose, to keep the focus on the central piece: combining search_docs with get_quote in the same turn.

Common mistakes

1. Thinking dispatch_parallel runs different turns in parallel. dispatch_parallel's parallelism is within a single turn — when the model requests two or more tools at once — not across consecutive turns of the loop. Turns [1] and [3] in this example are sequential with each other; what runs in parallel is search_docs and get_quote, both within turn [1].

2. Forgetting to run check_grounding on the final answer. Confirming get_quote returned 6000 isn't the same as confirming the final answer, as a user would actually read it, cites that number unaltered. check_grounding is the check on the text that would genuinely be delivered, not on the tools separately.

3. Confusing the real execution of search_docs/get_quote with the model's decision to call them. search_docs("...") and get_quote("Focus", "pro", 3) run on real Python, against this project's real index and rates. model_script — the decision of which tools to call and what text to write at the end — is conceptual: a script realistically representing what claude-sonnet-5 would decide, with no API genuinely called.

4. Assuming two tools in the same turn always means they're related. search_docs and get_quote, in this example, answer different parts of the same compound question — they don't pass data to each other. dispatch_parallel doesn't coordinate the content of the tools it runs, it just runs them together and gathers their results; semantic coordination (the two answers fitting into one coherent final text) is the final writing's job, not the runner's.

Exercises

Exercise 1: A question that only needs one tool (Easy)

Without running anything: if the question were only "How much does it cost to book Boardroom for 1 hour in basic mode?" (no mention of any policy), how many loop turns would you expect, and which tool would get called in the first one? Confirm with a two-turn script.

simple_script = [
    {"stop_reason": "tool_use", "content": [
        {"type": "tool_use", "id": "toolu_01", "name": "get_quote",
         "input": {"room": "Boardroom", "tier": "basic", "hours": 1}}]},
    {"stop_reason": "end_turn", "content": [
        {"type": "text", "text": "Boardroom basic 1h costs 8000 cents ($80.00)."}]},
]

final_simple, history_simple = run_agent(
    "How much does it cost to book Boardroom for 1 hour in basic mode?", simple_script, TOOLS,
)
print("turns:", sum(1 for m in history_simple if m["role"] == "assistant"))
print("answer:", final_simple["content"][0]["text"])
print("grounding:", check_grounding(final_simple["content"][0]["text"], history_simple))
print("anchor:", rt.get_quote("Boardroom", "basic", 1))

turns: 2
answer: Boardroom basic 1h costs 8000 cents ($80.00).
grounding: []
anchor: {'price_cents': 8000}

Exercise 2: Break grounding on purpose (Medium)

Write a version of the final text that swaps "6000 cents" for a made-up number, keeping the rest of the answer the same. Run check_grounding on that version, using the same history from the original run.

broken_text = (
    "Focus has a pro discount: you can cancel free of charge up to 4 hours before "
    "the reservation start time (cancellation-policy). Booking it for 3 hours in "
    "pro mode costs 6300 cents ($63.00)."
)
print(check_grounding(broken_text, history))

['6300']

Exercise 3: Add a third, independent hop (Hard)

Extend the original script with one more turn, before the final answer, that uses search_docs to answer "Is there wifi in the Lounge?" — a third information need, independent of the first two. Adjust the final text to mention the result, and confirm check_grounding still returns [].

model_script_v2 = [
    model_script[0],  # same turn 1: search_docs + get_quote in parallel
    {"stop_reason": "tool_use", "content": [
        {"type": "tool_use", "id": "toolu_02", "name": "search_docs",
         "input": {"query": "Is there wifi in the Lounge?", "k": 2}}]},
    {"stop_reason": "end_turn", "content": [
        {"type": "text", "text": (
            "Focus has a pro discount: you can cancel free of charge up to 4 hours before "
            "the reservation start time (cancellation-policy). Booking it for 3 hours in "
            "pro mode costs 6000 cents ($60.00). Also, Lounge has high-speed wifi "
            "included, like every Reservo room."
        )}]},
]

final_v2, history_v2 = run_agent(
    "What is Boardroom's cancellation policy in pro mode, how much would it cost "
    "to book Focus for 3 hours in pro mode, and does Lounge have wifi?",
    model_script_v2, TOOLS,
)
print("turns:", sum(1 for m in history_v2 if m["role"] == "assistant"))
print("grounding:", check_grounding(final_v2["content"][0]["text"], history_v2))

turns: 3
grounding: []

Summary and next step

• reservo_tools.py (verbatim from agent-fundamentals-and-tool-calling) and agent_runner.py (verbatim from Module 4) are now assembled alongside Lesson 03's search_docs.
• You ran a two-turn agent that combines search_docs and get_quote in the same turn via dispatch_parallel, with a final answer verified by check_grounding — zero numbers without backing.
• The anchor Focus pro 3h = 6000 cents got confirmed, once more, with real execution — the same figure running through this guide and agent-fundamentals-and-tool-calling since its origin.

Next lesson: 05 — Incremental reingest in the capstone. search_docs works today — but Reservo's documents change. The next piece confirms the pipeline survives a modified document and a deleted one, in the same run, with nothing duplicated or lost.

Additional resources

1. production-rag-and-document-ingestion-guide — Module 4 (module-04-agentic-retrieval-in-the-loop): the full source of run_agent/dispatch_parallel/check_grounding.
2. agent-fundamentals-and-tool-calling-guide — Modules 2, 4, and 5: the origin of reservo_tools.py, the agent loop, and check_grounding, reused unchanged in this lesson.
3. Anthropic — Tool use (function calling) overview — the full reference for the tool_use/tool_result protocol run in this lesson.
4. Python — concurrent.futures — the ThreadPoolExecutor behind dispatch_parallel.