Module 4: Agentic Retrieval in the Loop

Multi-hop Retrieval

Description

Lesson 03 solved one case: the same question, two attempts, the second query better phrased than the first. This lesson solves a different case, though similar at first glance: a question that's really two questions about independent topics, packed into a single sentence. "What's Reservo's cancellation policy, and do all rooms include high-speed wifi at no extra cost?" isn't a question a reformulation can fix — it's a question that needs two separate searches, with two different queries, because each half lives in a different corpus document.

This is called multi-hop retrieval: instead of a single search_docs call trying to cover everything, the agent makes several "hops," each with its own query focused on a single information need. You're going to see, with executed evidence, something that isn't obvious until you measure it: packing both questions into a single query doesn't just fail to help — it buries one of the two signals so deep in the ranking that, for practical purposes, it disappears.

Connection to the module

This lesson generalizes Lesson 03's "a second search" pattern to questions made up of genuinely different topics, not a reformulation of the same one. Lesson 05 revisits this same idea of decomposing a compound question, but when one of the two parts needs a structured tool (get_quote) instead of a second search.

Analogy: two questions at the same desk, two trips to the archive

The module introduction's researcher gets a compound question: "what's the cancellation policy, and is there wifi in every room?" They could try making a single request to the archive with both parts together — but the archivist, hearing a long question that mixes two topics, doesn't know which of the two to prioritize, and it's easy for the folder they bring back to answer one of the two topics well while leaving the other out entirely. A methodical researcher splits the question into two concrete requests, one at a time: first "bring me the cancellation policy," then "bring me information about wifi in the rooms." Two trips to the archive, yes — but each one comes back with the right folder, instead of one trip that comes back with a single, half-right folder.

The problem, measured: a single combined query

Before seeing the solution, you have to see the problem with real numbers. This is the compound question, turned into a single search_docs query:

from rag_index import INDEX

combined_query = "What is the cancellation policy and is wifi included in every room?"
hits = INDEX.search(combined_query, k=57)  # k=57: the entire corpus, just to measure

for rank, (chunk, score) in enumerate(hits[:6], start=1):
    print(f"  #{rank}  score={score:<7} {chunk.chunk_id}")

# what position does the chunk that actually answers "does every room
# have wifi" show up at (wifi-and-equipment-faq-000, not just any chunk of the doc)?
for rank, (chunk, score) in enumerate(hits, start=1):
    if chunk.chunk_id == "wifi-and-equipment-faq-000":
        print(f"\nwifi-and-equipment-faq-000: #{rank}  score={score}")
        break

What to expect:

  #1  score=12.158  cancellation-policy-003
  #2  score=10.113  no-show-policy-000
  #3  score=9.466   cancellation-policy-002
  #4  score=6.9     no-show-policy-001
  #5  score=6.086   operations-manual-raw-003
  #6  score=5.724   focus-room-manual-000

wifi-and-equipment-faq-000: #14  score=3.816

This is conclusive: the cancellation half of the question dominates the top spots — three different cancellation-policy chunks, one from no-show-policy, and nothing from wifi-and-equipment-faq in sight — and the chunk that actually answers the wifi half (wifi-and-equipment-faq-000, the one that says "Yes. All Reservo rooms include high-speed wifi at no extra charge") doesn't show up until position #14 of 57, with a score (3.816) less than a third of the top-1's. With k=3 — the typical value you'd use in production — the wifi half doesn't show up at all. It's not that BM25 "ignores" the wifi question: it's that "cancellation policy"'s words are, in this corpus, more numerous and more discriminating than "wifi included every room"'s, so their sum of contributions dominates the entire ranking, drowning out the other half of the question.

The solution: two hops, each with its own query

Instead of a combined query, two independent search_docs calls, each focused on a single topic:

from search_docs_tool import search_docs

hop1 = search_docs("What is Reservo's cancellation policy?", k=3)
hop2 = search_docs("Does every room include high-speed wifi at no extra charge?", k=3)

print("--- hop 1: cancellation policy ---")
for r in hop1:
    print(f"  score={r['score']:<7} doc_id={r['doc_id']:<24} {r['chunk_id']}")

print("--- hop 2: wifi in every room ---")
for r in hop2:
    print(f"  score={r['score']:<7} doc_id={r['doc_id']:<24} {r['chunk_id']}")

What to expect:

--- hop 1: cancellation policy ---
  score=10.734  doc_id=cancellation-policy       cancellation-policy-003
  score=7.217   doc_id=refund-policy             refund-policy-000
  score=6.364   doc_id=no-show-policy            no-show-policy-000
--- hop 2: wifi in every room ---
  score=25.4    doc_id=wifi-and-equipment-faq    wifi-and-equipment-faq-000
  score=5.717   doc_id=operations-manual-raw     operations-manual-raw-000
  score=3.652   doc_id=focus-room-manual         focus-room-manual-000

Compare it with the combined query: in hop2, wifi-and-equipment-faq-000 doesn't just show up — it's the top-1, with a score (25.4) more than four times second place. That same chunk, in the combined query, was buried at position #14 with a score of just 3.816 — almost seven times less. Nothing changed in the index or the corpus between the two searches: the only thing that changed is that hop2's query has all its terms focused on the wifi topic, instead of sharing space with the cancellation question's terms.

hop1, as you already know from Lesson 03, brings back the pointer note again (cancellation-policy-003) — this module doesn't repeat the full reformulation here (you already saw that), but it's worth noting: multi-hop and reformulating aren't mutually exclusive. A real script could reformulate hop1 with Lesson 03's specific query before combining both results into the final answer — Lesson 08 (the mini-project) does exactly that.

Why this happens: BM25's arithmetic, in one sentence

Recall Module 2's BM25 formula: a chunk's score is the sum, term by term, of the contribution from every query word that appears in the chunk. When the query has eight or ten terms spread across two different topics, each chunk can only add up the contribution from the terms of its topic — a wifi chunk gains nothing from the words "cancellation" or "policy." Meanwhile, chunks from the topic with more shared terms (in this case, cancellation, because the combined query has more words on that side) rack up more points in absolute terms, simply because there are more terms from their topic in the query to add up. It's not that BM25 "prefers" cancellation — it's summation arithmetic, applied to a query that gives one topic more opportunities to add up than the other.

Boundary: retrieving two chunk sets isn't the same as combining them in the prompt

This module stops exactly where retrieval ends: two search_docs calls, two lists of chunks, each with its own doc_id and score. How those two lists get combined into the text the model finally sees — do they go one after the other? Do they get interleaved? What happens if together they add up to more text than fits in the context window? — is context-engineering-guide's job, not this module's. This guide retrieves the candidate chunks from each hop; that sibling guide decides how they enter the prompt. A real production runner needs both pieces working together, but they're different decisions, with different engineering behind them.

Common mistakes

Combining two topics into a single query "to save a call." As you just measured, saving one search_docs call can cost you half the answer — the weaker topic's signal can end up completely buried. The cost of a second call (one more query to the index, instant on this corpus) is much smaller than the cost of losing relevant information.
Assuming multi-hop always means "two sequential calls, one waiting for the other." In this case, hop1 and hop2 are completely independent of each other — neither needs the other's result to run — so, as you'll see in Lesson 05, they could be requested in the same turn, in parallel, not one after the other. Multi-hop describes how many searches are needed, not necessarily in what order.
Using a high k on the combined query "to compensate" instead of splitting into hops. Raising k on the combined query (say, to k=57 like we did to measure the problem) exposes more results, but it doesn't fix the order — you'd still have to scan through fourteen positions to find the first wifi chunk, when the real tool's MAX_K=5 wouldn't even let you get that far.
Forgetting each hop needs its own reading, not just its own score. Just like in Lesson 03, a correct doc_id in a hop's top-1 doesn't guarantee the chunk is the ideal one — each hop gets evaluated with the same criteria as any other search_docs call.

Exercises

Exercise 1: Predict before running it (Easy)

Without running anything: for the compound question "How much discount does the pro tier get, and what equipment does the Boardroom room have?", do you expect a single combined query to find both halves well, or for one to dominate the other? Which two separate queries would you use? Confirm by running all three (the combined one and the two separate ones) with k=3.

See solution

Reasonable prediction: one of the two halves is likely to dominate, for the same arithmetic reason as this lesson — unless both topics share very little vocabulary with the rest of the corpus and each has a chunk with a very strong, specific signal.

combined = search_docs("How much discount does the pro tier get and what equipment does the Boardroom have?", k=3)
hop_a = search_docs("How much discount does the pro tier get?", k=3)
hop_b = search_docs("What equipment does the Boardroom have?", k=3)

for label, results in [("combined", combined), ("hop A (discount)", hop_a), ("hop B (equipment)", hop_b)]:
    print(f"--- {label} ---")
    for r in results:
        print(f"  {r['score']:<7} {r['doc_id']}")

Expected output:

--- combined ---
  12.298  membership-tiers-faq
  9.312   cancellation-policy
  5.981   boardroom-room-manual
--- hop A (discount) ---
  11.998  membership-tiers-faq
  8.936   cancellation-policy
  5.217   membership-tiers-faq
--- hop B (equipment) ---
  4.897   boardroom-room-manual
  4.632   cancellation-policy
  3.371   cancellation-policy

Explanation: in this specific case, the combined query does bring boardroom-room-manual into the top-3 (position #3, score 5.981) — even better, in fact, than hop B on its own, where its own top-1 (4.897) has only a razor-thin margin over cancellation-policy (4.632), practically a tie. This shows something more nuanced than "combining always buries": dilution isn't a guaranteed failure in every specific case — sometimes one topic's noise happens to help the other, as here. What is constant is that you can't know ahead of time which of the two effects you're going to get without measuring each case — the worked example (cancellation + wifi) showed a catastrophic burial; this case (discount + equipment) didn't. Trusting that "this time the combined query will be enough" is a gamble; splitting into hops removes the gamble.

Exercise 2: Confirm even a generous `k` doesn't rescue the buried hop (Medium)

Picking back up the worked example's combined query ("What is the cancellation policy and is wifi included in every room?"), confirm that not even with k=10 — double the tool's real MAX_K=5 — does wifi-and-equipment-faq-000 show up among the results. Compare with the focused hop, where k=1 is enough.

See solution

from rag_index import INDEX

combined_query = "What is the cancellation policy and is wifi included in every room?"
hits10 = INDEX.search(combined_query, k=10)
print("combined k=10, doc_ids:", [c.doc_id for c, s in hits10])
print("wifi-and-equipment-faq-000 present?",
      any(c.chunk_id == "wifi-and-equipment-faq-000" for c, s in hits10))

focused_hits1 = INDEX.search("Does every room include high-speed wifi at no extra charge?", k=1)
print("focused hop k=1:", [(c.chunk_id, s) for c, s in focused_hits1])

Expected output:

combined k=10, doc_ids: ['cancellation-policy', 'no-show-policy', 'cancellation-policy', 'no-show-policy', 'operations-manual-raw', 'focus-room-manual', 'wifi-and-equipment-faq', 'boardroom-room-manual', 'studio-room-manual', 'phonebooth-room-manual']
wifi-and-equipment-faq-000 present? False
focused hop k=1: [('wifi-and-equipment-faq-000', 25.4)]

Explanation: with k=10 the combined query does bring in a wifi-and-equipment-faq chunk (position #7, as you saw in the worked example), but not the specific chunk that answers the complete question (-000, which only shows up at position #14) — not even doubling the tool's real MAX_K is enough to rescue it. The focused hop, by contrast, finds it with the smallest possible k: k=1 is already enough, because it doesn't have to compete with any cancellation-topic term. Raising k on a combined query exposes more noise, not necessarily the missing signal.

Exercise 3: Design a `multi_hop_search` function (Hard)

Write multi_hop_search(queries: list[str], k: int = 3) -> dict[str, list[dict]] that runs search_docs once per query in the list and returns a query -> results dictionary. Run it with the worked example's two queries (cancellation and wifi) and confirm the result is identical to calling search_docs twice separately.

See solution

def multi_hop_search(queries, k=3):
    """Runs one search_docs hop per independent query.
    Doesn't decide WHICH queries to use -- that's the model's (concept)
    reformulation/decomposition; this function only runs the hops already decided."""
    return {query: search_docs(query, k=k) for query in queries}


hops = multi_hop_search([
    "What is Reservo's cancellation policy?",
    "Does every room include high-speed wifi at no extra charge?",
])

for query, results in hops.items():
    print(f"--- {query!r} ---")
    for r in results:
        print(f"  {r['score']:<7} {r['doc_id']}")

# Confirm equivalence with standalone calls
single_1 = search_docs("What is Reservo's cancellation policy?", k=3)
single_2 = search_docs("Does every room include high-speed wifi at no extra charge?", k=3)
assert hops["What is Reservo's cancellation policy?"] == single_1
assert hops["Does every room include high-speed wifi at no extra charge?"] == single_2
print("\nmulti_hop_search matches the standalone calls")

Expected output:

--- "What is Reservo's cancellation policy?" ---
  10.734  cancellation-policy
  7.217   refund-policy
  6.364   no-show-policy
--- "Does every room include high-speed wifi at no extra charge?" ---
  25.4    wifi-and-equipment-faq
  5.717   operations-manual-raw
  3.652   focus-room-manual

multi_hop_search matches the standalone calls

Explanation: multi_hop_search needs no new retrieval logic at all — it's a thin layer that runs search_docs once per hop and organizes results by query, the same way search_multi in Module 2 organized results by evaluation query. The function doesn't decide which queries to use or how many hops are needed — that's the conceptual part, the compound question's decomposition, done by the model — it only runs, deterministically and reproducibly, the hops that were already decided.

Summary and next step

A question made up of two independent topics needs two separate searches, not a combined query — we measured, by running it, that the combined query buries wifi-and-equipment-faq-000 at position #14 of 57, while a focused hop brings it to the top-1 with a score more than four times higher.
The cause is arithmetic: BM25 adds up each query term's contribution, and a query with more words from one topic gives that topic's chunks more opportunities to add up than the other's.
multi_hop_search runs the hops already decided — the decision of which queries to split into (decomposing the compound question) is still the model's, conceptual, not this function's.
Boundary: this module retrieves the chunks for each hop; how those lists get combined in the context window is context-engineering-guide's job.

Next lesson: 05 — Combining search_docs with Reservo's tools. When one of a compound question's two halves isn't a second search hop, but a structured tool like get_quote, in the same turn.

Additional resources

agent-fundamentals-and-tool-calling-guide, Module 4, Lesson 05 (Chaining tools across several steps) — the general pattern of solving a question with several tool calls, applied here to two calls of the same tool with different queries.
context-engineering-guide — where it's decided how to combine chunks from several hops into the context window; the exact boundary of where this module ends.
Manning, Raghavan & Schütze — Introduction to Information Retrieval, ch. 9: "Relevance feedback and query expansion" — the formal literature behind splitting a compound information need into several queries.
Python — Dictionary comprehensions — the {query: search_docs(query, k=k) for query in queries} construct used in multi_hop_search.

← PreviousReformulating and Retrying the Query Next →Combining `search_docs` with Reservo's Tools

Multi-hop Retrieval

Description

Connection to the module

Analogy: two questions at the same desk, two trips to the archive

The problem, measured: a single combined query

The solution: two hops, each with its own query

Why this happens: BM25's arithmetic, in one sentence

Boundary: retrieving two chunk sets isn't the same as combining them in the prompt

Common mistakes

Exercises

Exercise 1: Predict before running it (Easy)

Exercise 2: Confirm even a generous k doesn't rescue the buried hop (Medium)

Exercise 3: Design a multi_hop_search function (Hard)

Summary and next step

Additional resources

Exercise 2: Confirm even a generous `k` doesn't rescue the buried hop (Medium)

Exercise 3: Design a `multi_hop_search` function (Hard)