Citing sources in the answer

Description

search_docs returns chunks with their doc_id, their chunk_id, their text, and their score — all the information needed for a final answer to say not just what it claims, but where it got it from. The problem this lesson solves is that nothing forces that information to make it all the way into the text a user actually reads. It's perfectly possible to build an answer saying "Boardroom cancellation is free up to 4 hours before under the pro tier" with that text carrying, nowhere, a way to verify which document that figure came from. Module 4 (Lesson 06) already built a checker — check_grounding — that confirms whether a cited number is backed by some observed tool_result. This lesson solves a problem that comes before that one: even if a number is backed somewhere in the history, does the answer tell the reader which document and which fragment back it up? You're going to see, executed, that the answer to that question isn't always as simple as "cite the doc_id" — there's a case, new to this guide, where the doc_id Module 6 marked as correct wins first place with a chunk that, read carefully, doesn't support the claim you'd expect it to cite.

Connection to the module

This is the module's first operational piece: before handling 0 results (Lesson 03) or filtering by threshold (Lesson 04), you need the most basic piece of all — that whatever does get cited, gets cited verifiably. The following lessons build on cite_source without repeating it.

Analogy: the book's call number, not just the book

A library desk that answers "yes, we have something on that" isn't the same as one that answers "shelf 4, cancellations folder, the note that says such and such". The first answer forces you to trust; the second lets you verify. But there's a nuance this lesson explores with a real case: sometimes the person at the desk brings the correct folder — the topic is right, no doubt — but opens the wrong page, one that only says "for more details, see the folder next to it" instead of containing the fact that was asked about. Bringing the correct folder isn't the same as bringing the page that answers. Citing the correct doc_id isn't the same as citing the chunk_id that supports the claim.

cite_source: the minimal piece

from search_docs_tool import search_docs


def cite_source(hit):
    """Formats a search_docs result's source as a verifiable citation:
    doc_id AND chunk_id, not just doc_id. chunk_id already includes doc_id
    as a prefix (e.g. "cancellation-policy-001"), but repeating doc_id
    separately keeps the citation readable without having to split the string."""
    return f"[{hit['doc_id']}:{hit['chunk_id']}]"

search_docs doesn't return a section field — that decision was fixed in Module 3, Lesson 04: the return shape is chunk_id, doc_id, text, score, nothing more — so cite_source works with what's actually available. That's enough for real traceability: chunk_id is unique across the whole corpus, and its three-digit position (-001, not -1) lets you pinpoint exactly which fragment of which document backs the claim, unambiguously.

Worked example: an answer with two citations

Picking up the pro-tier discount query, already familiar from Module 3 (Lesson 07):

def format_answer_with_citations(answer_text, hits):
    citations = ", ".join(cite_source(h) for h in hits)
    return f"{answer_text} (Sources: {citations})"


hits = search_docs("How much discount does the pro tier get?", k=2)
for h in hits:
    print(h["score"], h["doc_id"], h["chunk_id"])

answer = "Pro members receive a 20% discount on the hourly rate."
print(format_answer_with_citations(answer, hits))

What to expect:

9.435 cancellation-policy cancellation-policy-001
6.537 membership-tiers-faq membership-tiers-faq-001
Pro members receive a 20% discount on the hourly rate. (Sources: [cancellation-policy:cancellation-policy-001], [membership-tiers-faq:membership-tiers-faq-001])

The claim is correct — Reservo does give pro members a 20% discount — but look carefully at the first citation: cancellation-policy-001 isn't a legitimate citation for this fact. Its actual text, as you already saw in Module 3, is about the pro cancellation window ("cancellations up to 4 hours before... free of charge"); it mentions the 20% discount in passing, inside a sentence about a different topic. Citing it as the source of the discount isn't false in the sense that the number is wrong — the 20% is correct — but it's a misleading citation: someone going to verify "where does the 20% come from?" and opening cancellation-policy-001 would find text about cancellation deadlines, not about the discount amount. The second citation, membership-tiers-faq-001 — the section literally titled "How Much Discount Does the Pro Tier Get?" — does directly support the claim.

This is exactly the same pattern Module 3 (Lesson 02) and Module 4 (Lesson 03) already showed with scores: search_docs's top-1 doc_id/chunk_id isn't always the one that best supports the specific question being answered. What's new in this lesson is the consequence for citing: automatically formatting all of search_docs's results as sources for a claim, without reading whether each one actually supports it, produces citations that are technically traceable but substantially misleading.

A deeper case: when even the "correct" doc_id isn't enough

Module 6 measured that, of the six anchor queries, the only one that hits the expected doc_id in the top-1 is the Boardroom one: search_docs("What is the cancellation policy for Boardroom bookings?", k=1) returns cancellation-policy — the correct document, unambiguously at the document level. But "the correct document" and "the fragment that supports the answer" aren't the same question. Look at the full top-5:

hits = search_docs("What is the cancellation policy for Boardroom bookings?", k=5)
for h in hits:
    print(f"{h['score']:<7} {h['chunk_id']}")
    print(f"   {h['text']}")

What to expect:

10.658  cancellation-policy-003
   See `refund-policy` for what happens to the money once a cancellation is processed, and `no-show-policy` for what happens if you simply do not show up without cancelling.
6.298   cancellation-policy-002
   Cancellations go through the same booking system used to reserve the room. There is no phone line for cancellations; the system timestamp is what determines whether the cancellation was made in time.
6.062   membership-tiers-faq-000
   Basic is the default tier for every new member, with no monthly fee. Pro is a paid upgrade that adds a shorter cancellation window, see `cancellation-policy`, and a discount on every booking.
5.982   refund-policy-002
   Cancellations made outside the free window are not eligible; the booking amount is forfeited under `cancellation-policy`. No-shows are never refunded, regardless of membership tier; see `no-show-policy` for that separate case.
5.315   no-show-policy-000
   A no-show is a booking where the member never checks in during the reserved hours and never cancelled beforehand. This is different from a late cancellation, which is covered in `cancellation-policy`.

None of the first five results contains the actual data the question needs — how many hours before you can cancel without a charge. The top-1, cancellation-policy-003, is the same "Related Policies" section Module 3 (Lesson 07) already identified as a lexical magnet: a single pure cross-reference sentence naming two other policies. The other four results are variations on the same problem: each one mentions `cancellation-policy` between backticks or shares generic vocabulary ("cancellation", "booking") without ever saying how many hours ahead cancellation is required. Where's the chunk with the real data — "Basic 24 hours, Pro 4 hours"? Off the top-5, way off:

from rag_index import INDEX  # direct access, only for this inspection --
                              # search_docs in production never exposes k=57

all_hits = INDEX.search("What is the cancellation policy for Boardroom bookings?", k=57)
for rank, (chunk, score) in enumerate(all_hits, start=1):
    if chunk.doc_id == "cancellation-policy" and chunk.position in (0, 1):
        print(f"rank {rank} of {len(all_hits)}   score={score}   {chunk.chunk_id}   {chunk.section!r}")

What to expect:

rank 39 of 52   score=0.753   cancellation-policy-001   'Pro Tier Cancellation Window'
rank 48 of 52   score=0.195   cancellation-policy-000   'Basic Tier Cancellation Window'

The two chunks that genuinely contain the answer — "Pro members... cancellations up to 4 hours..." and "Basic members can cancel... up to 24 hours..." — land in ranks 39 and 48 of 52 results with a positive score. It's not that they're missing from the index: they're indexed, correctly, with their full text. It's that, for this specific phrasing of the question, they share almost no vocabulary with it — neither mentions "Boardroom" (the cancellation window is by membership tier, not by room) nor repeats the question's exact words — while five different chunks, each mentioning "cancellation" in passing, beat them by a wide margin. Citing cancellation-policy (the doc_id) as the source of "Boardroom's cancellation policy is X hours" would, strictly speaking, be citing the correct document — and yet, no chunk from that document appearing within a reasonable k (MAX_K=5, search_docs's real limit) contains the concrete fact being claimed.

claim_supported: confirm before citing

The lesson from the two cases above is the same: it's not enough to cite a search_docs result as a source — you need to confirm that specific result's text actually contains what's being claimed. A simple, runnable check, for the most common case (a claim depending on a key word or phrase):

def claim_supported(hit, required_terms):
    """True if ALL required_terms appear, as a substring, in the cited
    chunk's text -- a cheap, literal check, not a semantic one."""
    text_lower = hit["text"].lower()
    return all(term.lower() in text_lower for term in required_terms)


boardroom_hits = search_docs("What is the cancellation policy for Boardroom bookings?", k=3)
for h in boardroom_hits:
    supported = claim_supported(h, ["hours", "cancel"])
    print(f"{h['chunk_id']:28s} claim_supported(['hours','cancel'])={supported}")

What to expect:

cancellation-policy-003      claim_supported(['hours','cancel'])=False
cancellation-policy-002      claim_supported(['hours','cancel'])=False
membership-tiers-faq-000     claim_supported(['hours','cancel'])=False

The first three results fail the check: none contains the word "hours" in its text, even though all three talk about cancellation in general terms. claim_supported doesn't replace a human reading of the text — it's deliberately a literal substring comparison, not a semantic verifier — but it does exactly the job this lesson needs: a cheap, runnable signal that citing this particular chunk for this specific claim would be, at minimum, suspicious. Combined with cite_source, it gives the full operational rule: only cite a chunk_id as the source of a claim if claim_supported (or an equivalent read) confirms the text supports it — not just because search_docs returned it in the top-k.

Common mistakes

1. Citing search_docs's doc_id without checking the specific chunk_id. As the Boardroom case showed, a correct doc_id (confirmed by Module 6) can win the entire top-5 with chunks that don't contain the claim's concrete fact. Citing "according to cancellation-policy" without saying which chunk_id, especially when none of the ones that actually appeared support the figure, hides the problem instead of exposing it.

2. Confusing cite_source with check_grounding. check_grounding (Module 4, Lesson 06) confirms a number cited in the final text appears in some tool_result from the history — it doesn't care which specific document it came from, or whether that document actually covers the topic. cite_source solves the complementary problem: given a search_docs result that's actually going to be used, format its provenance verifiably. The two work together; neither substitutes for the other.

3. Automatically citing ALL of search_docs's results, without filtering which ones support the claim. The discount example showed exactly this mistake: format_answer_with_citations with both unreviewed results produces a citation that's technically traceable (cancellation-policy-001 does exist, is a real result) but substantially misleading for that particular claim.

4. Assuming claim_supported is a general proof that a chunk is correct. It's a literal word comparison, with the same honest limitations this entire guide documents about BM25: it doesn't understand synonyms or paraphrasing, and a chunk can contain the exact words without actually answering the question (for example, mentioning them in a sentence about a different topic). It's a cheap first line of defense, not a semantic verifier.

Exercises

Exercise 1: Format a simple citation (Easy)

Run search_docs("Is there wifi in the Lounge?", k=1) and use cite_source to build the citation string for the single result. Which chunk_id shows up, and does its text support the question about Lounge's wifi?

hits = search_docs("Is there wifi in the Lounge?", k=1)
print(cite_source(hits[0]))
print(hits[0]["text"])

[lounge-room-manual:lounge-room-manual-004]
Lounge is open-plan and does not require a keypad code, unlike Focus, Phonebooth, and Boardroom. Because there is no door, members should keep calls at conversational volume.

Exercise 2: Apply claim_supported to the discount query (Medium)

Using the discount query's two results from the worked example (cancellation-policy-001 and membership-tiers-faq-001), apply claim_supported with required_terms=["discount", "20%"] to each. Which one passes the check? Does it match which of the two you should cite according to the worked example's human read?

hits = search_docs("How much discount does the pro tier get?", k=2)
for h in hits:
    supported = claim_supported(h, ["discount", "20%"])
    print(f"{h['chunk_id']:28s} claim_supported=['discount','20%']: {supported}")

cancellation-policy-001      claim_supported=['discount','20%']: True
membership-tiers-faq-001     claim_supported=['discount','20%']: True

Exercise 3: Design a citer that prefers the most specific chunk (Hard)

Write a function best_citation(hits, required_terms) that, from a list of search_docs results, returns the first one (in score order) that passes claim_supported, or None if none pass. Test it with the Boardroom query, using required_terms=["hours", "free of charge"] (the combination that only shows up in the pro cancellation window's real text, not in any chunk that mentions "hours" in passing), with k=5 (search_docs's real production limit), and compare it against searching across the 52 results with a positive score directly through INDEX.search. What does the difference tell you about citing only with what the tool returns in production?

def best_citation(hits, required_terms):
    """Returns the first result (by score) that supports the claim,
    or None if none of the available results support it."""
    for h in hits:
        if claim_supported(h, required_terms):
            return h
    return None


terms = ["hours", "free of charge"]

hits_k5 = search_docs("What is the cancellation policy for Boardroom bookings?", k=5)
result_k5 = best_citation(hits_k5, terms)
print("with k=5 (search_docs's real limit):", result_k5)

all_hits = INDEX.search("What is the cancellation policy for Boardroom bookings?", k=57)
result_full = None
for chunk, score in all_hits:
    fake_hit = {"chunk_id": chunk.chunk_id, "doc_id": chunk.doc_id, "text": chunk.text, "score": score}
    if claim_supported(fake_hit, terms):
        result_full = fake_hit
        break
print("searching across the 52 results with a positive score:", result_full["chunk_id"] if result_full else None)

with k=5 (search_docs's real limit): None
searching across the 52 results with a positive score: cancellation-policy-001

Summary and next step

• cite_source(hit) formats a search_docs result's source at the chunk_id level (not just doc_id), the minimum granularity for a citation to be genuinely verifiable.
• The discount case showed that automatically citing every search result, without checking whether each one supports the specific claim, produces citations that are technically real but substantially misleading.
• The Boardroom case — the only query where Module 6 confirmed the correct doc_id in the top-1 — revealed something more uncomfortable: not a single one of the five results search_docs can return in production contains the exact hour figure; the chunks that do contain it land at ranks 39 and 48 of 52.
• claim_supported(hit, required_terms) gives a cheap, literal check to confirm before citing — complementary to check_grounding (Module 4), which verifies cited numbers against the full history, not a specific citation's quality.

Next lesson: 03 — Handling zero results. What the answer does when there's absolutely nothing to cite — and an executed finding about how often that really happens.

Additional resources

1. production-rag-and-document-ingestion-guide, Module 4, Lesson 06 (Grounding: anchoring the answer in the chunks) — check_grounding, the complementary checker that verifies cited numbers against the history, not a specific citation's quality.
2. production-rag-and-document-ingestion-guide, Module 6 (Evaluating retrieval quality) — the recall@k/precision@k measurement that confirms which of the six anchor queries hits the expected doc_id, the foundation of this lesson's Boardroom case.
3. Anthropic — Tool use (function calling) overview — the shape of the tool_result that carries chunk_id/doc_id back to the model, the information cite_source reformats for the final text.
4. Python — f-strings — the syntax used in cite_source to format the citation string.