Bounding the Tool: `k` and Truncation

Description

search_docs already has a contract (Lesson 03), a return shape (Lesson 04), and a description that says when to use it (Lesson 05). What's missing is a piece that has no exact equivalent in get_quote or book_room: what happens if the model asks for k=500? What happens if a corpus chunk is 5,000 characters long? Neither question made sense with Reservo's tools — hours was just some integer, sure, but it was never going to generate a response thousands of words long. With a retrieval tool, it does matter, because both the number of results and each one's size directly determine how much text ends up competing for space in the agent's conversation.

This lesson builds the protection Lesson 03's Exercise 3 already anticipated: the input_schema can't (and shouldn't) declaratively express "k max 5" with this guide's basic tools — the same way MANAGE_BOOKING_SCHEMA in agent-fundamentals-and-tool-calling couldn't express conditional rules. The real protection lives in search_docs's code: a hard cap on k and a truncation of each chunk's text. You're going to run both and see, with real numbers, how much the response size changes.

Connection to the module

This lesson closes out the "build the tool" work before the synthesis: Lesson 07 runs the complete version — contract + return + description + bounded — against the corpus's six fixed queries, and Lesson 08 integrates it into a registry alongside Reservo's tools.

Analogy: the desk has a per-turn service limit

The internal-archive help desk, no matter how much goodwill it has, can't hand a visitor 500 complete documents at once, even if all 500 were technically "relevant" to some degree. A well-run desk has a clear policy: "we hand over at most 5 documents per inquiry, and each one comes as a reasonable-sized excerpt — if you need more detail on a specific one, ask for it separately." That policy isn't decided by the request form (the input_schema) — the form only says "write down how many you want here" — it's decided by the person behind the desk, applying an operational limit before handing anything over. search_docs needs exactly that same policy.

Why the risk is real, not hypothetical

Two different questions, two different risks:

Unbounded k. If the model asks for k=500 over a 57-chunk corpus, there aren't 500 relevant results to return — but with no cap, the function would return as many as the index has positive scores for, including chunks with near-zero relevance just because they share one common word with the query. More results isn't more precision: it's more noise the model has to process to find what actually matters.

Untruncated text. Every chunk in this corpus is between 103 and 290 characters — modest, for this guide, because Module 1's structure-aware chunking cuts by section, not by whole document. But a real production document can have chunks of several thousand characters, or sections that don't lend themselves to such fine-grained chunking. With no limit, search_docs could return, say, three chunks of 2,000 characters each for a single call — 6,000 characters competing with the rest of the conversation for the same context budget, a topic context-engineering-guide covers in depth (a boundary already flagged in this guide's introduction: here you retrieve, there you curate how it enters the prompt).

Worked example: MAX_K and text truncation

MAX_K = 5
MAX_CHARS = 280


def truncate(text, max_chars=MAX_CHARS):
    if len(text) <= max_chars:
        return text
    return text[:max_chars].rstrip() + "..."


def search_docs(query, k=3):
    """The complete tool: bounds k, runs the search, assembles the result
    with truncated text."""
    bounded_k = max(1, min(k, MAX_K))
    hits = INDEX.search(query, k=bounded_k)
    return [
        {
            "chunk_id": chunk.chunk_id,
            "doc_id": chunk.doc_id,
            "text": truncate(chunk.text),
            "score": score,
        }
        for chunk, score in hits
    ]

Two concrete decisions packed into those four lines around bounded_k: min(k, MAX_K) trims any request above the cap; max(1, ...) keeps a k=0 (or negative, if someone passed it by hand without going through the validator) from leaving the search executing nothing useful. Neither rule lives in the input_schema — they live in the function's body, exactly where they have to live based on what you already learned in agent-fundamentals-and-tool-calling Module 2, Lesson 06.

Let's try a request for k=8, well above the cap:

demo = search_docs("What is the cancellation policy for Boardroom bookings?", k=8)
print("k requested: 8, MAX_K:", MAX_K)
print("results returned:", len(demo))
for r in demo:
    print(f"  score={r['score']:<7} doc_id={r['doc_id']:<24} len(text)={len(r['text'])}")

What to expect:

k requested: 8, MAX_K: 5
results returned: 5
  score=10.658  doc_id=cancellation-policy      len(text)=170
  score=6.298   doc_id=cancellation-policy      len(text)=199
  score=6.062   doc_id=membership-tiers-faq     len(text)=191
  score=5.982   doc_id=refund-policy            len(text)=226
  score=5.315   doc_id=no-show-policy           len(text)=200

Even though 8 were requested, exactly 5 came back — MAX_K's cap applied without the input_schema having to reject the input as invalid (k=8 is still a perfectly valid integer per the schema; what's not valid is the function honoring it unbounded). None of the five is longer than MAX_CHARS=280, so truncate lets them all through unchanged — this top-5's longest chunk, refund-policy at 226 characters, is still well under the cap. Notice something more subtle: second and third place (cancellation-policy again, and membership-tiers-faq) aren't directly about Boardroom's cancellation policy — they share words like "cancellation," "booking," and "policy" with the query without either one being, on its own, the full answer. Neither truncation nor the k cap fixes this: it's exactly the same lexical pattern you're going to see, in more detail, in Lesson 07.

Worked example: truncation, before and after

from rag_index import CHUNKS

longest = next(c for c in CHUNKS if c.chunk_id == "operations-manual-raw-001")
print("original, len:", len(longest.text))
print(longest.text)
print()
print("truncated, len:", len(truncate(longest.text)))
print(truncate(longest.text))

What to expect:

original, len: 290
Rooms are cleaned between every booking when the gap is 30 minutes or longer. For back-to-back bookings under 30 minutes, cleaning is limited to wiping the table and checking for left-behind belongings. Boardroom receives a full clean every evening regardless of usage, because of its size.

truncated, len: 283
Rooms are cleaned between every booking when the gap is 30 minutes or longer. For back-to-back bookings under 30 minutes, cleaning is limited to wiping the table and checking for left-behind belongings. Boardroom receives a full clean every evening regardless of usage, because of...

operations-manual-raw-001 is, at 290 characters, the longest chunk in the whole canonical corpus — Module 1's structure-aware chunking cuts by section, so no chunk comes anywhere near the size of an entire document. Even so, with MAX_CHARS=280, truncate trims it to 283 (280 characters of content plus the three dots of "..."). Look at where the cut lands: right before the sentence "...because of its size." finishes, cutting off the explanation of why Boardroom gets a full clean every night, and leaving only the fact ("Boardroom receives a full clean every evening"). This isn't a truncation bug: it's its real, honest cost. Even with chunks as small as this corpus's, MAX_CHARS=280 can still cut before an idea finishes — a trade-off any fixed-size limit carries, and one worth saying plainly, not hiding.

The k trade-off, in both directions

k too small  -> the right chunk can fall outside the top-k, even though the
                index would have found it with a bigger k
k too large  -> marginally relevant results sneak in, and the model has
                to process more text to find what's useful

There's no "correct" k value in the abstract — it depends on how large the corpus is and how ambiguous a typical query tends to be. MAX_K=5 is a reasonable choice for a 57-chunk corpus over 13 documents: enough headroom that a query with more than one relevant chunk (as you saw in several of the outputs above, where the second or third result also made sense) doesn't come up short, without opening the door to returning almost the whole corpus. Module 6 (evaluation) gives you the real tool for deciding this with data — measured recall@k, not just intuition.

Common mistakes

1. Trying to express MAX_K inside the input_schema. Real JSON Schema does have "maximum" as a keyword, but this guide's minimal validator (reused from agent-fundamentals-and-tool-calling) doesn't implement it, and adding it without the validator checking it would be an empty promise in the schema. The real protection, with this guide's tools, lives in the code — the same principle you already saw with manage_booking's conditional rules.

2. Truncating the text before computing the score. Truncation has to be applied after INDEX.search has already computed scores over the complete text. If you truncated the text first and then indexed or searched over the trimmed version, you'd be impoverishing the search signal to save text at a step that doesn't need it.

3. Using min(k, MAX_K) without the max(1, ...). Without the second bound, a k=0 (which the input_schema doesn't explicitly forbid, since it has no "minimum": 1) would make INDEX.search always return an empty list, even when relevant results are available — a silent failure, with no error message, that would be hard to diagnose.

4. Treating the "..." as if it were real content. The truncation suffix is a signal to whoever reads the result (human or model) that the text was trimmed, not a real continuation of the document. A chunk shown without that marker, cut mid-sentence, could read as if the document actually ended there.

Exercises

Exercise 1: Compute the result without running it (Easy)

With MAX_K=5 and MAX_CHARS=280, what does search_docs("Is there wifi in the Lounge?", k=2) return in terms of number of results? What about search_docs("Is there wifi in the Lounge?", k=20)?

Exercise 2: Measure truncation on a short chunk (Medium)

boardroom-room-manual-004 (the "House Rules" section of the Boardroom manual) is 194 characters — shorter than MAX_CHARS=280. Run truncate on its text and confirm what happens when the original text is already shorter than the limit.

from rag_index import CHUNKS

boardroom_text = next(c for c in CHUNKS if c.chunk_id == "boardroom-room-manual-004").text
print("original len:", len(boardroom_text))
print("truncated len:", len(truncate(boardroom_text)))
print(truncate(boardroom_text) == boardroom_text)

original len: 194
truncated len: 194
True

Exercise 3: Design a different MAX_CHARS and justify the trade-off (Hard)

Suppose you change MAX_CHARS from 280 to 120. Without running anything yet, predict what would happen to most of this corpus's chunks (check their lengths: they range from 103 to 290 characters). Then run truncate with max_chars=120 on membership-tiers-faq-000 (191 characters) and confirm your prediction. What information gets lost with that more aggressive limit?

mt = next(c for c in CHUNKS if c.chunk_id == "membership-tiers-faq-000")
print(truncate(mt.text, max_chars=120))

Basic is the default tier for every new member, with no monthly fee. Pro is a paid upgrade that adds a shorter cancellat...

Summary and next step

• k and each chunk's text need a limit the input_schema can't express with this guide's tools — the same way manage_booking couldn't express conditional rules in agent-fundamentals-and-tool-calling. The protection lives in search_docs's code, not the schema.
• MAX_K=5 trims any request above the cap, executed: a k=8 request returns 5 results, without the input_schema rejecting the input as invalid.
• MAX_CHARS=280 truncates each chunk's text with a "..." suffix, executed on the corpus's longest chunk (operations-manual-raw-001, 290 characters, trimmed to 283) — and we saw, honestly, that the cut can land before a relevant idea finishes, even with chunks as short as this corpus's.
• Neither limit is "the right number" in the abstract: they're design decisions with measurable trade-offs, and Module 6 (evaluation) gives you the tools to tune them with data.

Next lesson: 07 — search_docs executed end to end. With the contract, the return, the description, and the limits all complete, we run the final tool against Reservo's corpus's six fixed queries, including the trap query.

Additional resources

1. agent-fundamentals-and-tool-calling-guide, Module 2, Lesson 06 (Designing bounded tools) — the same principle that certain rules live in the code, not the input_schema, applied here to size limits instead of conditional rules.
2. context-engineering-guide — where it gets decided which retrieved chunks fit in the context window and in what order; this lesson only decides what passes the tool's own filter.
3. JSON Schema — minimum/maximum — how real JSON Schema would express a declarative numeric limit, and why this guide's minimal validator doesn't implement it.
4. Python — string slicing — the text[:max_chars] operation truncate uses to trim the text.