Chunking Strategies: Fixed Size vs. Structure-Aware

Description

With all 13 documents parsed and clean (lessons 03 and 04), it's time to cut them into chunks: pieces of text small enough that a search system can return just the relevant part, and large enough that part makes sense on its own. How you cut isn't a detail — it's a design decision with measurable consequences, and this lesson shows you three different ways to make it, executed on the same document, so you see with your own eyes where each one wins and where each one fails. None of the three is "the correct one" in the abstract: each one assumes something different about what you know about the document, and this lesson gives you all three so you can choose deliberately, not out of habit.

Connection to the module

Lessons 03 and 04 answered "what text do I have?" This lesson answers "what pieces do I cut it into?" — and it's the first time in the module where the result depends on a decision you make, not just on running a parser. Lesson 06 takes exactly these three strategies and studies what happens when you change the chunk size, with executed evidence of the trade-off. Only after that does lesson 07 attach the metadata that turns these text fragments into something citable.

Analogy: three ways to cut the same orange

You have an orange and want to divide it up. You could cut it into even cubes with a knife, without looking at the natural segments — every cube comes out the same size, but some cubes will have a whole segment inside and others will have two halves of different segments, mixed together. That's fixed-size chunking: predictable, easy to code, but indifferent to the real structure of what you're cutting.

Or you could separate it by its natural segments — each piece is exactly one segment, no more, no less, respecting the way the fruit was already divided. That's structure-aware chunking: each chunk is a complete section of the document (or a piece of a section, if the section is huge), never a mix of two different sections.

Or you could decide the only thing that matters is never splitting a segment in half — you group whole segments together until you reach a reasonable size, without worrying whether that crosses the "natural" boundary of a flavor change. That's sentence-based chunking: it never cuts in the middle of an idea, but it also doesn't necessarily respect where the document changed topic.

All three divide up the same orange. Which one to use depends on what matters most to you: predictable size, respecting structure, or sentence integrity. Let's build all three.

Strategy 1: fixed size with overlap

The simplest one: a window of size characters that advances size - overlap characters each time. The overlap exists so that an idea that got cut right at a chunk's edge still shows up complete in the next chunk.

def chunk_fixed(text: str, size: int = 400, overlap: int = 50) -> list[str]:
    """Fixed-size window, in characters, with overlap."""
    if overlap >= size:
        raise ValueError("overlap must be smaller than size, or the window never advances")
    text = text.strip()
    if not text:
        return []
    step = size - overlap
    chunks = []
    start = 0
    while start < len(text):
        end = start + size
        chunks.append(text[start:end].strip())
        if end >= len(text):
            break
        start += step
    return [c for c in chunks if c]

To see the overlap mechanism without the noise of a real document, let's run it on an alphabet:

text = "ABCDEFGHIJKLMNOPQRST"
for i, c in enumerate(chunk_fixed(text, size=10, overlap=3)):
    print(i, repr(c))

What to expect:

0 'ABCDEFGHIJ'
1 'HIJKLMNOPQ'
2 'OPQRST'

Chunk 0 takes characters 0-10 (ABCDEFGHIJ). Chunk 1 doesn't start at 10 — it starts at 10 - 3 = 7 (step = size - overlap = 10 - 3 = 7), so it begins at H, and the three letters HIJ end up repeated between chunk 0 and chunk 1. That repetition is the overlap: if an important sentence had crossed right at the boundary between chunk 0 and chunk 1, with overlap it gets a second chance to show up complete in one of the two.

Strategy 2: by sentence, never cutting one in half

Instead of blindly counting characters, this strategy recognizes the end of a sentence and never cuts inside one. It groups complete sentences together until it gets close to max_size, and only then starts a new chunk:

import re

SENTENCE_RE = re.compile(r"(?<=[.!?])\s+(?=[A-Z(`])")


def split_sentences(text: str) -> list[str]:
    return [s.strip() for s in SENTENCE_RE.split(text) if s.strip()]


def chunk_by_sentence(text: str, max_size: int = 400) -> list[str]:
    """Packs complete sentences into chunks of up to max_size characters."""
    sentences = split_sentences(text)
    chunks: list[str] = []
    current: list[str] = []
    current_len = 0
    for sent in sentences:
        extra = len(sent) + (1 if current else 0)
        if current and current_len + extra > max_size:
            chunks.append(" ".join(current))
            current, current_len = [], 0
        current.append(sent)
        current_len += len(sent) + (1 if len(current) > 1 else 0)
    if current:
        chunks.append(" ".join(current))
    return chunks

SENTENCE_RE is a lookbehind ((?<=[.!?])) that requires a ., !, or ? right before the split, followed by whitespace, followed by a lookahead ((?=[A-Z(\])) that requires the next sentence to start with a capital letter, a parenthesis, or a backtick (so it doesn't cut in the middle of an identifier between backticks like `` refund-policy` ``). Neither lookaround consumes characters — they only check that they're there, so the period and the space don't disappear from the text.

Strategy 3: structure-aware, with sentence sub-chunking

This strategy starts from the structure we already extracted in lessons 03 and 04 ([(section, text), ...]) and respects its boundaries: every section is, by default, its own chunk. Only if an individual section exceeds max_size does it get sub-cut by sentence — it never mixes two sections into the same chunk, and it never cuts a sentence in half:

def chunk_by_structure(
    sections: list[tuple[str, str]], max_size: int = 400
) -> list[tuple[str, str]]:
    """Respects section boundaries; only sub-cuts a section if it exceeds max_size."""
    out: list[tuple[str, str]] = []
    for heading, body in sections:
        if len(body) <= max_size:
            out.append((heading, body))
        else:
            for piece in chunk_by_sentence(body, max_size=max_size):
                out.append((heading, piece))
    return out

All three, side by side, on the same document

Let's run all three strategies on cancellation-policy.md, with the same max_size=200 so the comparison is fair:

raw = Path("corpus/cancellation-policy.md").read_text()
title, sections = parse_markdown(raw)
full_text = " ".join(body for _, body in sections)   # plain text, no structure
print("len(full_text) =", len(full_text))

What to expect:

len(full_text) = 761

print("=== chunk_fixed(size=200, overlap=30) ===")
for i, c in enumerate(chunk_fixed(full_text, size=200, overlap=30)):
    print(f"[{i}] ({len(c)} chars) {c!r}")

What to expect:

=== chunk_fixed(size=200, overlap=30) ===
[0] (200 chars) 'Basic members can cancel a booking up to 24 hours before the reserved start time with no penalty. Cancellations made less than 24 hours in advance forfeit the full booking amount. Pro members get a sh'
[1] (200 chars) 'g amount. Pro members get a shorter, friendlier window: cancellations up to 4 hours before the reserved start time are free of charge. This is one of the perks of the pro tier, alongside the 20% disco'
[2] (199 chars) 'tier, alongside the 20% discount on hourly rates. Cancellations go through the same booking system used to reserve the room. There is no phone line for cancellations; the system timestamp is what det'
[3] (200 chars) 'e system timestamp is what determines whether the cancellation was made in time. See `refund-policy` for what happens to the money once a cancellation is processed, and `no-show-policy` for what happe'
[4] (81 chars) 'no-show-policy` for what happens if you simply do not show up without cancelling.'

Look at chunk 0: it ends at "...get a sh" — it cut the word "shorter" in half. chunk_fixed doesn't know anything about words or sentences; it counts characters, period.

print("=== chunk_by_sentence(max_size=200) ===")
for i, c in enumerate(chunk_by_sentence(full_text, max_size=200)):
    print(f"[{i}] ({len(c)} chars) {c!r}")

What to expect:

=== chunk_by_sentence(max_size=200) ===
[0] (179 chars) 'Basic members can cancel a booking up to 24 hours before the reserved start time with no penalty. Cancellations made less than 24 hours in advance forfeit the full booking amount.'
[1] (124 chars) 'Pro members get a shorter, friendlier window: cancellations up to 4 hours before the reserved start time are free of charge.'
[2] (160 chars) 'This is one of the perks of the pro tier, alongside the 20% discount on hourly rates. Cancellations go through the same booking system used to reserve the room.'
[3] (124 chars) 'There is no phone line for cancellations; the system timestamp is what determines whether the cancellation was made in time.'
[4] (170 chars) 'See `refund-policy` for what happens to the money once a cancellation is processed, and `no-show-policy` for what happens if you simply do not show up without cancelling.'

No word got cut — a real improvement over chunk_fixed. But look at chunk 2: it mixes "This is one of the perks of the pro tier..." (which belongs to the Pro Tier Cancellation Window section) with "Cancellations go through the same booking system..." (which belongs to the How to Cancel section, a different topic). chunk_by_sentence respects the sentence, but it doesn't know anything about the document's sections — because we never gave them to it.

print("=== chunk_by_structure(sections, max_size=200) ===")
for i, (h, c) in enumerate(chunk_by_structure(sections, max_size=200)):
    print(f"[{i}] section={h!r} ({len(c)} chars) {c!r}")

What to expect:

=== chunk_by_structure(sections, max_size=200) ===
[0] section='Basic Tier Cancellation Window' (179 chars) 'Basic members can cancel a booking up to 24 hours before the reserved start time with no penalty. Cancellations made less than 24 hours in advance forfeit the full booking amount.'
[1] section='Pro Tier Cancellation Window' (124 chars) 'Pro members get a shorter, friendlier window: cancellations up to 4 hours before the reserved start time are free of charge.'
[2] section='Pro Tier Cancellation Window' (85 chars) 'This is one of the perks of the pro tier, alongside the 20% discount on hourly rates.'
[3] section='How to Cancel' (199 chars) 'Cancellations go through the same booking system used to reserve the room. There is no phone line for cancellations; the system timestamp is what determines whether the cancellation was made in time.'
[4] section='Related Policies' (170 chars) 'See `refund-policy` for what happens to the money once a cancellation is processed, and `no-show-policy` for what happens if you simply do not show up without cancelling.'

There's the difference, measurable: where chunk_by_sentence produced a chunk 2 that mixed Pro Tier with How to Cancel, chunk_by_structure produced two separate chunks — [2] and [3] — each with its correct section, without mixing topics. The cost is that there are now 5 chunks instead of 4, and some are smaller than max_size (chunk [2] is just 85 characters) — chunk_by_structure prioritizes not mixing sections over filling each chunk to the max.

                    chunk_fixed         chunk_by_sentence    chunk_by_structure
words cut           YES ("...get a sh")  no                   no
mixes sections       YES (indifferent)    YES (chunk 2: 2 sec.) NO (never)
chunks produced       5                    5                    5 (but different)
knows structure       no                   no                   yes (given to it)

Common mistakes

1. Cutting characters without looking at words. You already saw it: chunk_fixed with size=200 split "shorter" into "sh" + "orter," divided across two different chunks. A search system that indexes by word (like Module 2's lexical BM25) won't even recognize "sh" or "orter" as the word "shorter" — the cut destroys the signal, not just the aesthetics.

2. overlap >= size hangs the function in an infinite loop. step = size - overlap has to be positive for the window to advance on each pass; if overlap is equal to or greater than size, step is zero or negative, and start never advances. This lesson's function prevents that with an explicit validation:

   try:
       chunk_fixed("any test text here", size=50, overlap=50)
   except ValueError as e:
       print("ValueError:", e)

   Real output:

   ValueError: overlap must be smaller than size, or the window never advances
   

   Without that validation, the same call would hang the process forever instead of failing fast with a clear message — the difference between a bug you catch on the spot and one you discover in production when the ingestion pipeline never finishes.

3. SENTENCE_RE doesn't recognize abbreviations. The lookbehind (?<=[.!?]) doesn't know that "Dr." is an abbreviation, not the end of a sentence:

   text = "Contact the front desk, e.g. by phone. Dr. Smith runs the front desk on Mondays."
   for s in split_sentences(text):
       print(repr(s))

   Real output:

   'Contact the front desk, e.g. by phone.'
   'Dr.'
   'Smith runs the front desk on Mondays.'
   

   "Dr." ended up as its own two-character "sentence." None of Reservo's 13 corpus documents uses abbreviations with a period (we made sure of that on purpose when writing them), so this doesn't affect the real pipeline — but if your real corpus has "Dr.," "Mr.," "etc.," or "e.g.," this simple regex will cut them wrong. Solving it properly calls for a known-exceptions list or a sentence-segmentation library — out of scope for a stdlib regex.

4. Choosing chunk_by_structure and expecting uniform chunk sizes. You won't get them: chunk [2] in the example above is 85 characters, well below max_size=200. This strategy optimizes for not mixing topics, not for evenness — if you need predictably sized chunks for some infrastructure reason, chunk_fixed is the right tool, at the cost of cutting words and mixing topics.

Exercises

Exercise 1: Calculate the step by hand (Easy)

Without running code, calculate how many chunks chunk_fixed("0123456789", size=4, overlap=1) produces and what they are. Then confirm by running it.

start=0 -> text[0:4] = '0123'
start=3 -> text[3:7] = '3456'
start=6 -> text[6:10] = '6789'
start=9 -> text[9:13] = '9' (the text ends at index 10, so only '9' is left)

for i, c in enumerate(chunk_fixed("0123456789", size=4, overlap=1)):
    print(i, repr(c))

0 '0123'
1 '3456'
2 '6789'
3 '9'

Exercise 2: Compare all three strategies on another document (Medium)

Repeat the comparison of the three strategies (chunk_fixed, chunk_by_sentence, chunk_by_structure) on no-show-policy.md, with max_size=180. Does chunk_by_sentence end up mixing any section, like it did with cancellation-policy.md? Identify which one.

raw = Path("corpus/no-show-policy.md").read_text()
title, sections = parse_markdown(raw)
full_text = " ".join(body for _, body in sections)

print("=== chunk_by_sentence(max_size=180) ===")
for i, c in enumerate(chunk_by_sentence(full_text, max_size=180)):
    print(f"[{i}] {c!r}")

print()
print("=== chunk_by_structure(sections, max_size=180) ===")
for i, (h, c) in enumerate(chunk_by_structure(sections, max_size=180)):
    print(f"[{i}] section={h!r} {c!r}")

=== chunk_by_sentence(max_size=180) ===
[0] 'A no-show is a booking where the member never checks in during the reserved hours and never cancelled beforehand.'
[1] 'This is different from a late cancellation, which is covered in `cancellation-policy`. No-shows are charged the full amount of the booking.'
[2] 'Unlike a late cancellation, there is no partial leniency: the no-show fee equals the entire reserved price, and it is never refunded under `refund-policy`.'
[3] 'Members with three or more no-shows in a rolling 30-day window lose the ability to book same-day reservations; all future bookings must be made at least 24 hours in advance until the pattern clears.'
[4] 'Rooms held for a no-show cannot be re-offered to another member during that window, so the fee reflects real lost capacity, not a punitive charge.'

=== chunk_by_structure(sections, max_size=180) ===
[0] section='What Counts as a No-Show' 'A no-show is a booking where the member never checks in during the reserved hours and never cancelled beforehand.'
[1] section='What Counts as a No-Show' 'This is different from a late cancellation, which is covered in `cancellation-policy`.'
[2] section='What Happens on a No-Show' 'No-shows are charged the full amount of the booking.'
[3] section='What Happens on a No-Show' 'Unlike a late cancellation, there is no partial leniency: the no-show fee equals the entire reserved price, and it is never refunded under `refund-policy`.'
[4] section='Repeated No-Shows' 'Members with three or more no-shows in a rolling 30-day window lose the ability to book same-day reservations; all future bookings must be made at least 24 hours in advance until the pattern clears.'
[5] section='Why This Policy Exists' 'Rooms held for a no-show cannot be re-offered to another member during that window, so the fee reflects real lost capacity, not a punitive charge.'

Exercise 3: When does fixed size make sense over structure? (Hard)

A colleague tells you: "chunk_by_structure is strictly better than chunk_fixed, so there's no reason to ever use fixed size." Give them a concrete example of a document (real or hypothetical) where chunk_by_structure would work poorly or couldn't be applied, and chunk_fixed would be the reasonable choice.

Summary and next step

• We built three chunking strategies: fixed size with overlap (chunk_fixed), by sentence (chunk_by_sentence), and structure-aware (chunk_by_structure, which reuses chunk_by_sentence to sub-cut sections that exceed max_size).
• We ran all three on the same document (cancellation-policy.md, max_size=200) and measured the difference: chunk_fixed cut a word in half ("sh" + "orter"); chunk_by_sentence didn't cut any words but mixed two different sections into one chunk; chunk_by_structure neither cut words NOR mixed sections, at the cost of producing unevenly sized chunks.
• No strategy is universally correct: chunk_fixed is the simplest and fastest when there's no structure (or it doesn't matter); chunk_by_structure best preserves meaning when structure exists and does matter.
• Each one's limits are real and got demonstrated: overlap >= size hangs chunk_fixed (which is why it carries a validation), and SENTENCE_RE mishandles abbreviations with a period.

Next lesson: 06 — The chunk-size trade-off. With the strategies already built, we study what happens when you change max_size: what you gain and lose by shrinking or growing a chunk, with executed evidence on the same document at three different sizes.

Additional resources

1. Python — re, lookahead and lookbehind — The (?<=...) and (?=...) syntax that SENTENCE_RE uses to detect the end of a sentence without consuming the period or the space.
2. Python — Sequence slicing — The exact mechanics of text[start:end] used by chunk_fixed, including what happens when end exceeds the text's length.
3. Anthropic — Building effective agents — The principle of choosing the simplest solution that solves the problem; the same criterion for choosing among this lesson's three strategies.
4. advanced-rag-techniques-guide — Semantic chunking (with a model, not pattern rules) for when the budget and use case justify it; out of scope for this guide on purpose.