Capsule overview

Recursive and semantic chunking treat text as a continuous sequence. They work well for narrative text. But when your content has formal structure — source code, HTML, markdown — that continuous sequence is an abstraction that breaks natural units.

Picture chunking Python code with chunk_size=500. The chunker cuts right in the middle of a function. The first half ends with if user.is_admin: dangling. The second half starts with indented code and no context. Neither chunk is interpretable on its own.

Structural chunking respects the language's syntax: it splits code by function and class, HTML by semantic section, markdown by heading. Every chunk is a complete logical unit that can be understood on its own — exactly what an LLM needs to answer well.

This capsule teaches you to implement structural chunking for the three most common formats in RAG (Python, HTML, Markdown), when to use it over recursive, and how to handle the pathological case: documents with malformed structure.

By the end of this capsule you'll be able to:

• ✅ Implement structural chunking for Python code with ast
• ✅ Implement structural chunking for HTML with BeautifulSoup
• ✅ Implement structural chunking for Markdown using headers as separators
• ✅ Design fallbacks for when the parser fails (syntactically invalid code, malformed HTML)
• ✅ Decide when structural beats recursive, with quantitative data
• ✅ Anticipate the traps: chunks that are too small (1-line functions) or too big (500-line classes)

Estimated time: 30-35 minutes

The insight: every format has natural units

Every content format has "units of meaning" that exist independently of size:

Python:   function  |   class   |   method
HTML:     <section> |  <article> | <div class="...">
Markdown:  #header  |  ##subheader |  ###subsubheader

Recursive chunking ignores these units — it cuts wherever the generic separators are (\n\n, . , ). The result is chunks that split natural units in half.

Structural chunking uses format-specific parsers that recognize those units:

Recursive with chunk_size=500 over Python code:

def calculate_total(items):
    """Compute the total."""
    if not items:
        return 0
    return sum(item.price for ite          ← the chunk cuts here
                                                                 ↓
m in items)                                ← the next chunk starts here
                                              (with no context)


Structural with AST:

# Chunk 1: the complete function
def calculate_total(items):
    """Compute the total."""
    if not items:
        return 0
    return sum(item.price for item in items)

The advantage: chunks a human (or an LLM) can actually understand. Each chunk answers a question like "what does this function do?" without needing any extra context.

Implementation: Python code with AST

# python_structural_chunking.py
import ast
from dataclasses import dataclass


@dataclass
class CodeChunk:
    type: str          # "function" | "class" | "module_top_level"
    name: str          # the function/class name
    content: str       # the chunk's source code
    line_start: int
    line_end: int
    docstring: str | None = None


def chunk_python_by_structure(code: str) -> list[CodeChunk]:
    """
    Splits Python code into structural chunks.
    Every function and class is an independent chunk.
    Top-level code (imports, constants) is grouped into its own chunk.
    """
    try:
        tree = ast.parse(code)
    except SyntaxError as e:
        # Fallback: if the code doesn't parse, use recursive
        print(f"AST parse failed: {e}. Falling back to recursive.")
        from langchain.text_splitter import RecursiveCharacterTextSplitter
        splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=80)
        return [CodeChunk(
            type="raw_fallback",
            name="unparseable",
            content=chunk,
            line_start=0,
            line_end=0,
        ) for chunk in splitter.split_text(code)]

    chunks = []
    code_lines = code.split('\n')

    # Top-level: imports, constants, direct statements
    top_level_lines = []
    handled_lines = set()

    for node in tree.body:
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
            content = ast.get_source_segment(code, node)
            docstring = ast.get_docstring(node)
            chunks.append(CodeChunk(
                type="function",
                name=node.name,
                content=content,
                line_start=node.lineno,
                line_end=node.end_lineno,
                docstring=docstring,
            ))
            handled_lines.update(range(node.lineno, node.end_lineno + 1))

        elif isinstance(node, ast.ClassDef):
            content = ast.get_source_segment(code, node)
            docstring = ast.get_docstring(node)
            chunks.append(CodeChunk(
                type="class",
                name=node.name,
                content=content,
                line_start=node.lineno,
                line_end=node.end_lineno,
                docstring=docstring,
            ))
            handled_lines.update(range(node.lineno, node.end_lineno + 1))

    # Top-level code (imports, constants) that is neither a function nor a class
    top_level_lines = [
        line for i, line in enumerate(code_lines, 1) if i not in handled_lines
    ]
    top_level_content = '\n'.join(top_level_lines).strip()
    if top_level_content:
        chunks.insert(0, CodeChunk(
            type="module_top_level",
            name="imports_and_constants",
            content=top_level_content,
            line_start=1,
            line_end=len(top_level_lines),
        ))

    return chunks


# Try it
python_code = '''
"""Module for user management."""
from datetime import datetime
import logging

logger = logging.getLogger(__name__)


def hash_password(password: str) -> str:
    """Hash a password using bcrypt."""
    from passlib.context import CryptContext
    pwd_context = CryptContext(schemes=["bcrypt"])
    return pwd_context.hash(password)


class User:
    """Represents a user in the system."""

    def __init__(self, name: str, email: str):
        self.name = name
        self.email = email
        self.created_at = datetime.utcnow()

    def to_dict(self) -> dict:
        return {
            "name": self.name,
            "email": self.email,
            "created_at": self.created_at.isoformat()
        }


def validate_email(email: str) -> bool:
    """Basic email validation."""
    return "@" in email and "." in email.split("@")[-1]
'''

chunks = chunk_python_by_structure(python_code)

for chunk in chunks:
    print(f"\n[{chunk.type}] {chunk.name} (lines {chunk.line_start}-{chunk.line_end})")
    if chunk.docstring:
        print(f"  Docstring: {chunk.docstring[:60]}")
    print(f"  Content:\n{chunk.content[:200]}...")

Output:

[module_top_level] imports_and_constants (lines 1-5)
  Content:
"""Module for user management."""
from datetime import datetime
import logging

logger = logging.getLogger(__name__)

[function] hash_password (lines 7-11)
  Docstring: Hash a password using bcrypt.
  Content:
def hash_password(password: str) -> str:
    """Hash a password using bcrypt."""
    from passlib.context import CryptContext...

[class] User (lines 14-26)
  Docstring: Represents a user in the system.
  Content:
class User:
    """Represents a user in the system."""

    def __init__(self, name: str, email: str):...

[function] validate_email (lines 29-31)
  Docstring: Basic email validation.
  Content:
def validate_email(email: str) -> bool:
    """Basic email validation."""
    return "@" in email and "." in email.split("@")[-1]

Every chunk is a coherent unit. A query like "how do I validate an email?" matches directly against the complete validate_email chunk.

Implementation: HTML with BeautifulSoup

# html_structural_chunking.py
from bs4 import BeautifulSoup
from dataclasses import dataclass


@dataclass
class HtmlChunk:
    tag_type: str   # section, article, div
    id: str | None
    heading: str | None
    text_content: str
    raw_html: str


def chunk_html_by_structure(html: str) -> list[HtmlChunk]:
    """
    Splits HTML by semantic element.
    Priority: section > article > div with a class.
    """
    soup = BeautifulSoup(html, 'html.parser')
    chunks = []

    # Look for semantic elements in priority order
    semantic_tags = ['section', 'article', 'main', 'nav']

    # If there are semantic tags, use them
    semantic_elements = []
    for tag in semantic_tags:
        semantic_elements.extend(soup.find_all(tag))

    if semantic_elements:
        for element in semantic_elements:
            heading_tag = element.find(['h1', 'h2', 'h3', 'h4', 'h5', 'h6'])
            heading = heading_tag.get_text(strip=True) if heading_tag else None

            chunks.append(HtmlChunk(
                tag_type=element.name,
                id=element.get('id'),
                heading=heading,
                text_content=element.get_text(separator='\n', strip=True),
                raw_html=str(element),
            ))
    else:
        # Fallback: use divs with a class
        for div in soup.find_all('div', class_=True):
            heading_tag = div.find(['h1', 'h2', 'h3'])
            heading = heading_tag.get_text(strip=True) if heading_tag else None

            chunks.append(HtmlChunk(
                tag_type='div',
                id=div.get('id'),
                heading=heading,
                text_content=div.get_text(separator='\n', strip=True),
                raw_html=str(div),
            ))

    # If there still aren't any chunks, fall back to recursive over the plain text
    if not chunks:
        from langchain.text_splitter import RecursiveCharacterTextSplitter
        splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=80)
        plain_text = soup.get_text(separator='\n', strip=True)
        for chunk_text in splitter.split_text(plain_text):
            chunks.append(HtmlChunk(
                tag_type='fallback',
                id=None,
                heading=None,
                text_content=chunk_text,
                raw_html='',
            ))

    return chunks

Implementation: Markdown with headers

# markdown_structural_chunking.py
import re
from dataclasses import dataclass


@dataclass
class MarkdownChunk:
    level: int      # 1=H1, 2=H2, 3=H3, ...
    header: str
    content: str    # includes the header + everything under it


def chunk_markdown_by_headers(markdown: str, max_level: int = 3) -> list[MarkdownChunk]:
    """
    Splits markdown by headers of level <= max_level.
    Each chunk is a header + all the content up to the next header of the same level or higher.
    """
    lines = markdown.split('\n')
    chunks = []
    current_chunk_lines = []
    current_header = "Top of document"
    current_level = 0

    header_pattern = re.compile(r'^(#{1,6})\s+(.+)$')

    for line in lines:
        match = header_pattern.match(line)

        if match and len(match.group(1)) <= max_level:
            level = len(match.group(1))
            header_text = match.group(2).strip()

            # If it's a header of level <= current_level, close the current chunk
            if current_chunk_lines and (current_level == 0 or level <= current_level):
                chunks.append(MarkdownChunk(
                    level=current_level,
                    header=current_header,
                    content='\n'.join(current_chunk_lines).strip(),
                ))
                current_chunk_lines = []

            current_header = header_text
            current_level = level
            current_chunk_lines.append(line)
        else:
            current_chunk_lines.append(line)

    # Add the last chunk
    if current_chunk_lines:
        chunks.append(MarkdownChunk(
            level=current_level,
            header=current_header,
            content='\n'.join(current_chunk_lines).strip(),
        ))

    return chunks


# Try it on technical markdown
markdown = """
# FastAPI User Guide

Welcome to the FastAPI documentation.

## Installation

Install FastAPI with pip:

```bash
pip install fastapi

Quick Start

Create your first FastAPI app:

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def read_root():
    return {"Hello": "World"}

Authentication

FastAPI supports OAuth2 out of the box.

OAuth2 Setup

Use OAuth2PasswordBearer from fastapi.security.

JWT Tokens

Sign tokens with python-jose. """

chunks = chunk_markdown_by_headers(markdown, max_level=2)

for chunk in chunks: print(f"\n[H{chunk.level}] {chunk.header}") print(f"Content ({len(chunk.content)} chars):") print(chunk.content[:200])

**Output:**

[H0] Top of document Content (52 chars):

FastAPI User Guide

Welcome to the FastAPI documentation.

[H1] FastAPI User Guide Content (44 chars):

FastAPI User Guide

Welcome to the FastAPI documentation.

[H2] Installation Content (78 chars):

Installation

Install FastAPI with pip:

pip install fastapi

[H2] Quick Start Content (180 chars):

Quick Start

Create your first FastAPI app:

from fastapi import FastAPI...

[H2] Authentication
Content (215 chars):
## Authentication

FastAPI supports OAuth2 out of the box.

### OAuth2 Setup
... (the H3 sub-headers are included inside)

Note: the H3s (OAuth2 Setup, JWT Tokens) ended up inside their parent H2's chunk because max_level=2. If you need finer chunks, raise max_level to 3 or 4.

Handling failures: when the parser can't cope

The most common error with structural chunking: the document arrives with broken syntax, malformed HTML, or non-standard markdown. The parser fails. Your pipeline needs a fallback.

The fallback pattern

def safe_structural_chunk(content: str, content_type: str) -> list[str]:
    """
    Tries structural chunking. If it fails, falls back to recursive.
    """
    try:
        if content_type == "python":
            chunks = chunk_python_by_structure(content)
            if not chunks:
                raise ValueError("No structural chunks found")
            return [c.content for c in chunks]

        elif content_type == "html":
            chunks = chunk_html_by_structure(content)
            if not chunks:
                raise ValueError("No structural chunks found")
            return [c.text_content for c in chunks]

        elif content_type == "markdown":
            chunks = chunk_markdown_by_headers(content)
            if not chunks:
                raise ValueError("No structural chunks found")
            return [c.content for c in chunks]

        else:
            raise ValueError(f"Unknown content_type: {content_type}")

    except Exception as e:
        print(f"Structural chunking failed for {content_type}: {e}")
        print("Falling back to recursive chunking")

        from langchain.text_splitter import RecursiveCharacterTextSplitter
        splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=80)
        return splitter.split_text(content)


# Logging the fallback rate
fallback_count = 0
total_count = 0

def chunk_with_logging(content, content_type):
    global fallback_count, total_count
    total_count += 1
    try:
        return safe_structural_chunk(content, content_type)
    except Exception:
        fallback_count += 1
        raise

# After processing 1000 docs
print(f"Fallback rate: {fallback_count}/{total_count} ({fallback_count/total_count:.0%})")

If the fallback rate goes above ~5%, something is wrong: an overly strict parser, or systematically malformed documents. It's worth investigating before you accept it.

When structural wins

Typical benchmarks over representative datasets:

How to read this:

• Structural wins dramatically on code.
• Markdown without headers has flat structure — recursive is equal or better.
• For narrative text, structural is worse, because it splits on arbitrary separators with no semantic coherence.

The simple rule: if your content has clear, formal syntactic units, use structural. If it doesn't, don't force structural — use recursive or semantic.

Traps and common mistakes

Trap 1: an overly strict parser that fails on valid but unusual syntax

The mistake: your Python parser rejects code with odd # noqa comments, Python 3.12+ type hints, or the walrus operator.

The symptom: a fallback rate >10%. Many documents end up chunked with recursive and lose the structural advantage.

How to prevent it: use an up-to-date parser (Python 3.11+'s ast is robust). If your corpus has version-specific syntax, validate against that version.

Trap 2: chunks that are too small

The mistake: lots of 1-2 line functions. Each one becomes a ~50-char chunk.

The symptom: retrieval returns individual chunks with no context. The LLM can't tell what a utility function does without seeing the rest of the module.

How to prevent it: merge the small chunks:

def merge_small_chunks(chunks: list, min_size: int = 200) -> list:
    """Joins consecutive small chunks until they reach min_size."""
    merged = []
    current = ""
    for chunk in chunks:
        if len(current) + len(chunk.content) < min_size * 2:
            current += "\n\n" + chunk.content
        else:
            if current:
                merged.append(current)
            current = chunk.content
    if current:
        merged.append(current)
    return merged

Trap 3: chunks that are too big

The mistake: a 500-line Python class becomes a single chunk.

The symptom: the chunk blows past the LLM's context window, or it dominates retrieval (it always ranks high just because of its size).

How to prevent it: sub-divide large classes by method:

def chunk_class_by_methods(class_node: ast.ClassDef, code: str) -> list[CodeChunk]:
    """If a class is large, split it into one chunk per method."""
    class_source = ast.get_source_segment(code, class_node)
    if len(class_source) < 1500:
        # A small class: one single chunk
        return [CodeChunk(type="class", name=class_node.name, content=class_source, ...)]

    # A large class: a header + one chunk per method
    chunks = []
    # The class header (signature + docstring + attributes)
    # ...
    # One chunk per method
    for method in class_node.body:
        if isinstance(method, ast.FunctionDef):
            method_source = ast.get_source_segment(code, method)
            chunks.append(CodeChunk(
                type="method",
                name=f"{class_node.name}.{method.name}",
                content=method_source,
                ...
            ))
    return chunks

Trap 4: HTML with lots of nested divs and no semantic structure

The mistake: legacy HTML with <div> everywhere, no <section> or <article>.

The symptom: chunk_html_by_structure finds no semantic elements. It falls back to recursive.

How to prevent it: detect this case and use a specific parser:

def chunk_html_by_divs_with_class(html: str):
    """For legacy HTML: use divs with a class as the units."""
    soup = BeautifulSoup(html, 'html.parser')
    return soup.find_all('div', class_=True)

Trap 5: throwing away valuable structural metadata

The mistake: you chunk code and save only the chunk's content. You lose the fact that it was a "function called validate_email".

The symptom: queries by function name ("what does validate_email do?") have low recall, because the exact name match never gets used.

How to prevent it: store the structural metadata on every chunk:

collection.add(
    documents=[chunk.content],
    metadatas=[{
        "chunk_type": chunk.type,         # "function", "class", "method"
        "name": chunk.name,                # "validate_email"
        "file_path": file_path,
        "line_start": chunk.line_start,
        "docstring": chunk.docstring or "",
    }],
    ids=[chunk_id],
)

# Later you can filter by metadata
results = collection.query(
    query_texts=["email validation"],
    where={"chunk_type": "function"},  # functions only
    n_results=5,
)

Trap 6: recursive as a fallback with no overlap

The mistake: when structural fails, the fallback uses recursive with chunk_overlap=0 "to keep it simple".

The symptom: the fallback chunks lose information at their boundaries. The quality becomes inconsistent: some docs (structural OK) are fine, others (fallback) are bad.

How to prevent it: fall back with a reasonable overlap:

splitter = RecursiveCharacterTextSplitter(
    chunk_size=800,
    chunk_overlap=80,  # 10% overlap
)

Applied exercise

The scenario: you're an AI Engineer at a company that indexes open-source code repositories to run RAG over code.

The data:

• 50K Python files from ~10K repositories
• The diversity: commercial libraries, scripts, tutorials, legacy code
• Typical queries: "how to implement X", "an example of Y", "a function to validate Z"

The metrics with recursive (chunk_size=800, overlap=80):

• Precision@5: 68%
• Recall@5: 55%
• The complaints: "the bot returns code fragments that don't compile, mixed together from several functions"

Your job:

1. Decide whether structural chunking applies.
2. Design the pipeline, including a robust fallback.
3. Estimate the impact and the cost of the change.

# code_chunking_pipeline.py
import ast
from langchain.text_splitter import RecursiveCharacterTextSplitter


def chunk_python_robust(code: str, file_path: str) -> list[dict]:
    """
    The complete chunking pipeline for Python.
    1. Try AST-based chunking
    2. Fall back to recursive if the AST fails
    3. Merge small chunks to avoid fragmentation
    4. Split large chunks (classes) to avoid domination
    """
    chunks = []

    try:
        tree = ast.parse(code)

        # Process the top-level definitions
        for node in tree.body:
            if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
                content = ast.get_source_segment(code, node)
                if content:
                    chunks.append({
                        "type": "function",
                        "name": node.name,
                        "content": content,
                        "metadata": {
                            "file_path": file_path,
                            "chunk_type": "function",
                            "function_name": node.name,
                            "line_start": node.lineno,
                            "line_end": node.end_lineno,
                            "docstring": ast.get_docstring(node) or "",
                        }
                    })

            elif isinstance(node, ast.ClassDef):
                class_content = ast.get_source_segment(code, node)
                if not class_content:
                    continue

                # If the class is large (>1500 chars), split it by method
                if len(class_content) > 1500:
                    for method in node.body:
                        if isinstance(method, (ast.FunctionDef, ast.AsyncFunctionDef)):
                            method_content = ast.get_source_segment(code, method)
                            if method_content:
                                chunks.append({
                                    "type": "method",
                                    "name": f"{node.name}.{method.name}",
                                    "content": method_content,
                                    "metadata": {
                                        "file_path": file_path,
                                        "chunk_type": "method",
                                        "class_name": node.name,
                                        "method_name": method.name,
                                        "line_start": method.lineno,
                                        "docstring": ast.get_docstring(method) or "",
                                    }
                                })
                else:
                    chunks.append({
                        "type": "class",
                        "name": node.name,
                        "content": class_content,
                        "metadata": {
                            "file_path": file_path,
                            "chunk_type": "class",
                            "class_name": node.name,
                            "line_start": node.lineno,
                            "line_end": node.end_lineno,
                            "docstring": ast.get_docstring(node) or "",
                        }
                    })

    except SyntaxError as e:
        # Fallback: recursive
        print(f"AST failed for {file_path}: {e}. Using recursive fallback.")
        splitter = RecursiveCharacterTextSplitter(
            chunk_size=800,
            chunk_overlap=80,
            separators=["\nclass ", "\ndef ", "\n\n", "\n", " "],
        )
        for i, chunk_text in enumerate(splitter.split_text(code)):
            chunks.append({
                "type": "fallback",
                "name": f"fragment_{i}",
                "content": chunk_text,
                "metadata": {
                    "file_path": file_path,
                    "chunk_type": "fallback",
                    "fragment_index": i,
                },
            })

    # Filter out chunks that are too small (probably noise)
    chunks = [c for c in chunks if len(c["content"]) >= 100]

    return chunks

Metric                Without structural  With structural   Change
─────────────────────────────────────────────────────────────────
Precision@5           68%                 86% (estimated)   +18 pts
Recall@5              55%                 72% (estimated)   +17 pts
"Fragments that       Frequent            Rare              ↓ 80%
 don't compile"

Summary and next step

What you learned:

• Structural chunking respects the natural syntactic units: functions/classes in code, sections in HTML, headers in markdown.
• The implementation: ast for Python, BeautifulSoup for HTML, regex for markdown.
• The typical gain: +15-25% precision on structured content vs recursive.
• A fallback is mandatory: parsers fail on unusual syntax, generated code, malformed HTML.
• Sub-dividing large chunks (classes >1500 chars → one chunk per method).
• Merging small chunks (1-2 line functions) to avoid fragmentation.
• Structural metadata (function name, class, file_path) is critical for advanced retrieval.
• Do NOT use structural for narrative text — recursive or semantic are better.

Checkpoint: before moving on, you should be able to:

• Implement structural chunking for Python code with ast and a robust fallback.
• Decide when structural beats recursive, with quantitative data.
• Design a pipeline that merges/splits chunks based on size.

Next capsule: 06 — Chunk overlap.

We've covered four chunking strategies. One technique complements all of them: chunk overlap. Already covered in M02/06. Capsule 07 (next) consolidates everything into a decision framework.

Resources

1. Python AST Documentation — The official ast documentation
2. BeautifulSoup Documentation — For HTML parsing
3. LangChain — Markdown Header Splitter — The official implementation
4. LlamaIndex — Code Splitter — An implementation with tree-sitter
5. tree-sitter — A robust multi-language parser (an alternative to ast)
6. GitHub Copilot — Code Chunking Strategy — How Copilot chunks code

Estimated time: 30-35 minutes Next: 06-chunk-overlap.md