Module 2: Chunking Strategies

Project: Chunking Strategy Optimizer

Next →

Project overview

This project pulls the whole module together: implement all 4 chunking strategies, compare them with quantitative benchmarks, select the optimal one for your use case, and integrate it into the Module 1 baseline RAG.

The goal: improve the baseline's precision by +10-15% with optimized chunking, document the decision with data, and end up with reusable production code.

🎯 Project objectives

  1. ✅ Implement the 4 chunking strategies (fixed, recursive, semantic, structural)
  2. ✅ Compare them with benchmarks (precision, recall, latency, coherence)
  3. ✅ Select the optimal strategy for each document type
  4. ✅ Integrate it into the baseline RAG (replacing fixed-size)
  5. ✅ Document the improvement (+10-15% precision expected)

📁 The project structure

rag_baseline_project/  (from Module 1)
├── src/
│   ├── chunking/                    # NEW
│   │   ├── __init__.py
│   │   ├── base_chunker.py          # The base interface
│   │   ├── fixed_size_chunker.py    # Strategy 1
│   │   ├── recursive_chunker.py     # Strategy 2
│   │   ├── semantic_chunker.py      # Strategy 3
│   │   ├── structural_chunker.py    # Strategy 4
│   │   └── chunking_comparator.py   # The benchmark tool
│   └── indexing.py                  # Update it with the optimized chunking
└── notebooks/
    └── chunking_comparison.ipynb    # An interactive demo

💻 Implementation

Step 1: The base chunker interface

# src/chunking/base_chunker.py
from abc import ABC, abstractmethod
from typing import List

class BaseChunker(ABC):
    """The base interface for chunking strategies"""
    
    @abstractmethod
    def chunk(self, text: str) -> List[str]:
        """Splits text into chunks"""
        pass
    
    @abstractmethod
    def name(self) -> str:
        """The strategy's name"""
        pass

Step 2: Implement the strategies

# src/chunking/recursive_chunker.py
from langchain.text_splitters import RecursiveCharacterTextSplitter
from .base_chunker import BaseChunker

class RecursiveChunker(BaseChunker):
    def __init__(self, chunk_size=500, chunk_overlap=50):
        self.splitter = RecursiveCharacterTextSplitter(
            chunk_size=chunk_size,
            chunk_overlap=chunk_overlap,
            separators=["\n\n", "\n", ". ", " ", ""]
        )
    
    def chunk(self, text: str) -> list[str]:
        return self.splitter.split_text(text)
    
    def name(self) -> str:
        return "recursive"

(Implement something similar for fixed_size, semantic and structural)

Step 3: The comparator

# src/chunking/chunking_comparator.py
import time
import statistics
from typing import Dict

class ChunkingComparator:
    """Compares chunking strategies with benchmarks"""
    
    def __init__(self, strategies: list):
        self.strategies = strategies
    
    def compare(self, document: str) -> Dict:
        """Compares every strategy"""
        results = {}
        
        for strategy in self.strategies:
            # Measure the latency
            start = time.time()
            chunks = strategy.chunk(document)
            latency = (time.time() - start) * 1000  # ms
            
            # Compute the metrics
            results[strategy.name()] = {
                "num_chunks": len(chunks),
                "avg_chunk_size": statistics.mean([len(c) for c in chunks]),
                "latency_ms": latency,
                "coherence": self._measure_coherence(chunks)
            }
        
        return results
    
    def _measure_coherence(self, chunks: list[str]) -> float:
        """Proxy: the % of chunks that end with punctuation"""
        ends_with_punct = sum(1 for c in chunks if c.rstrip()[-1] in '.!?')
        return ends_with_punct / len(chunks) if chunks else 0

Step 4: Run the benchmark

# benchmark_chunking.py
from src.chunking.fixed_size_chunker import FixedSizeChunker
from src.chunking.recursive_chunker import RecursiveChunker
from src.chunking.semantic_chunker import SemanticChunker
from src.chunking.chunking_comparator import ChunkingComparator

# Load the document
with open("data/fastapi_docs.txt", "r") as f:
    document = f.read()

# Configure the strategies
strategies = [
    FixedSizeChunker(chunk_size=500),
    RecursiveChunker(chunk_size=500, chunk_overlap=50),
    SemanticChunker(),  # Requires an OpenAI API key
]

# Compare them
comparator = ChunkingComparator(strategies)
results = comparator.compare(document)

# Show the results
for strategy_name, metrics in results.items():
    print(f"\n{strategy_name.upper()}:")
    print(f"  Chunks: {metrics['num_chunks']}")
    print(f"  Avg size: {metrics['avg_chunk_size']:.0f} chars")
    print(f"  Latency: {metrics['latency_ms']:.2f}ms")
    print(f"  Coherence: {metrics['coherence']:.2%}")

Expected output:

FIXED_SIZE:
  Chunks: 250
  Avg size: 500 chars
  Latency: 0.08ms
  Coherence: 45%

RECURSIVE:
  Chunks: 268
  Avg size: 467 chars
  Latency: 1.25ms
  Coherence: 89%

SEMANTIC:
  Chunks: 185
  Avg size: 676 chars
  Latency: 18500ms
  Coherence: 96%

Step 5: Integrate it into the baseline RAG

# src/indexing.py (updated)
from src.chunking.recursive_chunker import RecursiveChunker  # NEW

class OptimizedIndexingPipeline:
    def __init__(self, chunking_strategy="recursive"):
        # Select the chunking strategy
        if chunking_strategy == "recursive":
            self.chunker = RecursiveChunker(chunk_size=500, chunk_overlap=50)
        elif chunking_strategy == "semantic":
            self.chunker = SemanticChunker()
        # ... the others
        
        self.openai_client = OpenAI()
        self.chroma_client = chromadb.PersistentClient(path="./chroma_db")
    
    def chunk_document(self, document: str) -> list[str]:
        """Use the selected chunking strategy (not fixed-size)"""
        return self.chunker.chunk(document)
    
    # ... the rest of the code is unchanged

📊 Measuring the improvement

Before (the Module 1 baseline):

# Fixed-size chunking
baseline_results = {
    "precision@5": 0.68,
    "recall@50": 0.52,
    "coherence": 0.62
}

After (the Module 2 optimization):

# Recursive chunking
optimized_results = {
    "precision@5": 0.78,  # +10%
    "recall@50": 0.57,    # +5%
    "coherence": 0.89     # +27%
}

The improvement you achieved: ✅ +10% precision, +5% recall (target hit)

📝 Documenting the decision

The updated README.md:

# RAG System - Module 2: Optimized Chunking

## The Chunking Strategy Selected: Recursive

### The justification:
- **The gain:** +10% precision, +5% recall vs the fixed-size baseline
- **The cost:** zero (no extra API calls)
- **Latency:** +1.2ms (negligible)
- **Coherence:** 89% vs the baseline's 45% (+44%)

### The alternatives considered:
- **Semantic:** +15% precision but $12/10K docs and +16s latency → Ruled out on budget
- **Structural:** only applicable to code → Doesn't apply to a narrative dataset
- **Fixed-size:** a simple baseline but -10% precision → Rejected

### The configuration:
```python
RecursiveCharacterTextSplitter(
    chunk_size=500,
    chunk_overlap=50,  # 10% overlap
    separators=["\n\n", "\n", ". ", " ", ""]
)

Performance Comparison (Module 1 vs Module 2)

MetricModule 1 (Fixed)Module 2 (Recursive)Delta
Precision@568%78%+10%
Recall@5052%57%+5%
Coherence62%89%+27%

---

## 🎯 Success criteria

✅ **The project is complete if:**

1. All 4 strategies are implemented with working code
2. The comparison benchmark has been run and the results are documented
3. A strategy has been selected with a data-driven justification
4. The integration into the baseline RAG is complete
5. The +10-15% precision improvement over Module 1 has been achieved

---

## 🚀 Bonus (optional)

- An interactive notebook with comparison visualizations
- Unit tests for each chunking strategy
- A CLI for trying different strategies interactively
- Hybrid chunking (structural for code, recursive for text)

---

## 📚 Deliverables

1. **The code:** the complete src/chunking/ folder with all 4 strategies
2. **The benchmark results:** a documented comparison table
3. **The decision doc:** the justification for the strategy you selected
4. **The integration:** indexing.py updated with the optimized chunking
5. **The performance report:** the improvement over the baseline, quantified

---

## 🎯 Summary

**The Chunking Optimizer project:**

- ✅ Implement all 4 strategies (fixed, recursive, semantic, structural)
- ✅ Compare them with quantitative benchmarks
- ✅ Select the optimal strategy (typically: recursive)
- ✅ Improve the baseline by +10-15% precision
- ✅ Document the decision with data

**The expected improvement:** precision 68% → 78% (+10%)

**The next module:** Module 3 optimizes query processing (+15-25% recall with query expansion, rewriting, HyDE).

---

## 📚 Additional resources

1. **[LangChain Text Splitters Cookbook](https://python.langchain.com/docs/modules/data_connection/document_transformers/)** - Code examples
2. **[Chunking Strategies Evaluation](https://arxiv.org/abs/2307.03172)** - The research paper
3. **[RAG Chunking Best Practices](https://www.pinecone.io/learn/chunking-strategies/)** - Pinecone's guide
4. **[LlamaIndex Chunking](https://docs.llamaindex.ai/en/stable/module_guides/loading/node_parsers/)** - Alternative implementations

---

**Created:** February 6, 2026  
**Version:** 1.0
← PreviousCapsule 07: The decision framework — which chunking strategy to chooseNext →Introduction to Query Optimization