Module 1: The Complete RAG Pipeline (Architecture Overview)
Architecture Overview: Modules 2-8
Capsule overview
This guide has 8 modules. Module 1 (this one) gave you the baseline: the complete RAG pipeline, architecture decisions, metrics, real-world cases. Modules 2-8 teach you the advanced techniques that improve each component of that pipeline.
This capsule shows you the full roadmap: what you'll learn in each module, how they connect to each other, what performance gains to expect, and how the evolving project integrates every technique into a production-ready RAG system.
Without this overview, modules 2-8 would feel disconnected. With it, you'll understand that chunking (M2) optimizes Indexing, re-ranking (M4) optimizes Retrieval, metadata filtering (M6) optimizes the search space, and evaluation (M8) measures the whole system so you can spot gaps and apply improvements iteratively.
🗺️ The guide's roadmap
High-level view:
Module 1: The Complete RAG Pipeline (Baseline)
↓
Phase 1: Advanced Fundamentals
├─ Module 2: Chunking Strategies
├─ Module 3: Query Optimization
└─ Module 4: Re-ranking Techniques
↓
Phase 2: Retrieval Techniques
├─ Module 5: Hybrid Search (BM25 + Embeddings)
└─ Module 6: Metadata Filtering
↓
Phase 3: Production & Integration
├─ Module 7: Production Vector DBs (Pinecone)
└─ Module 8: RAG Evaluation & Testing (RAGAS)
📐 The complete RAG architecture
The baseline pipeline (Module 1):
┌─────────────────────────────────────────┐
│ INDEXING (Offline) │
├─────────────────────────────────────────┤
│ 1. Chunking: Fixed-size (naive) │
│ 2. Embeddings: OpenAI ada-002 │
│ 3. Storage: ChromaDB (local) │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ RETRIEVAL (Query time) │
├─────────────────────────────────────────┤
│ 1. Query: Direct (no optimization) │
│ 2. Search: Semantic (embeddings) │
│ 3. Ranking: Cosine similarity │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ GENERATION (Query time) │
├─────────────────────────────────────────┤
│ 1. Context: Top-K docs │
│ 2. LLM: GPT-3.5-turbo │
│ 3. Response: Text │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ EVALUATION (Manual) │
├─────────────────────────────────────────┤
│ 1. Metrics: Latency, Precision (manual) │
│ 2. Testing: Ad-hoc queries │
└─────────────────────────────────────────┘
Baseline performance:
- Latency P95: 800ms
- Precision@5: 68%
- Recall@50: 52%
- Faithfulness: N/A (no framework)
The advanced pipeline (Modules 2-8):
┌─────────────────────────────────────────┐
│ INDEXING (Offline) │
├─────────────────────────────────────────┤
│ 1. Chunking: Recursive + Semantic (M2) │ ← +15% precision
│ 2. Embeddings: OpenAI ada-002 │
│ 3. Storage: Pinecone (production) (M7) │ ← Scalability
│ 4. Metadata: date, type, source (M6) │ ← Filtering
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ RETRIEVAL (Query time) │
├─────────────────────────────────────────┤
│ 1. Query Optimization: Expansion (M3) │ ← +25% recall
│ 2. Hybrid Search: BM25 + Semantic (M5) │ ← +18% precision
│ 3. Metadata Filter: Pre-filtering (M6) │ ← -90% search space
│ 4. Re-ranking: Cross-encoder (M4) │ ← +20% precision
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ GENERATION (Query time) │
├─────────────────────────────────────────┤
│ 1. Context: Top-K re-ranked docs │
│ 2. LLM: GPT-3.5-turbo │
│ 3. Response: Text + Sources │
└─────────────────────────────────────────┘
↓
┌─────────────────────────────────────────┐
│ EVALUATION (Automated) (M8) │
├─────────────────────────────────────────┤
│ 1. RAGAS: faithfulness, relevancy │ ← Automated metrics
│ 2. Golden Dataset: 50 test queries │ ← Regression testing
│ 3. CI/CD: Auto-eval on every deploy │ ← Continuous quality
└─────────────────────────────────────────┘
Advanced performance (the Module 8 target):
- Latency P95: 1,200ms (+400ms from the optimizations)
- Precision@5: 88% (+20% vs baseline)
- Recall@50: 77% (+25% vs baseline)
- Faithfulness: 0.92 (automated with RAGAS)
The trade-off we accept: +400ms of latency for +20-25% accuracy (worth it in most cases).
📦 Module 2: Chunking Strategies
The goal:
Optimize how you split long documents into chunks to maximize retrieval quality.
The baseline's problem:
# Fixed-size chunking (naive)
chunks = [document[i:i+500] for i in range(0, len(document), 500)]
# Problem 1: it cuts in the middle of a sentence
# "FastAPI is a web framework. It was crea..." ← Sentence cut off
# Problem 2: it loses semantic context
# Chunk 1: "Python is..."
# Chunk 2: "...an interpreted language"
# The problem: "Python" and "language" ended up in different chunksThe techniques you'll learn:
1. Recursive chunking (LangChain):
from langchain.text_splitter import RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter(
chunk_size=500,
chunk_overlap=50, # The overlap preserves context
separators=["\n\n", "\n", ". ", " ", ""]
)
chunks = splitter.split_text(document)
# The benefit: it respects paragraphs and sentencesExpected improvement: +10-15% precision (more coherent chunks)
2. Semantic chunking (embeddings-based):
from langchain.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings
splitter = SemanticChunker(OpenAIEmbeddings())
chunks = splitter.split_text(document)
# The benefit: it splits by topic (semantic coherence)Expected improvement: +15-20% precision (best quality)
Trade-off: +embedding cost, +indexing latency
3. Structural chunking (code, HTML, Markdown):
# For code: chunk by function
import ast
def chunk_by_functions(python_code):
tree = ast.parse(python_code)
return [ast.get_source_segment(python_code, node)
for node in ast.walk(tree)
if isinstance(node, ast.FunctionDef)]
# For HTML: chunk by section
from bs4 import BeautifulSoup
def chunk_by_sections(html):
soup = BeautifulSoup(html, 'html.parser')
return [section.get_text() for section in soup.find_all('section')]Expected improvement: +20-25% precision for code/HTML (it respects the structure)
The Module 2 project:
Implement 3 chunking strategies in the baseline RAG and compare their performance:
# Chunking comparison
baseline_fixed = benchmark_rag(chunking="fixed")
# Precision@5: 68%
recursive = benchmark_rag(chunking="recursive")
# Precision@5: 78% (+10%)
semantic = benchmark_rag(chunking="semantic")
# Precision@5: 83% (+15%)🔍 Module 3: Query Optimization
The goal:
Optimize the user's query before searching, to raise both recall and precision.
The baseline's problem:
# A direct query, no optimization
user_query = "What is FastAPI?"
# The problem: the query can be ambiguous, incomplete, or badly phrased
# "FastAPI" → is the user asking what it is, how to install it, or for examples?The techniques you'll learn:
1. Query expansion (raising recall):
# Generate several similar queries
expanded_queries = [
"What is FastAPI?",
"FastAPI framework explained",
"FastAPI features",
"FastAPI vs Flask"
]
# Search with all of them and merge the resultsExpected improvement: +20-30% recall (you find more relevant docs)
2. Query rewriting (clarity):
# Reformulate the query for clarity
user_query = "fastapi auth" # Ambiguous
rewritten_query = llm.invoke(f"""
Reformulate this query for a documentation search:
Query: {user_query}
Reformulated:
""")
# Output: "How to implement authentication in FastAPI using OAuth2"Expected improvement: +10-15% precision (clearer queries)
3. Query decomposition (multi-hop):
# A complex query → simple sub-queries
user_query = "Compare FastAPI and Flask authentication approaches"
decomposed = [
"FastAPI authentication methods",
"Flask authentication methods",
"Compare FastAPI vs Flask"
]
# Search each sub-query and aggregate the resultsExpected improvement: +15-20% precision on complex queries
4. HyDE (Hypothetical Document Embeddings):
# Generate a hypothetical document that would answer the query
hypothetical_doc = llm.invoke(f"""
Write a document that would answer: {user_query}
""")
# Search with the hypothetical document's embedding (not the query's)
hyde_embedding = create_embedding(hypothetical_doc)
results = collection.query(query_embeddings=[hyde_embedding])Expected improvement: +15-25% recall (doc embeddings > query embeddings)
The Module 3 project:
Implement query optimization in the baseline RAG:
baseline = benchmark_rag(query_optimization=None)
# Recall@50: 52%
expansion = benchmark_rag(query_optimization="expansion")
# Recall@50: 67% (+15%)
hyde = benchmark_rag(query_optimization="hyde")
# Recall@50: 72% (+20%)🎯 Module 4: Re-ranking Techniques
The goal:
Improve retrieval precision by re-ordering the top-K documents with more sophisticated models.
The baseline's problem:
# Similarity search returns the top-5
results = collection.query(query_embeddings=[...], n_results=5)
# The problem: cosine similarity isn't perfect
# Some docs with a high similarity score are false positives
# Precision@5: 68% (32% are irrelevant)The techniques you'll learn:
1. Cross-encoder re-ranking:
from sentence_transformers import CrossEncoder
# 1. Wide retrieval (top-20)
results = collection.query(query_embeddings=[...], n_results=20)
# 2. Re-ranking with a cross-encoder
model = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
scores = model.predict([(user_query, doc) for doc in results['documents'][0]])
# 3. Select the re-ranked top-5
top_5 = [results['documents'][0][i] for i in np.argsort(scores)[::-1][:5]]Expected improvement: +20-25% precision
Trade-off: +150-200ms latency
2. LLM-based re-ranking:
# Re-rank with an LLM
reranking_prompt = f"""
Query: {user_query}
Documents:
1. {doc1}
2. {doc2}
3. {doc3}
Rank documents by relevance (output: 2,1,3 for example):
"""
ranking = llm.invoke(reranking_prompt)
# Output: "2,1,3" (doc2 is the most relevant, then doc1, then doc3)Expected improvement: +25-30% precision (best quality)
Trade-off: +500-800ms latency, +LLM cost
3. Cohere Rerank API:
import cohere
co = cohere.Client("api_key")
reranked = co.rerank(
query=user_query,
documents=[doc1, doc2, doc3],
top_n=5,
model="rerank-english-v2.0"
)
# Output: the documents re-ranked by CohereExpected improvement: +22-27% precision
Trade-off: +100-150ms latency, $2/1K rerank calls
The Module 4 project:
Implement re-ranking in the baseline RAG:
baseline = benchmark_rag(reranking=None)
# Precision@5: 68%
cross_encoder = benchmark_rag(reranking="cross_encoder")
# Precision@5: 88% (+20%)
# Latency: +180ms
cohere = benchmark_rag(reranking="cohere")
# Precision@5: 90% (+22%)
# Latency: +120ms🔀 Module 5: Hybrid Search (BM25 + Embeddings)
The goal:
Combine keyword search (BM25) and semantic search (embeddings) for better coverage.
The baseline's problem:
# Semantic search alone
query = "FastAPI OAuth2PasswordBearer"
# Problem 1: it misses the exact match on "OAuth2PasswordBearer" (a class name)
# Semantic can confuse it with "OAuth2AuthorizationCodeBearer"
# Problem 2: it misses keyword-heavy queries ("API endpoint /users/{id}")The techniques you'll learn:
1. BM25 + embeddings with RRF:
from rank_bm25 import BM25Okapi
# 1. BM25 keyword search
bm25_results = bm25.get_top_n(query.split(), corpus, n=50)
# 2. Semantic search
semantic_results = collection.query(query_embeddings=[...], n_results=50)
# 3. Reciprocal Rank Fusion
def rrf(bm25_results, semantic_results, k=60):
scores = {}
for rank, doc in enumerate(bm25_results, 1):
scores[doc] = scores.get(doc, 0) + 1/(k + rank)
for rank, doc in enumerate(semantic_results, 1):
scores[doc] = scores.get(doc, 0) + 1/(k + rank)
return sorted(scores.items(), key=lambda x: x[1], reverse=True)
merged = rrf(bm25_results, semantic_results)Expected improvement: +15-20% precision, +10-15% recall
2. Weighted hybrid (α blending):
# Combine the scores with a weight
alpha = 0.7 # 70% semantic, 30% BM25
hybrid_score = alpha * semantic_score + (1 - alpha) * bm25_scoreExpected improvement: +12-18% precision (tunable with α)
The Module 5 project:
Implement hybrid search:
baseline_semantic = benchmark_rag(search="semantic")
# Precision@5: 68%, Recall@50: 52%
baseline_bm25 = benchmark_rag(search="bm25")
# Precision@5: 72%, Recall@50: 48%
hybrid_rrf = benchmark_rag(search="hybrid_rrf")
# Precision@5: 83% (+15%), Recall@50: 62% (+10%)🏷️ Module 6: Metadata Filtering
The goal:
Shrink the search space with metadata (date, type, source) before the semantic search runs.
The baseline's problem:
# Search with no filtering
results = collection.query(query_embeddings=[...], n_results=5)
# The problem: it searches the ENTIRE corpus (100K docs)
# Many docs are contextually irrelevant (e.g. docs from 5 years ago)The techniques you'll learn:
1. Pre-filtering (metadata WHERE):
# Filter BEFORE the semantic search
results = collection.query(
query_embeddings=[...],
n_results=5,
where={
"date": {"$gte": "2024-01-01"}, # Only 2024 docs
"type": "tutorial", # Only tutorials
"source": "official_docs" # Only official docs
}
)
# The benefit: the search space shrinks 95% (100K → 5K docs)Expected improvement: +10-15% precision, -90% search space
2. Post-filtering (filter afterwards):
# Retrieve first, filter afterwards
results = collection.query(query_embeddings=[...], n_results=50)
# Filter by metadata
filtered = [doc for doc in results['documents'][0]
if doc['metadata']['type'] == 'tutorial'][:5]Trade-off: pre-filtering > post-filtering (it's more efficient)
3. Multi-tenant filtering (privacy):
# Every user has a workspace_id
results = collection.query(
query_embeddings=[...],
n_results=5,
where={"workspace_id": user.workspace_id} # Privacy-critical
)
# The benefit: zero cross-workspace leakageThe Module 6 project:
Implement metadata filtering:
baseline = benchmark_rag(filtering=None)
# Precision@5: 68%, Search space: 100K docs
pre_filtering = benchmark_rag(filtering="pre", metadata=["date", "type"])
# Precision@5: 81% (+13%), Search space: 8K docs (-92%)🚀 Module 7: Production Vector DBs (Pinecone)
The goal:
Migrate from ChromaDB (dev, local) to Pinecone (production, managed) for scalability.
The baseline's problem:
# ChromaDB local
client = chromadb.Client()
# Problem 1: performance degrades beyond 100K docs
# 10K docs: 30ms query ✅
# 100K docs: 300ms query ⚠️
# 1M docs: 3,000ms query ❌
# Problem 2: single-machine (not distributed)
# Problem 3: not managed (you handle backups, updates, etc.)The techniques you'll learn:
1. Migrating ChromaDB → Pinecone:
from pinecone import Pinecone
# Set up Pinecone
pc = Pinecone(api_key="...")
index = pc.create_index(name="my-index", dimension=1536, metric="cosine")
# Migrate the data from ChromaDB
chroma_docs = chroma_collection.get()
# Batch upsert into Pinecone
index.upsert(
vectors=[
(id, embedding, metadata)
for id, embedding, metadata in zip(
chroma_docs['ids'],
chroma_docs['embeddings'],
chroma_docs['metadatas']
)
],
batch_size=100
)2. Pinecone features:
# Namespaces (multi-tenant)
index.upsert(vectors=[...], namespace="user_123")
index.query(vector=[...], namespace="user_123")
# Metadata filtering (pre-filtering)
index.query(
vector=[...],
filter={"type": {"$eq": "tutorial"}},
top_k=5
)
# Sparse-dense vectors (native hybrid search)
index.upsert(
vectors=[
("id1", dense_vector, sparse_vector, metadata)
]
)Performance comparison:
| Vector DB | 10K docs | 100K docs | 1M docs | Cost |
|---|---|---|---|---|
| ChromaDB | 30ms | 300ms | 3,000ms | Free |
| Pinecone | 40ms | 45ms | 50ms | $70/mo |
The decision: Pinecone if your dataset is >100K docs or you need a guaranteed <100ms.
The Module 7 project:
Migrate the baseline RAG to Pinecone:
# Before (ChromaDB, 100K docs)
latency_p95 = 850ms
# After (Pinecone, 100K docs)
latency_p95 = 520ms (-40%)📊 Module 8: RAG Evaluation & Testing (RAGAS)
The goal:
Evaluate the RAG system's quality with automated metrics (faithfulness, relevancy).
The baseline's problem:
# Manual evaluation
# 1. Generate an answer
answer = rag_system.query("What is FastAPI?")
# 2. Read the answer and judge it by hand
# "Is it correct? Is it grounded in the docs? Does it answer the question?"
# The problem: it doesn't scale, it isn't consistent, it isn't automatedThe techniques you'll learn:
1. The RAGAS framework (automated metrics):
from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_recall
# Evaluation dataset
dataset = {
"question": ["What is FastAPI?"],
"answer": ["FastAPI is a web framework..."],
"contexts": [["FastAPI is a modern web framework..."]],
"ground_truth": ["FastAPI is a Python web framework"]
}
# Evaluate
results = evaluate(
dataset,
metrics=[faithfulness, answer_relevancy, context_recall]
)
print(results)
# Output:
# {
# 'faithfulness': 0.92, # Grounded in the context?
# 'answer_relevancy': 0.89, # Does it answer the question?
# 'context_recall': 0.75 # Did we retrieve the relevant docs?
# }2. Golden dataset (regression testing):
# Build a dataset of 50 test queries + ground truth
golden_dataset = [
{
"question": "What is FastAPI?",
"ground_truth": "FastAPI is a web framework...",
"relevant_docs": ["doc_123", "doc_456"]
},
# ... 49 more
]
# Evaluate on every deploy
def regression_test(rag_system):
results = evaluate(rag_system, golden_dataset)
assert results['faithfulness'] > 0.85, "Faithfulness degraded!"
assert results['answer_relevancy'] > 0.80, "Relevancy degraded!"
return results3. CI/CD integration (continuous evaluation):
# Auto-eval on every PR/deploy
# .github/workflows/rag-eval.yml
steps:
- name: Run RAG evaluation
run: |
python test_rag.py
# If the metrics drop below the threshold, fail CIThe Module 8 project:
Implement the complete evaluation:
# Baseline (manual)
faithfulness = "Unknown"
answer_relevancy = "Unknown"
# With RAGAS (automated)
results = evaluate(rag_system, golden_dataset)
# faithfulness: 0.92
# answer_relevancy: 0.89
# context_recall: 0.75
# Identify the gaps and apply improvements
if results['context_recall'] < 0.80:
apply_improvement("query_expansion") # Module 3🔗 How the modules connect
The evolving pipeline:
Module 1: Baseline RAG
↓ (Precision 68%, Recall 52%)
Module 2: + Optimized chunking
↓ (Precision 78%, Recall 52%) [+10% precision]
Module 3: + Query optimization
↓ (Precision 78%, Recall 67%) [+15% recall]
Module 4: + Re-ranking
↓ (Precision 88%, Recall 67%) [+10% precision]
Module 5: + Hybrid search
↓ (Precision 91%, Recall 77%) [+3% precision, +10% recall]
Module 6: + Metadata filtering
↓ (Precision 93%, Recall 77%) [+2% precision, -90% search space]
Module 7: + Pinecone migration
↓ (Latency -40%, Scalability ∞)
Module 8: + RAGAS evaluation
↓ (Automated quality gates, CI/CD integration)
Final performance:
- Precision@5: 93% (baseline: 68%, gain: +25%)
- Recall@50: 77% (baseline: 52%, gain: +25%)
- Latency P95: 1,200ms (baseline: 800ms, trade-off: +400ms for quality)
- Faithfulness: 0.92 (baseline: N/A)
- Cost per query: $0.003 (acceptable for production)
🎯 The evolving project (Module 8)
The final goal:
A production-ready RAG system that integrates ALL the techniques:
# advanced_rag_system.py (the final Module 8 build)
class AdvancedRAGSystem:
def __init__(self):
# Module 2: Chunking
self.chunker = RecursiveCharacterTextSplitter(...)
# Module 3: Query optimization
self.query_optimizer = QueryExpansionOptimizer(...)
# Module 4: Re-ranking
self.reranker = CrossEncoder('cross-encoder/...')
# Module 5: Hybrid search
self.bm25 = BM25Okapi(...)
self.semantic_search = PineconeIndex(...)
# Module 6: Metadata filtering
self.metadata_filters = {...}
# Module 7: Production vector DB
self.vector_db = Pinecone(...)
# Module 8: Evaluation
self.evaluator = RAGASEvaluator(...)
def query(self, user_query: str) -> dict:
# 1. Query optimization (M3)
optimized_queries = self.query_optimizer.expand(user_query)
# 2. Hybrid retrieval (M5) with metadata filtering (M6)
bm25_results = self.bm25.search(user_query)
semantic_results = self.semantic_search.query(
optimized_queries[0],
filter=self.metadata_filters
)
merged = reciprocal_rank_fusion(bm25_results, semantic_results)
# 3. Re-ranking (M4)
reranked = self.reranker.predict([(user_query, doc) for doc in merged])
top_k = select_top_k(reranked, k=5)
# 4. Generation
answer = self.llm.invoke(context=top_k, query=user_query)
# 5. Evaluation (M8)
metrics = self.evaluator.evaluate(
question=user_query,
answer=answer,
contexts=top_k
)
return {
"answer": answer,
"sources": top_k,
"metrics": metrics
}
# The final benchmark
rag_system = AdvancedRAGSystem()
final_results = benchmark(rag_system, golden_dataset)
print(f"""
Advanced RAG System Results:
- Precision@5: {final_results['precision']:.2%}
- Recall@50: {final_results['recall']:.2%}
- Faithfulness: {final_results['faithfulness']:.2f}
- Latency P95: {final_results['latency_p95']:.0f}ms
- Cost per query: ${final_results['cost_per_query']:.4f}
""")Expected output:
Advanced RAG System Results:
- Precision@5: 93%
- Recall@50: 77%
- Faithfulness: 0.92
- Latency P95: 1,200ms
- Cost per query: $0.0031
🎯 Summary
Key concepts:
- ✅ Module 1: Baseline RAG (68% precision, 52% recall)
- ✅ Module 2: Chunking strategies → +10-15% precision
- ✅ Module 3: Query optimization → +15-25% recall
- ✅ Module 4: Re-ranking → +20-25% precision
- ✅ Module 5: Hybrid search → +15-18% precision, +10-15% recall
- ✅ Module 6: Metadata filtering → +10-13% precision, -90% search space
- ✅ Module 7: Pinecone migration → -40% latency, ∞ scalability
- ✅ Module 8: RAGAS evaluation → automated quality gates
- ✅ The evolving pipeline: baseline → advanced (68% → 93% precision)
- ✅ The trade-off we accept: +400ms latency for +25% accuracy
What's next:
Capsule 08 gives you the Module 1 mini-project: implement the baseline RAG with ChromaDB + OpenAI, measure its performance, and document your decisions so you can compare them against the improvements in modules 2-8.
📚 Additional resources
- RAG Architecture Patterns - LangChain patterns
- Advanced RAG Techniques - Pinecone's guide
- Production RAG Best Practices - The LlamaIndex blog
- RAGAS Documentation - The evaluation framework
- Hybrid Search Deep Dive - Elasticsearch's guide
- Chunking Strategies Comparison - An analysis of chunk size
Created: February 6, 2026
Version: 1.0