Module Introduction: The Complete RAG Pipeline

Capsule overview

Welcome to Module 1 of Advanced RAG Techniques. This module is your starting point: you'll understand the complete architecture of advanced RAG, its critical components, the decisions you'll make, and you'll build a baseline RAG system that will serve as the comparison point for measuring future improvements.

Why start with architecture before techniques? Because you need the full map before you navigate. Advanced RAG isn't applying techniques at random (chunking here, re-ranking there). It's a pipeline of components that interact: if your chunking is bad, re-ranking won't help. If your embeddings are wrong, hybrid search won't improve anything. Architecture first, techniques later.

This module gives you three critical things: (1) the big picture of advanced RAG, (2) a decision framework for choosing the right techniques, (3) a baseline RAG system to measure improvements against. Without these three, you'd be optimizing blind.

---

🎯 Module 1 objectives

By the end of this module, you'll be able to:

Architecture and design:

• ✅ Explain RAG's 4 critical components (indexing, retrieval, generation, evaluation)
• ✅ Draw a complete end-to-end RAG architecture with its data flow
• ✅ Identify where each advanced technique applies (chunking in indexing, re-ranking in retrieval, etc.)

Technical decisions:

• ✅ Make justified chunking decisions (fixed vs semantic vs recursive)
• ✅ Choose the right embeddings (OpenAI vs local, dimensions, metric)
• ✅ Select a vector DB based on the use case (ChromaDB local vs Pinecone production)
• ✅ Decide when to apply re-ranking (latency vs precision trade-off)

Metrics and evaluation:

• ✅ Define success metrics for RAG (latency, accuracy, cost)
• ✅ Measure baseline metrics (retrieval quality, generation quality)
• ✅ Set realistic targets (what "good" means for your use case)

Implementation:

• ✅ Implement a Baseline RAG System with ChromaDB + OpenAI
• ✅ Measure the baseline's performance (latency, retrieval quality)
• ✅ Document the baseline for future comparisons

---

📚 Module roadmap

Capsule 01: Module introduction (this capsule)

• Overview of advanced vs basic RAG
• Module objectives
• Complete roadmap
• Initial technical setup

Capsule 02: RAG pipeline components

• Indexing: Chunking, embeddings, storage
• Retrieval: Query processing, similarity search, top-K selection
• Generation: Context injection, LLM prompting, response formatting
• Evaluation: Retrieval metrics, generation metrics, feedback loop

Capsule 03: Architecture decisions

• Chunking: Fixed vs semantic vs recursive (when to use each)
• Embeddings: OpenAI vs local vs Cohere (trade-offs)
• Vector DB: ChromaDB vs Pinecone vs Weaviate (decision matrix)
• Re-ranking: When the extra latency is worth it

Capsule 04: Success metrics

• Latency: P50, P95, P99 (what's acceptable for each use case)
• Accuracy: Retrieval precision/recall, generation faithfulness
• Cost: Cost per query, typical optimizations
• User satisfaction: Implicit signals (thumbs up/down, re-queries)

Capsule 05: Real-world use cases

• Perplexity: Hybrid search + re-ranking + citations
• Notion AI: Metadata filtering + personalization
• ChatGPT Plugins: Tool use + RAG integration
• Lessons: Common patterns in production

Capsule 06: When to use advanced techniques

• Simple RAG: When it's enough (basic cases)
• Advanced RAG: When you need it (production, scale, precision)
• Trade-offs: Complexity vs improvement, cost vs quality
• Decision matrix: The technical decision table

Capsule 07: Setting up the evolving project

• Code structure: Organization for modules 2-8
• Config management: .env, settings.py, constants
• Logging: Structured logging for debugging
• Testing: Basic testing framework

Capsule 08: Project - Baseline RAG System

• Goal: A simple RAG system with ChromaDB + OpenAI
• Features: Index documents, query, generate a response
• No advanced techniques: Naive chunking, no re-ranking, no hybrid
• Purpose: A baseline to compare future improvements against

---

🔧 Technical setup

Software prerequisites:

Before you start, check that you have:

# Python 3.9+
python --version  # Must be 3.9 or higher

# Updated pip
pip install --upgrade pip

Installing dependencies:

Create a virtual environment and install the dependencies:

# Create virtual environment
python -m venv venv

# Activate (Mac/Linux)
source venv/bin/activate

# Activate (Windows)
venv\Scripts\activate

# Install the basic dependencies for module 1
pip install chromadb==0.4.22 openai==1.12.0 python-dotenv==1.0.0

Specific versions:

• chromadb==0.4.22: Local vector database (free)
• openai==1.12.0: OpenAI API (embeddings + LLM)
• python-dotenv==1.0.0: Environment variable handling

API keys you'll need:

1. OpenAI API Key:

   • Create an account at https://platform.openai.com/
   • Generate an API key at https://platform.openai.com/api-keys
   • Estimated cost for module 1: ~$0.50-1.00 (embeddings + LLM calls)

2. .env file:

Create a .env file at the root of the project:

# .env
OPENAI_API_KEY=sk-proj-...your-key-here...

⚠️ IMPORTANT: Add .env to .gitignore so you don't commit secrets:

# .gitignore
.env
venv/
__pycache__/
*.pyc

Project structure:

Create this folder structure:

advanced-rag-guide/
├── .env                    # API keys (DO NOT commit)
├── .gitignore              # Ignore .env and venv
├── requirements.txt        # Dependencies
├── module-01/              # Module 1 code
│   ├── baseline_rag.py     # Baseline RAG system
│   ├── config.py           # Configuration
│   └── utils.py            # Utilities
├── data/                   # Documents to index
│   └── sample_docs.txt     # Sample documents
└── README.md               # Documentation

Verifying the setup:

Run this script to check that everything works:

# test_setup.py
import chromadb
import openai
from dotenv import load_dotenv
import os

# Load .env
load_dotenv()

# Check the OpenAI API key
openai_key = os.getenv("OPENAI_API_KEY")
if not openai_key:
    print("❌ OPENAI_API_KEY not found in .env")
else:
    print("✅ OPENAI_API_KEY configured")

# Check ChromaDB
try:
    client = chromadb.Client()
    print("✅ ChromaDB works correctly")
except Exception as e:
    print(f"❌ ChromaDB error: {e}")

# Check the OpenAI API (embeddings)
try:
    from openai import OpenAI
    client = OpenAI()
    response = client.embeddings.create(
        model="text-embedding-ada-002",
        input="Test"
    )
    print(f"✅ OpenAI embeddings works (dimensions: {len(response.data[0].embedding)})")
except Exception as e:
    print(f"❌ OpenAI error: {e}")

print("\n🎉 Setup complete! Ready for module 1.")

Run it:

python test_setup.py

Expected output:

✅ OPENAI_API_KEY configured
✅ ChromaDB works correctly
✅ OpenAI embeddings works (dimensions: 1536)

🎉 Setup complete! Ready for module 1.

If you see ✅ everywhere, you're ready. If you see ❌, go back over the previous steps.

---

🗺️ Context: where are we in the guide?

Position in the guide:

Phase 1: Advanced Fundamentals
├── [NOW] Module 1: The Complete RAG Pipeline ← You are here
├── Module 2: Chunking Strategies
└── Module 3: Query Optimization

Phase 2: Retrieval Techniques
├── Module 4: Re-ranking
├── Module 5: Hybrid Search
└── Module 6: Metadata Filtering

Phase 3: Production & Integration
├── Module 7: Production RAG with Pinecone
└── Module 8: Evaluation + Capstone Project

What comes before?

Nothing. This is the first module. But it assumes that:

• You've already implemented basic RAG at least once (embedding + search + LLM)
• You understand what embeddings and cosine similarity are
• You've used ChromaDB or a similar vector DB

What comes after?

Module 2: Chunking Strategies

• You'll optimize the baseline's naive chunking
• You'll implement fixed, semantic and recursive chunking
• You'll compare strategies with objective metrics

Module 3: Query Optimization

• You'll improve the user's queries before retrieval
• You'll implement expansion, rewriting, decomposition

Modules 4-8:

• Advanced retrieval techniques and production deployment

---

🎯 The module's professional goal

By the end of this module, you'll have:

1. A complete mental map of advanced RAG:

   • A clear end-to-end architecture
   • The components and how they interact
   • Where each technique applies

2. A decision framework:

   • When to use each chunking strategy
   • When to apply re-ranking
   • When hybrid search is worth it

3. A working baseline RAG system:

   • ChromaDB + OpenAI
   • No advanced techniques
   • Documented baseline metrics

4. The ability to design RAG systems:

   • Make justified architectural decisions
   • Define the right success metrics
   • Clear trade-offs (complexity vs improvement)

---

💡 Why advanced RAG?

Basic RAG works, but...

Basic RAG (embedding + search + LLM) is enough for:

• ✅ Prototypes and MVPs
• ✅ Small datasets (<1,000 documents)
• ✅ Simple, direct queries
• ✅ No production requirements

But basic RAG has problems:

Problem 1: Naive chunking loses context

Document: "Python is a language. FastAPI is a web framework."
Naive chunking (100 characters): ["Python is a language. FastAPI is", "a web framework."]
→ The second chunk loses context (what is "a web framework"?)

Problem 2: Ambiguous queries

User query: "performance issues"
→ The system doesn't know whether to look for "latency" or "throughput" or "memory usage"
→ Retrieval returns irrelevant documents

Problem 3: Top-K full of irrelevant hits

Top-5 documents:
1. Relevant (score: 0.85)
2. Irrelevant (score: 0.83)  ← Very similar in embedding space but not relevant
3. Relevant (score: 0.81)
4. Irrelevant (score: 0.80)
5. Relevant (score: 0.79)
→ 40% of top-K is junk → confused LLM

Problem 4: Semantic search alone fails on keywords

User query: "Error code E4502"
Semantic search: Searches by "error" and "code" → returns generic errors
→ It doesn't find the specific E4502 (keyword exact match)

Advanced RAG solves this:

Problem	Advanced technique	Module
Naive chunking	Semantic/Recursive chunking	Module 2
Ambiguous queries	Query optimization (expansion, rewriting)	Module 3
Top-K full of irrelevant hits	Re-ranking (cross-encoder, LLM)	Module 4
Fails on keywords	Hybrid search (BM25 + semantic)	Module 5
Searches the whole corpus	Metadata filtering	Module 6
Doesn't scale	Pinecone managed	Module 7
No metrics	RAGAS evaluation	Module 8

---

🧭 Connection with the project

Module 1 project: Baseline RAG System

What will you build?

A simple RAG system with:

• ✅ Indexing with ChromaDB (OpenAI embeddings)
• ✅ Querying with similarity search (top-5 documents)
• ✅ Generation with an LLM (GPT-3.5-turbo)
• ❌ No advanced chunking (naive fixed-size)
• ❌ No query optimization
• ❌ No re-ranking
• ❌ No hybrid search

Why build a baseline if we already know it isn't optimal?

1. A comparison point: Without a baseline, you don't know whether the advanced techniques actually improve anything
2. Measuring improvements: Modules 2-7 will add techniques → you'll measure the delta vs the baseline
3. Justifying complexity: If an advanced technique only improves things by 2%, maybe it isn't worth it

Metrics you'll measure:

Baseline metrics (module 1):
- Latency: ~500-800ms (embedding + search + LLM)
- Retrieval quality: ~60-70% precision (top-5)
- Generation quality: ~70-80% faithfulness

Advanced metrics (module 8):
- Latency: ~600-1000ms (with re-ranking +200ms)
- Retrieval quality: ~85-90% precision (with re-ranking)
- Generation quality: ~90-95% faithfulness (better retrieval → better generation)

Improvement: +20-30% precision, +15-20% faithfulness
Cost: +200ms latency, +complexity
→ Decision: Is it worth it for production? You decide, with data.

The evolving project (modules 1-8):

Module 1: Baseline RAG (naive)
  ↓
Module 2: + Semantic chunking (better context)
  ↓
Module 3: + Query optimization (better queries)
  ↓
Module 4: + Re-ranking (better precision)
  ↓
Module 5: + Hybrid search (better coverage)
  ↓
Module 6: + Metadata filtering (better relevance)
  ↓
Module 7: + Pinecone production (better scale)
  ↓
Module 8: The complete system, evaluated with RAGAS

Each module adds ONE technique → you measure the improvement → you decide whether to keep it.

---

✅ Evidence of success

By the end of this module, you should be able to:

Architecture:

• Draw a complete RAG architecture with its 4 components (indexing, retrieval, generation, evaluation)
• Explain the end-to-end data flow (document → chunks → embeddings → vector DB → query → retrieval → generation)
• Identify where each advanced technique applies (chunking in indexing, re-ranking in retrieval, etc.)

Decisions:

• Justify your choice of chunking strategy (fixed vs semantic vs recursive) based on the document type
• Justify your choice of embeddings (OpenAI vs local) based on the use case
• Justify your choice of vector DB (ChromaDB vs Pinecone) based on scale and budget

Metrics:

• Define success metrics for your use case (latency, accuracy, cost)
• Measure baseline metrics (latency, retrieval quality, generation quality)
• Set realistic targets (what "good" means in the industry)

Implementation:

• A working RAG system with ChromaDB + OpenAI
• Runnable code (index documents, query, generate a response)
• Documented baseline metrics (latency, retrieval quality)

If you answer "YES" to all of them → ✅ Ready for Module 2
If you answer "NO" to 3 or more → ⚠️ Review the Module 1 capsules

---

📝 Next steps

1. Read Capsule 02: RAG pipeline components (indexing, retrieval, generation, evaluation)
2. Read Capsule 03: Architecture decisions (chunking, embeddings, vector DB, re-ranking)
3. Read Capsule 04: Success metrics (latency, accuracy, cost)
4. Read Capsule 05: Real-world use cases (Perplexity, Notion AI, ChatGPT)
5. Read Capsule 06: When to use advanced techniques (simple vs advanced trade-offs)
6. Read Capsule 07: Setting up the evolving project (code structure)
7. Implement Capsule 08: Baseline RAG System (hands-on project)

---

🎓 Summary

Key concepts from this capsule:

• ✅ Advanced RAG solves basic RAG's problems (chunking, queries, precision, keywords, metadata)
• ✅ Module 1 gives you the big picture: architecture, decisions, metrics, use cases, baseline
• ✅ The baseline RAG system is the comparison point for measuring future improvements
• ✅ The evolving project: each module adds ONE technique and you measure the delta
• ✅ Technical setup: ChromaDB + OpenAI + a properly configured .env
• ✅ A clear roadmap: 8 capsules, from architecture to implementation

What's next:

Capsule 02 teaches you RAG's 4 critical components: indexing (chunks, embeddings, storage), retrieval (query processing, search, top-K), generation (context injection, prompting), and evaluation (metrics, feedback loop).

---

📚 Additional resources

1. Retrieval-Augmented Generation (Paper) - The original RAG paper (Lewis et al., 2020)
2. LangChain RAG Tutorial - LangChain's official tutorial
3. ChromaDB Docs - ChromaDB's official documentation
4. OpenAI Embeddings Guide - OpenAI's embeddings guide
5. Pinecone RAG Guide - Pinecone's RAG guide
6. Building RAG Systems (Video) - A visual explanation of RAG (30 min)

---

Created: February 6, 2026
Version: 1.0