Module 1: The Complete RAG Pipeline (Architecture Overview)

Real-World RAG Use Cases in Production

Next →

Capsule overview

RAG isn't theoretical. Companies like Perplexity, Notion AI, ChatGPT Plugins, GitHub Copilot and Stripe Documentation run RAG in production every single day. This capsule breaks down real architectures: which techniques they use, why they made those decisions, which trade-offs they accepted, and what you can learn for your own system.

Seeing real cases gives you critical context. When you learn re-ranking in Module 4, you'll understand that Perplexity uses it because >90% precision is critical for web search. When you learn hybrid search in Module 5, you'll understand that Stripe uses it because its queries contain exact API names that semantic search misses.

This capsule breaks down 5 real architectures: (1) what problem they solve, (2) which RAG components they use, (3) their specific technical decisions, (4) their success metrics, (5) the lessons that apply to your case.

🌐 Case 1: Perplexity AI (search engine)

The problem:

Conversational web search with cited sources. A user asks "What happened with SVB bank?", Perplexity searches the web, generates a grounded answer, and cites its sources.

The RAG architecture:

User Query
    ↓
Query Optimization (expansion + rewriting)
    ↓
Hybrid Retrieval (Web search + Vector search)
    ↓
Re-ranking (Top-100 → Top-5)
    ↓
LLM Generation (GPT-4 with citations)
    ↓
Response + Sources

The specific components:

1. Query optimization:

# Perplexity expands queries to boost recall
user_query = "What happened with SVB bank?"

# Expansion:
expanded_queries = [
    "Silicon Valley Bank collapse 2023",
    "SVB bank failure causes",
    "What happened to Silicon Valley Bank"
]

# Rewriting (for clarity):
rewritten = "Silicon Valley Bank collapse March 2023 timeline and causes"

Why: user queries are ambiguous ("SVB" isn't obvious). Expansion boosts recall (finding more results); rewriting clarifies intent.

2. Hybrid retrieval:

# Perplexity combines keyword search (BM25) + semantic search

# Keyword search (fast, exact)
keyword_results = search_web_bm25(query)  # Top-50

# Semantic search (context-aware)
semantic_results = search_web_embeddings(query)  # Top-50

# Merge with reciprocal rank fusion
merged = reciprocal_rank_fusion(keyword_results, semantic_results)  # Top-100

Why: web search needs both: keywords for proper nouns ("SVB"), semantic for concepts ("collapse", "bank failure").

Technique: Module 5 teaches hybrid search in detail.

3. Re-ranking:

# Perplexity re-ranks top-100 → top-5 with a cross-encoder

from sentence_transformers import CrossEncoder

model = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-12-v2')

# Re-rank the top-100
scores = model.predict([
    (user_query, doc['text']) for doc in merged[:100]
])

# Select the top-5
top_5_indices = np.argsort(scores)[::-1][:5]
final_docs = [merged[i] for i in top_5_indices]

Why: >90% precision is critical for search. Re-ranking removes the false positives.

Trade-off: +200ms latency, but it's worth it.

Technique: Module 4 teaches re-ranking.

4. LLM generation with citations:

# Perplexity generates the answer with numbered sources

context = "\n\n".join([
    f"[{i+1}] {doc['text']}\nSource: {doc['url']}"
    for i, doc in enumerate(final_docs)
])

prompt = f"""
Use the following context to answer the question. Cite the sources with [1], [2], etc.

Context:
{context}

Question: {user_query}

Answer with citations:
"""

response = openai.ChatCompletion.create(
    model="gpt-4-turbo",
    messages=[
        {"role": "system", "content": "You are a conversational search engine. Always cite your sources."},
        {"role": "user", "content": prompt}
    ]
)

answer = response.choices[0].message.content

# Output: "Silicon Valley Bank (SVB) collapsed on March 10, 2023 [1] due to a liquidity crisis [2]..."

Why: citations increase trustworthiness. The user can verify the information.

Perplexity's metrics:

MetricTargetActualTechnique used
Latency P95<3,000ms~2,500msHybrid search (fast), re-ranking (optimized)
Precision@5>90%~92%Cross-encoder re-ranking
Recall@100>70%~75%Query expansion + hybrid
Faithfulness>95%~96%Citations required in the prompt

The lessons that apply to you:

✅ Query optimization matters: expansion boosts recall +20-30%
✅ Hybrid search for the web: keywords + semantic together beat either one alone
✅ Re-ranking is worth it: +200ms latency → +25% precision
✅ Citations build trust: users can validate the information

📝 Case 2: Notion AI (document Q&A)

The problem:

A user asks about their Notion workspace (e.g. "What did we decide in the Q4 strategy meeting?"). Notion AI searches the workspace's private docs and answers.

The RAG architecture:

User Query
    ↓
Metadata Filtering (workspace_id, date_range)
    ↓
Semantic Search (embeddings)
    ↓
Context Ranking (recent docs prioritized)
    ↓
LLM Generation (GPT-4)
    ↓
Response + Page Links

The specific components:

1. Metadata filtering:

# Notion AI filters by workspace and date range BEFORE the semantic search

user_query = "What did we decide in the Q4 strategy meeting?"
workspace_id = "user123_workspace"

# Extract the metadata from the query
metadata_filter = {
    "workspace_id": workspace_id,
    "page_type": "meeting_notes",
    "date_range": ("2023-10-01", "2023-12-31"),  # Q4 2023
    "tags": ["strategy", "meeting"]
}

# Search only within the relevant subset
results = collection.query(
    query_embeddings=[query_embedding],
    n_results=10,
    where=metadata_filter  # Pre-filtering
)

Why: a private workspace has 50,000+ pages. Without filtering, semantic search returns irrelevant docs from other projects. Metadata filtering shrinks the search space by 95%.

Technique: Module 6 teaches metadata filtering.

2. Context ranking:

# Notion AI prioritizes recent documents (relevance decay)

def time_weighted_score(doc, query_embedding):
    """Combines the similarity score with recency"""
    
    # Similarity score (0-1)
    similarity = cosine_similarity(query_embedding, doc['embedding'])
    
    # Time decay (recent documents weigh more)
    days_ago = (datetime.now() - doc['created_at']).days
    recency_weight = 1 / (1 + days_ago / 30)  # Decay over 30 days
    
    # Combined score
    final_score = 0.7 * similarity + 0.3 * recency_weight
    
    return final_score

# Re-rank by the time-weighted score
docs_ranked = sorted(docs, key=lambda d: time_weighted_score(d, query_embedding), reverse=True)
top_k = docs_ranked[:5]

Why: in workspaces, recent documents tend to be more relevant. Meeting notes from 2 years ago are less useful than ones from 2 weeks ago.

3. Privacy constraints:

# Notion AI NEVER crosses workspaces

# Embeddings are workspace-scoped
collection_name = f"workspace_{workspace_id}"
collection = client.get_collection(collection_name)

# Queries only search within the user's collection
results = collection.query(...)  # Limited scope

# The LLM context ONLY has the workspace's docs
# Zero-shot learning: the LLM has no data from other users

Why: privacy is critical. Notion can't leak answers from one workspace into another.

Notion AI's metrics:

MetricTargetActualTechnique used
Latency P95<2,000ms~1,800msPre-filtering shrinks the search space
Precision@5>80%~83%Metadata filtering + recency
Privacy violations00Workspace-scoped collections
Faithfulness>90%~91%Cites only the workspace's docs

The lessons that apply to you:

✅ Metadata filtering is critical: it shrinks the search space 10-100x
✅ Recency matters in docs: time-weighted ranking improves relevance +15%
✅ Privacy by design: separate collections for multi-tenant
✅ An extra ranking layer: semantic search alone isn't enough, custom ranking helps

🧑‍💻 Case 3: GitHub Copilot Chat (code Q&A)

The problem:

A developer asks "How do I parse JSON in Python?" or "What does this function do?". Copilot searches the Python docs + the project's code + the current file's context.

The RAG architecture:

User Query + Code Context
    ↓
Hybrid Retrieval (Code search + Docs search)
    ↓
Re-ranking (Code relevance)
    ↓
Code-Aware LLM (GPT-4)
    ↓
Response + Code Snippets

The specific components:

1. Code-specific chunking:

# GitHub Copilot chunks code by function/class (not fixed-size)

# Naive fixed-size (BAD for code):
chunks_fixed = [code[i:i+500] for i in range(0, len(code), 500)]
# The problem: it cuts in the middle of a function

# Code-aware chunking (GOOD):
import ast

def chunk_by_functions(python_code: str) -> list[str]:
    """Splits code by function/class"""
    
    tree = ast.parse(python_code)
    chunks = []
    
    for node in ast.walk(tree):
        if isinstance(node, (ast.FunctionDef, ast.ClassDef)):
            chunk = ast.get_source_segment(python_code, node)
            chunks.append(chunk)
    
    return chunks

# Output: every chunk is a complete function
chunks = chunk_by_functions(python_code)

Why: code has to respect its structure (functions, classes). Fixed-size breaks the context.

Technique: Module 2 teaches chunking strategies, including structural.

2. Hybrid retrieval (code + docs):

# Copilot searches multiple sources simultaneously

query = "How do I parse JSON in Python?"

# Source 1: Python documentation
docs_results = search_docs(
    query=query,
    collection="python_stdlib_docs"
)

# Source 2: Project code (the user's repo)
code_results = search_code(
    query=query,
    collection=f"repo_{repo_id}_code"
)

# Source 3: Open source examples (public GitHub repos)
examples_results = search_examples(
    query=query,
    collection="github_public_python"
)

# Merge: docs first, then the project's code, then examples
merged = docs_results[:3] + code_results[:2] + examples_results[:2]

Why: developers need docs (concepts) + code examples (implementation) together.

3. Code-aware LLM:

# Copilot uses GPT-4 with a system prompt tailored to code

system_prompt = """
You are an expert programming assistant.

Rules:
1. Provide working code snippets (not pseudocode)
2. Explain what the code does
3. Include the necessary imports
4. Mention edge cases or common errors
5. If there are multiple ways, show the simplest one first
"""

user_prompt = f"""
Context (from the documentation and the user's code):
{merged_context}

Question: {query}

Answer with code:
"""

response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_prompt}
    ],
    temperature=0.2  # Low temperature for code (deterministic)
)

Why: code demands high precision. A low temperature avoids syntax hallucinations.

GitHub Copilot's metrics:

MetricTargetActualTechnique used
Latency P95<3,000ms~2,800msParallel retrieval (docs + code)
Code correctness>85%~87%Low temperature, code-aware prompts
Precision@5>75%~78%Code-specific chunking
User satisfaction>4.0/5~4.2/5Hybrid retrieval (docs + examples)

The lessons that apply to you:

✅ Domain-specific chunking: code needs to be chunked by function, not fixed-size
✅ Multi-source retrieval: docs + code + examples together > any one alone
✅ Low temperature for code: temperature 0.2 reduces syntax hallucinations
✅ Code-aware prompts: a tailored system prompt improves correctness +15%

💳 Case 4: Stripe Documentation Search

The problem:

A developer searches the Stripe docs: "How to create a subscription with trial period". The docs have 5,000+ pages (API reference, guides, examples).

The RAG architecture:

User Query
    ↓
Query Classification (API vs Guide vs Example)
    ↓
Hybrid Search (Keyword + Semantic)
    ↓
Metadata Filtering (language, API version)
    ↓
Section Re-ranking (API ref prioritized)
    ↓
Response + Direct Links

The specific components:

1. Query classification:

# Stripe classifies the query so it searches the right section

user_query = "How to create a subscription with trial?"

# Classify the query
query_type = classify_query(user_query)
# Output: "API" (not "guide" or "example")

# Search the specific section
if query_type == "API":
    search_collection = "stripe_api_reference"
elif query_type == "guide":
    search_collection = "stripe_guides"
else:
    search_collection = "stripe_examples"

results = collection[search_collection].query(...)

Why: the API reference has exact syntax. The guides have concepts. Classification boosts precision +25%.

2. Hybrid search (keywords are critical):

# Stripe uses hybrid search because its queries contain exact API names

query = "create subscription trial_period_days parameter"

# BM25 keyword search (excellent for "subscription", "trial_period_days")
keyword_results = bm25_search(query, collection="stripe_api")

# Semantic search (excellent for concepts like "trial")
semantic_results = semantic_search(query_embedding, collection="stripe_api")

# Reciprocal Rank Fusion (RRF)
merged = reciprocal_rank_fusion(keyword_results, semantic_results)

Why: API names ("trial_period_days") need an exact match (keyword). Concepts ("subscription trial") need semantic.

Technique: Module 5 teaches hybrid search with BM25 + embeddings.

3. Metadata filtering:

# Stripe filters by language and API version

# User context
user_language = "python"  # From the user's settings
api_version = "2024-01-01"  # Latest

# Filter the docs
results = collection.query(
    query_embeddings=[query_embedding],
    n_results=10,
    where={
        "language": user_language,
        "api_version": api_version
    }
)

# Output: only Python API v2024 docs

Why: Stripe has docs for 9 languages (Python, Ruby, Node, etc.). Without filtering, it hands Ruby code to a Python developer.

Stripe's metrics:

MetricTargetActualTechnique used
Latency P95<1,500ms~1,200msHybrid search (pre-indexed BM25)
Precision@3>90%~93%Query classification + hybrid
Zero-result rate<5%~3%Hybrid search (fallback to semantic)
User satisfaction>4.5/5~4.6/5Direct links + code snippets

The lessons that apply to you:

✅ Hybrid search for technical docs: API names need keywords + semantic
✅ Query classification: classifying before searching improves precision +25%
✅ Metadata filtering is critical: language + version filtering prevents confusion
✅ Direct links in the answer: direct links raise satisfaction

🤖 Case 5: ChatGPT Plugins (external knowledge)

The problem:

ChatGPT doesn't know recent information (its training data stops in 2023). Plugins let it search the web, databases, or external APIs.

The RAG architecture:

User Query
    ↓
Plugin Selection (which plugin do we use?)
    ↓
Plugin API Call (external retrieval)
    ↓
LLM Generation (GPT-4 with the plugin's context)
    ↓
Response

The specific components:

1. Plugin selection:

# ChatGPT decides which plugin to use based on the query

user_query = "What's the weather in SF?"

# GPT-4 analyzes the query and selects a plugin
plugin_selection_prompt = f"""
Query: {user_query}

Available plugins:
1. weather_plugin: Get current weather
2. web_search_plugin: Search the web
3. calculator_plugin: Perform calculations

Which plugin(s) should be used? Respond with plugin name only.
"""

selected_plugin = gpt4.invoke(plugin_selection_prompt)
# Output: "weather_plugin"

2. Plugin API call (RAG retrieval):

# The plugin performs the external retrieval

# Weather plugin example
def weather_plugin(location: str) -> dict:
    """External retrieval from a weather API"""
    
    response = requests.get(
        f"https://api.weather.com/current?location={location}"
    )
    
    return {
        "temperature": response.json()['temp'],
        "conditions": response.json()['conditions'],
        "humidity": response.json()['humidity']
    }

# ChatGPT calls the plugin
weather_data = weather_plugin("San Francisco")

# Output: {"temperature": 62, "conditions": "Cloudy", "humidity": 75}

3. LLM generation with the plugin's context:

# ChatGPT generates the answer using the plugin's data

context = f"""
Weather data from plugin:
- Location: San Francisco
- Temperature: {weather_data['temperature']}°F
- Conditions: {weather_data['conditions']}
- Humidity: {weather_data['humidity']}%
"""

prompt = f"""
{context}

User query: {user_query}

Respond naturally:
"""

response = gpt4.invoke(prompt)

# Output: "The weather in San Francisco is currently 62°F and cloudy, with 75% humidity."

ChatGPT Plugins' metrics:

MetricTargetActualTechnique used
Plugin selection accuracy>95%~96%GPT-4 function calling
Latency P95<4,000ms~3,500msParallel plugin calls
Faithfulness>95%~97%Direct API data (no hallucination)
Error rate<5%~4%Fallback to web search

The lessons that apply to you:

✅ External retrieval: RAG isn't only about internal docs, it can be external APIs
✅ Function calling: LLMs can decide which tool/plugin to use dynamically
✅ Parallel calls: multiple plugins can be called in parallel (-40% latency)
✅ Direct API data: reduces hallucinations vs scraping/parsing

📊 The architectures compared

CompanyPrimary UseChunkingRetrievalRe-rankingSpecial Feature
PerplexityWeb searchSemanticHybrid (BM25 + embeddings)✅ Cross-encoderQuery expansion
Notion AIPrivate docsFixed (page-based)Semantic❌ (metadata filter is enough)Time-weighted ranking
GitHub CopilotCode Q&AStructural (functions)Multi-source (docs+code)✅ Code relevanceLow temperature
StripeTechnical docsRecursiveHybrid (BM25 + embeddings)✅ Section priorityQuery classification
ChatGPT PluginsExternal APIsN/A (API calls)External APIs❌ (direct data)Function calling

🎯 General lessons for your RAG

Lesson 1: there is no single architecture

Every use case has different requirements:

  • Web search (Perplexity): precision is critical → re-ranking + query expansion
  • Private docs (Notion): privacy is critical → metadata filtering + workspace-scoped
  • Code Q&A (Copilot): correctness is critical → structural chunking + low temperature
  • Technical docs (Stripe): exact matches are critical → hybrid search + classification
  • External data (ChatGPT): freshness is critical → external APIs + function calling

Your decision: define your requirements first → then select your techniques.

Lesson 2: hybrid search is common

4 of the 5 cases use hybrid search (BM25 + embeddings):

  • Perplexity and Stripe use hybrid explicitly
  • Copilot combines docs (semantic) + code (keyword)
  • Only Notion uses pure semantic (because metadata filtering is enough)

What this means: Module 5 (Hybrid Search) is critical for production.

Lesson 3: re-ranking isn't always necessary

  • They use re-ranking: Perplexity, Copilot, Stripe (>90% precision is critical)
  • They don't: Notion (metadata filtering + recency is enough), ChatGPT (API data is already relevant)

The decision: re-rank if >85% precision is critical and your latency budget is >500ms.

Lesson 4: metadata filtering shrinks the search space dramatically

Notion and Stripe use metadata filtering aggressively:

  • Notion: workspace_id + date_range → -95% search space
  • Stripe: language + api_version → -80% search space

What this means: Module 6 (Metadata Filtering) is critical for multi-tenant and large datasets.

Lesson 5: domain-specific optimizations matter

  • Code: structural chunking, low temperature, code-aware prompts
  • Technical docs: query classification, hybrid search, direct links
  • Private workspaces: time-weighted ranking, privacy by design
  • Web search: query expansion, citations, cross-encoder re-ranking

What this means: generic techniques are the baseline. Domain-specific optimizations are where the real edge is.

🎯 Summary

Key concepts:

  • ✅ Perplexity: query expansion + hybrid + re-ranking → 92% precision for web search
  • ✅ Notion AI: metadata filtering + recency ranking → privacy + relevance in private docs
  • ✅ GitHub Copilot: structural chunking + multi-source + low temp → 87% correctness on code
  • ✅ Stripe: query classification + hybrid + metadata → 93% precision on technical docs
  • ✅ ChatGPT Plugins: external APIs + function calling → freshness + zero hallucinations
  • ✅ Hybrid search is the standard: 4 of the 5 cases use BM25 + embeddings
  • ✅ Re-rank when precision is critical: +200ms latency → +25% precision
  • ✅ Domain-specific optimizations: code, docs, private workspaces and web each have their own

What's next:

Capsule 06 compares RAG with traditional search (keyword search, SQL queries, grep) so you understand when RAG is the right tool and when it isn't.

📚 Additional resources

  1. Perplexity Architecture - The official architecture blog
  2. Notion AI Technical Deep Dive - How Notion AI works
  3. GitHub Copilot Explained - The official guide
  4. Stripe Developer Tools - Stripe's blog on their docs
  5. ChatGPT Plugins Architecture - The official documentation
  6. RAG in Production (LangChain) - Real cases from LangChain

Created: February 6, 2026
Version: 1.0

← PreviousSuccess Metrics in RAGNext →RAG vs Traditional Search (Keywords)