Module 3: Query Optimization

Introduction to Query Optimization

Next →

Capsule overview

Query optimization transforms the user's query before searching, to raise recall (finding more relevant docs) and precision (more specific docs). Direct queries fail often: "fastapi auth" is ambiguous (OAuth2? JWT? Basic auth?), and "how to do X" is badly phrased for semantic search.

This module teaches 4 techniques that improve recall by +15-25%: (1) Query Expansion (generating related queries), (2) Query Rewriting (reformulating for clarity), (3) Query Decomposition (splitting complex ones), (4) HyDE (generating a hypothetical document). Each technique has its own trade-offs in latency, cost, and gain.

🎯 Learning objectives

By the end of this module, you'll be able to:

  1. ✅ Explain why direct queries are suboptimal (-20-30% recall)
  2. ✅ Implement query expansion with an LLM (+20-30% recall)
  3. ✅ Implement query rewriting (+10-15% precision)
  4. ✅ Implement query decomposition (+15-20% precision on complex queries)
  5. ✅ Implement HyDE (+15-25% recall)
  6. ✅ Compare the techniques with benchmarks
  7. ✅ Select the optimal technique for a given query type
  8. ✅ Integrate it into the evolving project (+15-25% total recall)

📐 Why query optimization matters

The problem: direct queries are ambiguous/incomplete

# The user's direct query
user_query = "fastapi auth"

# The problems:
# 1. Ambiguity: OAuth2? JWT? Basic auth? API keys?
# 2. Missing context: an implementation? a tutorial? best practices?
# 3. Keyword style: "fastapi auth" vs "How to implement authentication in FastAPI?"

# Semantic search with the direct query
results = collection.query(
    query_embeddings=[create_embedding("fastapi auth")],
    n_results=5
)

# Recall: 52% (it misses many relevant docs)
# Precision: 68% (some docs aren't specifically about auth)

The solution: query optimization

# Technique 1: Query Expansion
expanded_queries = [
    "How to implement OAuth2 authentication in FastAPI",
    "FastAPI JWT token authentication tutorial",
    "FastAPI security and authentication best practices",
    "Implementing API key authentication in FastAPI"
]

# Search with all of them and merge the results
all_results = []
for query in expanded_queries:
    results = collection.query(
        query_embeddings=[create_embedding(query)],
        n_results=10
    )
    all_results.extend(results['documents'][0])

# Deduplicate and rank
final_results = reciprocal_rank_fusion(all_results)[:5]

# Recall: 77% (+25% vs the direct query)
# Precision: 78% (+10% vs the direct query)

🗺️ The module's roadmap

Capsule 01 (this one): Introduction

  • Why direct queries fail
  • An overview of the 4 techniques
  • Technical setup

Capsule 02: The problems with direct queries

  • Ambiguity, incompleteness, bad phrasing
  • The impact on recall: -20-30%

Capsule 03: Query expansion

  • Generating multiple queries with an LLM
  • Merging with reciprocal rank fusion
  • The gain: +20-30% recall

Capsule 04: Query rewriting

  • Reformulating for clarity
  • The cases: typos, clarification, natural language
  • The gain: +10-15% precision

Capsule 05: Query decomposition

  • Splitting complex queries
  • Multi-hop reasoning
  • The gain: +15-20% precision

Capsule 06: HyDE

  • Generating a hypothetical document
  • Searching with the doc's embedding (not the query's)
  • The gain: +15-25% recall

Capsule 07: The techniques compared

  • A complete benchmark
  • The trade-offs: recall vs latency vs cost
  • The decision matrix

Capsule 08: Project - Query Optimizer

  • Implement all 4 techniques
  • Compare them and select one
  • Integrate it into the RAG

📊 An overview of the query optimization techniques

Technique 1: Query expansion

The concept: generate several related queries and search with all of them.

# Input
user_query = "fastapi auth"

# Query expansion (LLM)
expanded = [
    "How to implement OAuth2 authentication in FastAPI",
    "FastAPI JWT token authentication",
    "FastAPI security best practices"
]

# Search with all of them → merge the results
# The gain: +20-30% recall

Capsule: 03

Technique 2: Query rewriting

The concept: reformulate the query for clarity and completeness.

# Input (ambiguous)
user_query = "fastapi auth"

# Query rewriting (LLM)
rewritten = "How to implement authentication in FastAPI using OAuth2 or JWT tokens"

# Search with the rewritten (clearer) query
# The gain: +10-15% precision

Capsule: 04

Technique 3: Query decomposition

The concept: split a complex query into simple sub-queries.

# Input (complex)
user_query = "Compare FastAPI and Flask authentication approaches"

# Decomposition (LLM)
sub_queries = [
    "How does authentication work in FastAPI?",
    "How does authentication work in Flask?",
    "What are the differences between FastAPI and Flask?"
]

# Search each sub-query → aggregate the results
# The gain: +15-20% precision on complex queries

Capsule: 05

Technique 4: HyDE (Hypothetical Document Embeddings)

The concept: generate a hypothetical document that would answer the query, then search with its embedding.

# Input
user_query = "How to implement authentication in FastAPI?"

# Generate a hypothetical document (LLM)
hypothetical_doc = """
To implement authentication in FastAPI, you can use OAuth2 with JWT tokens.
First, install python-jose and passlib libraries.
Then, create a User model with password hashing...
"""

# Search with the document's embedding (not the query's)
doc_embedding = create_embedding(hypothetical_doc)
results = collection.query(query_embeddings=[doc_embedding])

# The gain: +15-25% recall (doc embeddings > query embeddings)

Capsule: 06

🛠️ Technical setup

No new installation required:

# We already have the OpenAI API (Module 1)
# We already have ChromaDB (Module 1)
# We already have LangChain (Module 2)

# Verify the setup
python -c "from openai import OpenAI; print('✅ OpenAI ready')"
python -c "import chromadb; print('✅ ChromaDB ready')"

Configuration:

# .env (it already exists from Module 1)
OPENAI_API_KEY=sk-proj-...

# Use GPT-3.5-turbo for query optimization (a good cost/quality balance)

📊 The gain you should expect from each technique

Baseline (direct queries):

# No query optimization
user_query = "fastapi auth"
results = rag_system.query(user_query)

# Baseline metrics:
# - Recall@50: 52%
# - Precision@5: 78% (from Module 2, with optimized chunking)

Query expansion (+20-30% recall):

# With expansion
expanded_queries = expand_query(user_query)  # Generates 3-5 queries
results = search_with_multiple_queries(expanded_queries)

# The expected gain:
# - Recall@50: 67% (+15% vs baseline)
# - Precision@5: 80% (+2%)
# - Latency: +600ms (an LLM call + multiple searches)
# - Cost: +$0.001/query

Query rewriting (+10-15% precision):

# With rewriting
rewritten_query = rewrite_query(user_query)  # It clarifies
results = rag_system.query(rewritten_query)

# The expected gain:
# - Recall@50: 55% (+3%)
# - Precision@5: 88% (+10% vs baseline)
# - Latency: +500ms (an LLM call)
# - Cost: +$0.0008/query

HyDE (+15-25% recall):

# With HyDE
hypothetical_doc = generate_hypothetical_document(user_query)
doc_embedding = create_embedding(hypothetical_doc)
results = collection.query(query_embeddings=[doc_embedding])

# The expected gain:
# - Recall@50: 70% (+18% vs baseline)
# - Precision@5: 82% (+4%)
# - Latency: +700ms (LLM generation + the embedding)
# - Cost: +$0.0015/query

🔗 Connection with the evolving project

Module 1 (baseline):

# retrieval.py (Module 1)
def retrieve(query: str, top_k: int = 5):
    """A direct query, no optimization"""
    query_embedding = create_embedding(query)
    return collection.query(query_embeddings=[query_embedding], n_results=top_k)

The metrics: recall 52%

Module 2 (optimized chunking):

# indexing.py (Module 2)
# Better chunking → better doc quality → precision 78%

The metrics: precision 78% (+10%), recall 57% (+5%)

Module 3 (query optimization):

# retrieval.py (Module 3 - updated)
def retrieve(query: str, top_k: int = 5, optimize_query: bool = True):
    """Retrieval with query optimization"""
    
    if optimize_query:
        # Query expansion
        expanded_queries = expand_query(query)
        
        # Search with all of the queries
        all_results = []
        for q in expanded_queries:
            q_embedding = create_embedding(q)
            results = collection.query(query_embeddings=[q_embedding], n_results=10)
            all_results.append(results)
        
        # Merge with reciprocal rank fusion
        final_results = reciprocal_rank_fusion(all_results)[:top_k]
        return final_results
    else:
        # Fall back to the direct query
        query_embedding = create_embedding(query)
        return collection.query(query_embeddings=[query_embedding], n_results=top_k)

The optimized metrics: precision 80% (+2%), recall 72% (+15%)

Module 8 (final):

# advanced_rag_system.py (Module 8)
class AdvancedRAGSystem:
    def __init__(self, query_optimization: str = "expansion"):
        # Module 2: optimized chunking
        self.chunker = RecursiveChunker()
        
        # Module 3: query optimization
        if query_optimization == "expansion":
            self.query_optimizer = QueryExpansion()
        elif query_optimization == "hyde":
            self.query_optimizer = HyDE()
        # ... modules 4-7

The final metrics (accumulated): precision 93%, recall 77%

🎯 Summary

Key concepts:

  • ✅ Direct queries are suboptimal: ambiguous, incomplete, badly phrased (-20-30% recall)
  • ✅ 4 optimization techniques: expansion (recall), rewriting (precision), decomposition (complex queries), HyDE (maximum recall)
  • ✅ The trade-offs: latency (+500-800ms), cost (+$0.001-0.002/query), recall (+15-30%)
  • ✅ The combined gain: +15-25% recall typically, +10-15% precision
  • ✅ The decision depends on: the query type, your latency budget, your cost constraints

What's next:

Capsule 02 breaks down the specific problems with direct queries, with concrete examples and impact metrics.

📚 Additional resources

  1. Query Understanding for RAG - The research paper
  2. Multi-Query Retrieval - The LangChain docs
  3. HyDE Original Paper - Hypothetical Document Embeddings
  4. Query Expansion Techniques - Pinecone's guide
  5. Query Decomposition - The LlamaIndex blog

Created: February 6, 2026
Version: 1.0

← PreviousProject: Chunking Strategy OptimizerNext →Capsule 02: Why the user's query is almost never the optimal query