Module 3: Essential Features for RAG

Capsule 02: Metadata Filtering (Where Clauses)

Understand WHAT metadata filtering is, why it's the most critical feature for RAG in production, and how it reduces latency 10x while improving accuracy.

By the end of this capsule:

✅ You'll explain what metadata filtering is and why it's essential
✅ You'll compare pre-filtering vs post-filtering (trade-offs)
✅ You'll identify the key operators (equality, range, membership, logical)
✅ You'll calculate the impact on performance (latency, accuracy, costs)

Estimated time: 10-12 minutes

🔍 What is Metadata Filtering?

Definition

Metadata filtering is the ability to search for similar vectors ONLY in a subset of documents that meet specific conditions (where clauses).

Without metadata filtering (naive search)

# ❌ Search across the WHOLE database
query = "How do I reset my password?"
results = db.query(
    query_embedding=embed(query),
    k=10
)
# Searches in: 500K documents (all)
# Latency: 200ms
# Accuracy: 85% (lots of noise)

Problem: It searches in irrelevant documents (marketing, legal, product specs).

With metadata filtering (smart search)

# ✅ Search in the relevant subset
query = "How do I reset my password?"
results = db.query(
    query_embedding=embed(query),
    k=10,
    where={"category": "support", "language": "en"}
)
# Searches in: 10K documents (only support docs in English)
# Latency: 20ms (10x faster)
# Accuracy: 95% (no noise)

Advantage: Reduced search space → Lower latency + Higher accuracy.

📊 Why it's critical for RAG

Problem 1: Relevance (Accuracy)

Scenario: E-commerce RAG with 1M products

Query: "Nike Air Max price"

Without filtering:

Top 10 results:
1. Nike Air Max (product page) ✅
2. Adidas similar shoe (product page) ❌
3. Nike history article ❌
4. Air conditioning product ❌ (false positive: "air")
5. Max brand electronics ❌ (false positive: "max")
...

Accuracy: 60% (4/10 irrelevant)

With filtering (category='shoes'):

Top 10 results:
1. Nike Air Max (product page) ✅
2. Nike Air Max variants ✅
3. Similar Nike shoes ✅
...

Accuracy: 95% (9/10 relevant)

Gain: 35% accuracy improvement.

Problem 2: Latency (Performance)

Benchmark (1M vectors, 1536-dim, HNSW):

Scenario	Search Space	Latency	vs Baseline
Without filtering	1M vectors	180 ms	1x
Filtering (10% data)	100K vectors	18 ms	10x faster
Filtering (1% data)	10K vectors	2 ms	90x faster

Key: Latency is proportional to search space. Filtering reduces search space dramatically.

Problem 3: Costs (Compute)

With a Layer 1 (HNSW) architecture:

Without filtering: HNSW navigates 1M vectors → CPU intensive
With filtering: HNSW navigates 10K vectors → 100x less CPU

Impact on cloud:

AWS cost (r6i.xlarge, $180/month):
- Without filtering: CPU usage 80% → Need a larger instance ($360/month)
- With filtering: CPU usage 10% → Can downsize ($90/month)

Savings: $270/month = $3240/year

🏗️ Architecture: Pre-filtering vs Post-filtering

Pre-filtering (Filter-then-search)

Process:

Apply the metadata filter first (SQL-like where clause)
Search for similar vectors ONLY in the filtered subset
Return top-k

Advantage:

✅ Faster (reduced search space)
✅ Less CPU (doesn't compute similarity on irrelevant docs)

Disadvantage:

❌ If the subset is empty → Returns no results
❌ Requires a metadata index (overhead)

Used by: Pinecone, Weaviate (default)

Visualization:

1M documents
    ↓
[Filter: category='support'] → 10K documents
    ↓
[Vector search HNSW] → Top 10 results

Post-filtering (Search-then-filter)

Process:

Search for similar vectors across the WHOLE database
Apply the metadata filter to the results
If <k results, search for more vectors
Return top-k

Advantage:

✅ Always returns k results (doesn't fail if the subset is empty)
✅ Doesn't require a metadata index

Disadvantage:

❌ Slower (searches everything first)
❌ More CPU (computes similarity on docs it later filters out)

Used by: ChromaDB (default)

Visualization:

1M documents
    ↓
[Vector search HNSW] → Top 100 candidates
    ↓
[Filter: category='support'] → Top 10 filtered results

Comparison: Pre vs Post

Dimension	Pre-filtering	Post-filtering
Latency	20ms ✅	50ms ⚠️
CPU	Low ✅	High ❌
k results guarantee	No ❌	Yes ✅
Requires metadata index	Yes ⚠️	No ✅
Best for	High cardinality filters	Low cardinality filters

Decision:

Pre-filtering if the filter is selective (reduces >50% of data)
Post-filtering if the filter is not very selective (<10% reduction)

🔧 Metadata Filtering Operators

1. Equality (Exact match)

# Single value
where={"category": "support"}

# Searches docs where category == "support"
# Use: Categories, IDs, status

RAG example: Search only in technical documentation (not marketing).

2. Range (Comparisons)

# Greater than, less than
where={
    "timestamp": {"$gt": 1640000000},  # After 2021-12-20
    "price": {"$lt": 100}               # Less than $100
}

# Operators: $gt, $gte, $lt, $lte, $ne

RAG example: Only recent documents (last 30 days).

3. Membership (IN operator)

# Multiple values
where={
    "category": {"$in": ["support", "faq", "docs"]}
}

# Searches docs where category IN ["support", "faq", "docs"]

RAG example: Search across multiple relevant categories.

4. Logical operators (AND, OR, NOT)

# AND (implicit with multiple conditions)
where={
    "category": "support",
    "language": "en",
    "status": "published"
}

# OR
where={
    "$or": [
        {"category": "support"},
        {"category": "faq"}
    ]
}

# NOT
where={
    "category": {"$ne": "archived"}
}

# Combined (complex)
where={
    "$and": [
        {"category": {"$in": ["support", "faq"]}},
        {"language": "en"},
        {"timestamp": {"$gt": 1640000000}}
    ]
}

RAG example: Support documents IN ENGLISH, NOT archived, FROM the last 30 days.

📊 Benchmark: Real impact of Metadata Filtering

Scenario: Customer Support RAG

Setup:

Database: 500K documents
- Support articles: 50K (10%)
- Marketing content: 200K (40%)
- Product specs: 150K (30%)
- Legal/HR docs: 100K (20%)
Algorithm: HNSW (ChromaDB)
Hardware: r6i.xlarge (32 GB RAM)

Query: "How to reset password?"

Test A: Without metadata filtering

results = db.query(
    query_embedding=embed("How to reset password?"),
    k=10
)

Results:

Search space: 500K docs
Latency: 210ms
Top 10 accuracy: 70% (7/10 relevant)
- 3 false positives (marketing with a "password" mention)
CPU usage: 60%

Test B: With metadata filtering (category='support')

results = db.query(
    query_embedding=embed("How to reset password?"),
    k=10,
    where={"category": "support"}
)

Results:

Search space: 50K docs (10x reduction)
Latency: 22ms (9.5x faster)
Top 10 accuracy: 95% (9.5/10 relevant)
CPU usage: 8%

Gain:

9.5x latency reduction
25% accuracy improvement
7.5x CPU reduction

Test C: With multiple filters (category + language + recency)

results = db.query(
    query_embedding=embed("How to reset password?"),
    k=10,
    where={
        "category": "support",
        "language": "en",
        "timestamp": {"$gt": time.now() - 90_days}
    }
)

Results:

Search space: 8K docs (62x reduction)
Latency: 4ms (52x faster)
Top 10 accuracy: 98% (9.8/10 relevant)
CPU usage: 2%

Gain:

52x latency reduction
28% accuracy improvement
30x CPU reduction

Conclusion: More filters = More speedup + Higher accuracy (if the filters are relevant).

🎯 Metadata Filtering Strategies for RAG

Strategy 1: Categorical filtering

Use: Multi-domain RAG (support + sales + product)

# User selects a category in the UI
category = user_input  # "support"

results = db.query(
    query_embedding=embed(query),
    where={"category": category}
)

Advantage: The user controls the scope explicitly.

Strategy 2: Recency filtering

Use: Technical documentation (prioritize recent versions)

# Only documents from the last 90 days
cutoff = time.now() - 90_days

results = db.query(
    query_embedding=embed(query),
    where={"timestamp": {"$gte": cutoff}}
)

Advantage: Avoids obsolete documentation.

Strategy 3: Multi-tenant filtering

Use: SaaS RAG (isolate data per customer)

# Each tenant has its own namespace
results = db.query(
    query_embedding=embed(query),
    where={"tenant_id": current_user.tenant_id}
)

Advantage: Security + performance (search in a subset).

Strategy 4: Hierarchical filtering

Use: Documents with a hierarchy (department → team → project)

# Search in an increasingly broad scope if there are no results
scopes = [
    {"department": "engineering", "team": "backend", "project": "api"},
    {"department": "engineering", "team": "backend"},
    {"department": "engineering"},
    {}  # Fallback: search everything
]

for scope in scopes:
    results = db.query(
        query_embedding=embed(query),
        where=scope,
        k=10
    )
    if len(results) >= 5:  # Threshold
        break

Advantage: Balance between relevance (narrow scope) and coverage (broad scope).

🚫 Common anti-patterns

Anti-pattern 1: Not using metadata filtering

# ❌ BAD: Searching everything without filters
results = db.query(query_embedding=embed(query), k=10)

# ✅ GOOD: Filter by relevant context
results = db.query(
    query_embedding=embed(query),
    where={"category": "support"},
    k=10
)

Impact: 10x latency + 25% accuracy loss.

Anti-pattern 2: Over-filtering (empty subset)

# ❌ BAD: Very specific filters → 0 results
results = db.query(
    query_embedding=embed(query),
    where={
        "category": "support",
        "subcategory": "password",
        "language": "en",
        "region": "US-West",
        "version": "2.3.1"
    },
    k=10
)
# Result: 0 docs match → Returns nothing

Fix: Use hierarchical filtering (try with broader filters).

Anti-pattern 3: Filtering in the application layer (not in the DB)

# ❌ BAD: Fetch everything, filter in Python
all_results = db.query(query_embedding=embed(query), k=1000)
filtered = [r for r in all_results if r.metadata['category'] == 'support']
top_10 = filtered[:10]

# ✅ GOOD: Filter in the DB
results = db.query(
    query_embedding=embed(query),
    where={"category": "support"},
    k=10
)

Impact: 10-50x latency (transferring 1000 docs vs 10), wasted CPU.

✅ Comprehension checklist

Verify that you understood this capsule:

What is metadata filtering?
- Answer: Searching for similar vectors ONLY in a subset of docs that meet where clauses (e.g., category='support').
What's the difference between pre-filtering and post-filtering?
- Answer: Pre-filtering filters first (faster), post-filtering searches first (guarantees k results).
What's the typical impact of metadata filtering on latency?
- Answer: 10x speedup typical (200ms → 20ms) when the filter reduces search space by 90%.
What filtering operators exist?
- Answer: Equality (==), Range ($gt, $lt), Membership ($in), Logical ($and, $or, $not).
When NOT to use metadata filtering?
- Answer: When you don't have relevant metadata, or when the query is very general (searching everything is correct).

If you answered 4-5/5 correctly → ✅ Ready for Capsule 03 (Hybrid Search)

🔗 Connection with RAG

How does this improve your RAG system?

Case A: Customer Support Chatbot

Without filtering:

User: "How do I reset my password?"
RAG searches in: 500K docs (support + marketing + legal + product)
Latency: 200ms
LLM receives: 3 support docs + 7 marketing docs with "password"
Response quality: 70% (contaminated context)

With filtering:

User: "How do I reset my password?"
RAG searches in: 50K docs (only support)
Latency: 20ms (10x faster)
LLM receives: 10 relevant support docs
Response quality: 95%

Gain: 10x latency + 25% quality improvement.

Case B: Multi-tenant SaaS RAG

Without filtering (INSECURE):

Tenant A query: "Show me sales data"
RAG searches in: All tenants (data leakage risk!)

With filtering (SECURE):

Tenant A query: "Show me sales data"
RAG searches in: where={"tenant_id": "tenant_a"}
Isolation guaranteed

Critical: Security compliance (GDPR, SOC2).

🚀 Next step

Metadata filtering is the #1 most critical feature. Now you'll learn feature #2: Hybrid Search.

Next capsule: 03 - Hybrid Search (Keyword + Semantic)

You'll learn:

Why pure semantic search fails on specific queries
How to combine BM25 (keyword) + vector search (semantic)
Ranking strategies (RRF, weighted fusion)
Impact on accuracy (75% → 92%)

Key: Metadata filtering reduces the search space. Hybrid search improves the ranking within that space.

Reading time: 10-12 minutes
Next: 03-hybrid-search.md

← PreviousModule 3: Essential Features for RAG Next →Capsule 03: Hybrid Search (Keyword + Semantic)

Capsule 02: Metadata Filtering (Where Clauses)

🎯 Capsule objective

🔍 What is Metadata Filtering?

Definition

Without metadata filtering (naive search)

With metadata filtering (smart search)

📊 Why it's critical for RAG

Problem 1: Relevance (Accuracy)

Problem 2: Latency (Performance)

Problem 3: Costs (Compute)

🏗️ Architecture: Pre-filtering vs Post-filtering

Pre-filtering (Filter-then-search)

Post-filtering (Search-then-filter)

Comparison: Pre vs Post

🔧 Metadata Filtering Operators

1. Equality (Exact match)

2. Range (Comparisons)

3. Membership (IN operator)

4. Logical operators (AND, OR, NOT)

📊 Benchmark: Real impact of Metadata Filtering

Scenario: Customer Support RAG

Test A: Without metadata filtering

Test B: With metadata filtering (category='support')

Test C: With multiple filters (category + language + recency)

🎯 Metadata Filtering Strategies for RAG

Strategy 1: Categorical filtering

Strategy 2: Recency filtering

Strategy 3: Multi-tenant filtering

Strategy 4: Hierarchical filtering

🚫 Common anti-patterns

Anti-pattern 1: Not using metadata filtering

Anti-pattern 2: Over-filtering (empty subset)

Anti-pattern 3: Filtering in the application layer (not in the DB)

✅ Comprehension checklist

🔗 Connection with RAG

Case A: Customer Support Chatbot

Case B: Multi-tenant SaaS RAG

🚀 Next step