Module 3: Essential Features for RAG

Module 3: Essential Features for RAG

Next →

Module description

You already know WHY you need vector databases (Module 1) and HOW they work internally (Module 2). Now comes the question: WHAT features do you need for RAG in production?

The short answer: It's not just "search for similar vectors". A production-ready RAG system requires:

  • Metadata filtering (searching only in relevant documents)
  • Hybrid search (combining keyword + semantic)
  • Multi-tenancy (isolating data per user/company)
  • Batch operations (ingesting 1M documents efficiently)
  • Monitoring (detecting accuracy degradation)

This module explains WHAT features are critical and WHY, using your knowledge of internal architecture (Module 2) to understand the impact on performance.

By the end of this module, you'll be able to:

  • Identify essential vs nice-to-have features
  • Evaluate vector databases by features (ChromaDB vs Pinecone vs Weaviate)
  • Design a RAG architecture with the right features
  • Avoid common anti-patterns (e.g., searching everything without filters)

This module is 100% conceptual. Implementation comes in Module 4 (ChromaDB Hands-On).

🎯 Module objective

Professional objective:

Identify and justify essential vector database features for RAG systems in production, evaluating the impact on accuracy, performance, and costs.

Why does it matter?

Choosing a vector database WITHOUT evaluating features = risk of:

  • Costly re-architecture: "ChromaDB doesn't support native multi-tenancy. Migrating to Pinecone."
  • Degraded performance: "Search without a metadata filter = 500ms. With a filter = 50ms."
  • Low accuracy: "Pure semantic search = 75% accuracy. Hybrid search = 92%."

Analogy: Buying a car just for the engine (HNSW) without checking the features (AC, airbags, GPS). It works, but it isn't production-ready.

📚 Module content

Capsule 01: Module introduction (you're here)

  • Module objective and philosophy
  • Why features matter as much as algorithms
  • Module progression

Capsule 02: Metadata Filtering (Where Clauses)

  • What metadata filtering is (searching in a subset of documents)
  • Why it's critical for RAG (relevance, latency, costs)
  • Pre-filtering vs Post-filtering (trade-offs)
  • Operators: Equality, Range, Membership, Logical
  • Impact on performance (10x speedup typical)

Capsule 03: Hybrid Search (Keyword + Semantic)

  • What hybrid search is (combining BM25 + vector search)
  • Why pure semantic search fails on specific queries
  • Ranking strategies (RRF, weighted fusion)
  • When to use hybrid vs pure semantic
  • Impact on accuracy (75% → 92% typical)

Capsule 04: Multi-tenancy (Data isolation)

  • What multi-tenancy is (multiple users/companies in the same DB)
  • Strategies: Collection per tenant, Metadata filtering, Namespace
  • Security considerations (data leakage prevention)
  • Performance trade-offs (horizontal vs vertical scale)
  • When it's critical (SaaS, enterprise RAG)

Capsule 05: Batch Operations (Efficient ingestion)

  • Why single insert is inefficient (1M inserts × 10ms = 2.7 hours)
  • Batch insert strategies (optimal batch size)
  • Bulk updates and deletes
  • Rebuild index strategies (incremental vs full)
  • Trade-offs: Throughput vs Latency

Capsule 06: Distance Metrics (Beyond cosine)

  • Cosine similarity (default for text embeddings)
  • Euclidean distance (L2)
  • Dot product
  • When to use each one (normalized vs non-normalized embeddings)
  • Impact on accuracy (2-5% typically)

Capsule 07: Observability and Monitoring

  • What to monitor in production RAG
  • Key metrics: Latency (p50, p95, p99), Throughput, Accuracy
  • Detecting accuracy degradation (embeddings drift)
  • Debugging slow queries
  • Alerting strategies

Capsule 08: Nice-to-Have vs Critical Features

  • Feature checklist by RAG scenario
  • Comparison: ChromaDB vs Pinecone vs Weaviate vs Qdrant
  • Decision matrix (features × requirements)
  • Common anti-patterns and how to avoid them
  • Summary and transition to Module 4

🔗 Connection with other modules

Prerequisites:

  • Module 1: Why Vector DBs (need justified)
  • Module 2: How they work (internal architecture)

This module prepares you for:

  • Module 4: ChromaDB Setup (you'll implement the features learned here)
  • Module 5: ChromaDB + full RAG (you'll use metadata filtering, hybrid search)
  • Modules 6-8: Production (you'll evaluate databases by features)

Recommended flow:

Module 1: Why Vector DBs
  ↓
Module 2: How they work (HNSW, IVF, PQ)
  ↓
Module 3: WHAT features you need ← You're here
  ↓
Module 4-5: Implement with ChromaDB
  ↓
Module 6-8: Production considerations

⏱️ Estimated time

Reading and comprehension: 60-75 minutes

Breakdown per capsule:

  • Capsule 01: 5 min (introduction)
  • Capsule 02: 10-12 min (metadata filtering, in depth)
  • Capsule 03: 10-12 min (hybrid search)
  • Capsule 04: 8-10 min (multi-tenancy)
  • Capsule 05: 8-10 min (batch operations)
  • Capsule 06: 6-8 min (distance metrics)
  • Capsule 07: 8-10 min (monitoring)
  • Capsule 08: 8-10 min (comparison and summary)

Total: 63-82 minutes

Note: This module is 100% conceptual (no code). Implementation in Module 4.

🎓 What will you learn in this module?

By the end of this module, you'll be able to:

1. Identify critical features for RAG

  • ✅ Metadata filtering (reduce search space 10x)
  • ✅ Hybrid search (accuracy 75% → 92%)
  • ✅ Multi-tenancy (SaaS/enterprise)
  • ✅ Batch operations (efficient ingestion)
  • ✅ Monitoring (detect degradation)

2. Evaluate vector databases by features

  • ✅ ChromaDB: Metadata filtering ✅, Hybrid search ❌, Multi-tenancy ⚠️
  • ✅ Pinecone: Everything ✅, but managed (cost)
  • ✅ Weaviate: Native hybrid search ✅, self-hosted
  • ✅ Qdrant: Advanced features ✅, self-hosted

3. Design a RAG architecture correctly

  • ✅ Use metadata filtering for relevance
  • ✅ Implement hybrid search when needed
  • ✅ Configure multi-tenancy according to scale
  • ✅ Optimize batch ingestion

4. Avoid anti-patterns

  • ❌ Searching everything without metadata filtering
  • ❌ Pure semantic search on specific queries
  • ❌ Single inserts for 1M documents
  • ❌ Not monitoring accuracy degradation

💡 Module philosophy

Why features > algorithms in production

You might ask yourself: "I already know HNSW is better than IVF. Why do I need to know features?"

Answer: The right algorithm + the wrong features = an inefficient system.

Real example:

Scenario: RAG chatbot with 500K documents
Algorithm: HNSW (98% accuracy)

Without metadata filtering:
- Query: "Password reset instructions"
- Searches in: 500K documents (all)
- Latency: 200ms
- Accuracy: 85% (noise from irrelevant docs)

With metadata filtering:
- Query: "Password reset instructions"
- Searches in: 10K documents (category='support')
- Latency: 20ms (10x faster)
- Accuracy: 95% (only relevant docs)

Trade-off: The perfect algorithm (HNSW) + a critical feature (filtering) = a 10x better system.

Key differentiator vs the competition

90% of tutorials:

  • Show only "search for similar vectors"
  • Don't cover metadata filtering, hybrid search, multi-tenancy
  • Result: The MVP works, but it isn't production-ready

This module:

  • Covers essential features for production
  • Explains WHY each feature matters (with benchmarks)
  • Decision framework: When to use each feature

🚫 What this module does NOT cover

This module does NOT cover:

❌ Feature implementation (that's Module 4)

  • You won't see metadata filtering code
  • You won't implement hybrid search
  • Only concepts and decisions

❌ Advanced edge-case features (that's Modules 7-8)

  • Doesn't cover distributed sharding
  • Doesn't cover custom distance metrics
  • Only essential 80/20 features

❌ Exhaustive DB comparison (that's Module 6)

  • Only an overview of the main features
  • The deep comparison comes later

Clear scope: This module is about WHAT features you need and WHY, not HOW to implement them.

✅ Success criteria

You successfully completed this module when:

You can answer these questions:

  1. ✅ What is metadata filtering and why is it critical for RAG?

    • Answer: Searching in a subset of documents using where clauses. It reduces latency 10x and improves relevance by filtering out irrelevant docs.

  2. ✅ When to use hybrid search vs pure semantic search?

    • Answer: Hybrid when the query is specific (proper nouns, IDs, exact dates). Pure semantic when the query is conceptual/ambiguous.

  3. ✅ What multi-tenancy strategy for a SaaS with 1000 clients?

    • Answer: Metadata filtering (tenant_id field) for <10K clients. Collection per tenant for >10K or strict isolation.

  4. ✅ Why is batch insert critical for 1M documents?

    • Answer: Single insert = 2.7 hours (1M × 10ms). Batch insert (1000 batch size) = 16 minutes (10x faster).

  5. ✅ What to monitor in RAG production?

    • Answer: Latency (p95 <500ms), Accuracy (>90%), Throughput (queries/sec), Index health (rebuild frequency).

If you answered 4-5/5 correctly → ✅ Ready for Module 4 (ChromaDB Hands-On)

🎯 Skills you'll develop

This module develops RAG systems architecture skills, not programming.

Evaluation skills:

  1. Feature assessment - Identify critical vs nice-to-have features
  2. Database comparison - Evaluate ChromaDB vs Pinecone vs Weaviate by features
  3. Trade-off analysis - Managed vs self-hosted, features vs cost

Design skills:

  1. RAG architecture - Design with metadata filtering, hybrid search, multi-tenancy
  2. Performance optimization - Reduce latency with filtering, batch operations
  3. Scalability planning - Multi-tenancy strategy, batch ingestion

Prevention skills:

  1. Anti-pattern detection - Identify inefficient designs
  2. Risk assessment - Evaluate the impact of missing features
  3. Migration planning - When to migrate DB based on features

📖 How to use this module

Recommended strategy:

  1. Read sequentially (Capsules 01 → 02 → ... → 08)

    • Metadata filtering first (the most critical feature)
    • Then hybrid search and multi-tenancy
    • Finally comparison and decisions

  2. Connect with your use case

    • "Does my RAG need multi-tenancy?"
    • "Would hybrid search improve my accuracy?"
    • "How do I optimize ingestion of 500K docs?"

  3. Use the decision checklists (Capsule 08)

    • Evaluate critical features for your scenario
    • Compare databases against the checklist
    • Make an informed decision

Suggested time:

Option A: One session (60-75 min)

  • Read it all in one go
  • Advantage: Complete context, clear comparisons

Option B: Two sessions

  • Session 1: Capsules 01-04 (core features: filtering, hybrid, multi-tenancy)
  • Session 2: Capsules 05-08 (operations: batch, monitoring, comparison)

Recommendation: Option A (the content is complementary, better in one session).

🔗 Resources for this module

Official documentation:

  1. ChromaDB Features - Metadata filtering, collections
  2. Pinecone Features - Namespaces, hybrid search
  3. Weaviate Hybrid Search - BM25 + vector

Technical articles:

  1. "Metadata Filtering in Vector Databases" - Pinecone blog
  2. "Hybrid Search Explained" - Weaviate blog
  3. "Multi-tenancy Strategies" - Qdrant blog

Benchmarks:

  1. Vector Database Benchmarks - Performance comparisons
  2. Hybrid Search Accuracy - RRF vs weighted

Note: These resources are for going deeper AFTER the module. The capsules are self-contained.

🚀 Ready to start?

Next step:

Go to Capsule 02: Metadata Filtering (Where Clauses)

There you'll learn:

  1. What metadata filtering is (searching in a subset)
  2. Pre-filtering vs Post-filtering (trade-offs)
  3. Operators (equality, range, membership, logical)
  4. Impact on performance (real 10x speedup)

This is the most critical feature for RAG in production. Literally a 10x difference in latency and accuracy.

Reading time: 5 minutes
Next: 02-metadata-filtering.md

← PreviousCapsule 08: Why this matters for RAG - Module SummaryNext →Capsule 02: Metadata Filtering (Where Clauses)