Capsule 08: Features Comparison and Module Summary

🎯 Objective

Consolidate the Module 3 learnings with a feature checklist by scenario, a comparison of vector databases, and a decision matrix for choosing the right DB.

Estimated time: 8-10 minutes

---

📋 Feature Checklist by Scenario

Scenario A: RAG Chatbot (Customer-facing)

Critical features:

• ✅ Metadata filtering (category, language)
• ✅ Low latency (<500ms)
• ⚠️ Hybrid search (if specific queries)
• ❌ Multi-tenancy (single customer)
• ⚠️ Batch operations (MVP can use single inserts)
• ✅ Monitoring (accuracy is critical)

Recommended database: ChromaDB (MVP) → Pinecone (scale)

Scenario B: RAG SaaS (Multi-tenant)

Critical features:

• ✅ Metadata filtering
• ✅ Multi-tenancy (metadata or collection strategy)
• ✅ Security/isolation
• ✅ Batch operations (customer onboarding)
• ✅ Monitoring (SLA)
• ⚠️ Hybrid search (depending on queries)

Recommended database: Pinecone (managed) or Weaviate (self-hosted)

Scenario C: Internal Knowledge Base

Critical features:

• ✅ Metadata filtering
• ✅ Hybrid search (technical queries)
• ✅ Batch operations (bulk ingestion)
• ❌ Multi-tenancy (single org)
• ⚠️ Low latency (not critical)
• ⚠️ Monitoring (basic metrics OK)

Recommended database: ChromaDB (self-hosted) or Weaviate

Scenario D: E-commerce Search

Critical features:

• ✅ Metadata filtering (category, price, availability)
• ✅ Hybrid search (exact product names)
• ✅ Batch operations (catalog updates)
• ⚠️ Multi-tenancy (if a marketplace)
• ✅ High throughput (1000+ QPS)
• ✅ Monitoring

Recommended database: Weaviate (native hybrid) or Elasticsearch + vector plugin

---

🗂️ Comparison: ChromaDB vs Pinecone vs Weaviate vs Qdrant

Feature	ChromaDB	Pinecone	Weaviate	Qdrant
Metadata Filtering	✅ Post-filter	✅ Pre-filter	✅ Pre-filter	✅ Pre-filter
Hybrid Search	❌ Manual	❌ Manual	✅ Native	⚠️ Plugin
Multi-tenancy	⚠️ Metadata	✅ Namespace	✅ Tenants	✅ Collections
Batch Operations	✅ Yes	✅ Yes	✅ Yes	✅ Yes
Monitoring	⚠️ Basic	✅ Advanced	✅ Prometheus	✅ Metrics
Deployment	Local/Server	Managed	Self/Managed	Self/Managed
Cost (1M vecs)	$0 (self)	$70/month	$0-200	$0-150
Best for	MVP, prototyping	Production SaaS	Hybrid search	High performance

---

🎯 Decision Matrix

Step 1: Evaluate critical features

Feature Priority (1-5):

Metadata filtering:     5 (always critical)
Hybrid search:          ___ (evaluate based on queries)
Multi-tenancy:          ___ (only if SaaS)
Batch operations:       ___ (only if >100K docs)
Low latency (<50ms):    ___ (chatbot = 5, analytics = 2)
High throughput:        ___ (e-commerce = 5, MVP = 2)
Monitoring:             ___ (production = 5, MVP = 2)
Managed service:        ___ (prefer = 5, self-host = 1)

Step 2: Compute the score per database

def calculate_score(features_priority, db_support):
    """
    features_priority: dict {"metadata_filtering": 5, ...}
    db_support: dict {"metadata_filtering": 1.0, ...}  # 0-1
    """
    total_score = 0
    max_score = 0
    
    for feature, priority in features_priority.items():
        support = db_support.get(feature, 0)
        total_score += priority * support
        max_score += priority
    
    return (total_score / max_score) * 100  # Percentage

# Example: RAG Chatbot
priorities = {
    "metadata_filtering": 5,
    "hybrid_search": 3,
    "multi_tenancy": 1,
    "batch_ops": 2,
    "low_latency": 5,
    "monitoring": 4
}

chromadb_support = {
    "metadata_filtering": 0.8,  # Post-filter (less efficient)
    "hybrid_search": 0.3,       # Manual implementation
    "multi_tenancy": 0.5,       # Metadata only
    "batch_ops": 1.0,
    "low_latency": 0.9,
    "monitoring": 0.4
}

score_chromadb = calculate_score(priorities, chromadb_support)
# = 67% (acceptable for MVP)

pinecone_support = {
    "metadata_filtering": 1.0,  # Pre-filter
    "hybrid_search": 0.4,       # Manual but better than ChromaDB
    "multi_tenancy": 1.0,       # Native namespace
    "batch_ops": 1.0,
    "low_latency": 0.95,
    "monitoring": 1.0
}

score_pinecone = calculate_score(priorities, pinecone_support)
# = 89% (better for production)

Step 3: Decision

Score > 80%: ✅ Excellent match
Score 60-80%: ⚠️ Acceptable (evaluate trade-offs)
Score < 60%: ❌ Look for an alternative

---

🚫 Common Anti-patterns

Anti-pattern 1: Not using metadata filtering

# ❌ BAD
results = db.query(query_embedding, k=10)

# ✅ GOOD
results = db.query(
    query_embedding,
    where={"category": "support"},
    k=10
)

Impact: 10x latency + 25% accuracy loss.

Anti-pattern 2: Single inserts for bulk data

# ❌ BAD: 2.7 hours
for doc in 1M_docs:
    db.add(documents=[doc])

# ✅ GOOD: 1.6 minutes
for batch in batches(1M_docs, size=1000):
    db.add(documents=batch)

Impact: 100x time.

Anti-pattern 3: Not monitoring accuracy

# ❌ BAD: No testing
deploy_to_prod()

# ✅ GOOD: Daily accuracy test
@daily
def test_accuracy():
    accuracy = evaluate(test_set)
    if accuracy < baseline * 0.95:
        alert("Accuracy dropped 5%")

Impact: Silent degradation (frustrated users).

Anti-pattern 4: Pure semantic for specific queries

# ❌ BAD: "GPT-4 docs" → Returns GPT-3
results = db.query(embed("GPT-4 documentation"), k=10)

# ✅ GOOD: Hybrid search
results = hybrid_search(
    query="GPT-4 documentation",
    alpha=0.5  # 50% keyword, 50% semantic
)

Impact: 20-30% accuracy loss on specific queries.

---

✅ Module 3 Summary

What you mastered

Essential features:

1. ✅ Metadata filtering (10x latency + 25% accuracy)
2. ✅ Hybrid search (15-20% accuracy on specific queries)
3. ✅ Multi-tenancy (SaaS isolation)
4. ✅ Batch operations (100x ingestion speedup)
5. ✅ Distance metrics (cosine default)
6. ✅ Monitoring (latency, throughput, accuracy)

Unlocked skills:

• ✅ Evaluate a vector DB by features
• ✅ Design a production-ready RAG architecture
• ✅ Avoid common anti-patterns
• ✅ Calculate trade-offs (features vs cost)

---

🎓 Final Module Test

Question 1

Your RAG has 500K docs, queries include "Invoice #12345", accuracy must be >90%.
What features do you need?

Solution

Critical features:

1. Metadata filtering (reduce search space)
2. Hybrid search (captures the exact "Invoice #12345")

Database: Weaviate (native hybrid) or Pinecone + Elasticsearch

Config:

results = hybrid_search(
    query="Invoice #12345",
    alpha=0.7,  # 70% keyword (exact ID)
    where={"category": "invoices"}
)

Question 2

Your SaaS has 5000 small clients (1K docs each).
What multi-tenancy strategy?

Solution

Strategy: Metadata filtering (shared collection)

Reason:

• 5000 collections = excessive overhead
• Metadata filtering scales well (>10K tenants)

Implementation:

results = db.query(
    query_embedding,
    where={"tenant_id": current_user.tenant_id},
    k=10
)

Security: Middleware enforces tenant_id (don't trust the client).

Question 3

Your ingestion pipeline takes 2 hours for 1M docs.
How to optimize?

Solution

Optimizations:

1. Batch insert (size=1000) → 1.6 min (100x speedup)
2. Concurrent batches (4 workers) → 24 sec (4x additional)
3. Disable auto-indexing + rebuild at the end → 6.6 min total

Implementation:

with ThreadPoolExecutor(max_workers=4) as executor:
    batches = [docs[i:i+1000] for i in range(0, len(docs), 1000)]
    executor.map(ingest_batch, batches)

collection.rebuild_index()

Result: 2 hours → 6.6 min (18x speedup). If you answered 2-3/3 correctly → ✅ YOU MASTERED MODULE 3

---

🚀 Next Step: Module 4

You already understand:

• ✅ WHY vector databases (Module 1)
• ✅ HOW they work (HNSW, IVF, PQ) (Module 2)
• ✅ WHAT features you need (Filtering, Hybrid, Multi-tenancy) (Module 3)

Now you'll implement:

• 🎯 ChromaDB setup and configuration (Module 4)

Module 4: ChromaDB Hands-On

Topics:

1. ChromaDB installation and setup
2. Collection configuration (HNSW params)
3. Metadata filtering implementation
4. Batch ingestion pipeline
5. Basic monitoring

Duration: 60-75 minutes (40% conceptual, 60% code)

Ready? Go to Module 4 - ChromaDB Hands-On

---

Reading time: 8-10 minutes
Next module: ../../module-04-chromadb-setup/en/01-module-introduction-4.md