Module 3: Essential Features for RAG

Capsule 04: Multi-tenancy (Data isolation)

Understand multi-tenancy strategies for SaaS RAG, compare trade-offs (collection per tenant vs metadata filtering), and design a secure architecture with guaranteed isolation.

By the end of this capsule:

✅ You'll explain what multi-tenancy is and why it's critical for SaaS
✅ You'll compare 3 strategies (collection, metadata, namespace)
✅ You'll identify security considerations (data leakage)
✅ You'll design multi-tenancy according to scale

Estimated time: 8-10 minutes

🏢 What is Multi-tenancy?

Definition

Multi-tenancy = Multiple users/companies (tenants) share infrastructure, but their data is logically isolated.

Example: SaaS RAG

SaaS Product: "KnowledgeBase AI"
- Customer A (Acme Corp): 50K documents
- Customer B (Beta Inc): 30K documents  
- Customer C (Gamma LLC): 100K documents

Total: 180K documents in the same vector database
Requirement: Customer A CANNOT see Customer B/C data

Key: Logical isolation (not physical). Same DB, but queries filtered by tenant.

🔒 Why is it critical for SaaS RAG?

Problem 1: Data Leakage (Security)

Without multi-tenancy:

# ❌ VULNERABLE: Searches across ALL tenants
results = db.query(
    query_embedding=embed("Show me sales data"),
    k=10
)

# Result: Customer A can see Customer B, C docs
# ← GDPR/SOC2 violation!

With multi-tenancy:

# ✅ SECURE: Only searches in the current tenant
results = db.query(
    query_embedding=embed("Show me sales data"),
    where={"tenant_id": current_user.tenant_id},  # Filter
    k=10
)

# Result: Customer A only sees their own docs

Problem 2: Performance Degradation

Without isolation:

Customer A (10K docs) + Customer B (1M docs) in the same search space
Customer A's query: Searches in 1.01M docs (slow)
Latency: 150ms (dominated by Customer B's 1M docs)

With isolation:

Customer A's query: Searches in 10K docs (fast)
Latency: 5ms

Gain: 30x speedup.

Problem 3: Noisy Neighbors

Scenario: Customer B runs a bulk ingestion (1M inserts)

Without isolation:

CPU/RAM spike affects Customer A's queries
Customer A latency: 50ms → 500ms (10x degradation)

With isolation (collection per tenant):

Customer B's bulk ingestion does NOT affect Customer A
Stable latency

🏗️ Strategy 1: Collection per Tenant

Architecture

Vector Database
├─ Collection: tenant_a (50K vectors)
├─ Collection: tenant_b (30K vectors)
└─ Collection: tenant_c (100K vectors)

Query Customer A:
  db.get_collection("tenant_a").query(...)

Key: Each tenant has a dedicated collection (physical namespace).

Advantages

✅ Perfect isolation
- Physically separated (impossible data leakage)
- Doesn't require metadata filtering
✅ Independent performance
- A query in tenant A does NOT affect tenant B
- No noisy neighbors
✅ Independent indexes
- Each tenant can have a different HNSW configuration
- Rebuilding tenant A's index doesn't affect B
✅ Easy to migrate
- You can move tenant A to another DB without affecting B/C

Disadvantages

❌ Collection overhead
- Each collection consumes metadata (an HNSW index)
- 1000 collections × 50 MB overhead = 50 GB
❌ Doesn't scale with many tenants
- ChromaDB: Max 10K collections recommended
- Pinecone: Max 100 collections per project
❌ Setup complexity
- Creating/deleting collections dynamically
- Handling tenant creation/deletion

When to use

✅ <1000 tenants
✅ Large tenants (>10K docs each)
✅ Strict isolation required (compliance)
✅ Managed service (Pinecone, Weaviate)

Example: Enterprise SaaS with 50-500 large clients.

🏗️ Strategy 2: Metadata Filtering (Shared Collection)

Architecture

Vector Database
└─ Collection: all_tenants (180K vectors)
   ├─ Doc 1: {tenant_id: "a", content: "..."}
   ├─ Doc 2: {tenant_id: "a", content: "..."}
   ├─ Doc 3: {tenant_id: "b", content: "..."}
   └─ Doc 4: {tenant_id: "c", content: "..."}

Query Customer A:
  collection.query(
      ...,
      where={"tenant_id": "a"}
  )

Key: All tenants in the same collection, isolation via a metadata filter.

Advantages

✅ Scales with many tenants
- 10K+ tenants in a single collection (no overhead)
✅ Simple setup
- No creating/deleting collections dynamically
- Just add a tenant_id field
✅ Flexible
- You can add more metadata (region, tier)
✅ Economical
- Less overhead (a single HNSW index)

Disadvantages

❌ Isolation NOT perfect
- A bug in filtering = data leakage risk
- Requires exhaustive testing
❌ Shared performance
- Tenant B's bulk ingestion affects tenant A's latency
- Noisy neighbors problem
❌ Shared index
- Rebuilding the index affects ALL tenants
❌ Requires pre-filtering support
- The DB must support efficient metadata filtering
- Post-filtering is inefficient (searches everything first)

When to use

✅ >1000 tenants (many small tenants)
✅ Small tenants (<10K docs each)
✅ Self-hosted with limited RAM
✅ DB with efficient pre-filtering (Pinecone, Weaviate)

Example: SMB SaaS with 5000+ small companies.

🏗️ Strategy 3: Namespace (Hybrid)

Architecture

Vector Database
├─ Namespace: region_us
│  ├─ Collection: tenant_a
│  └─ Collection: tenant_b
└─ Namespace: region_eu
   ├─ Collection: tenant_c
   └─ Collection: tenant_d

Query Customer A (US):
  db.namespace("region_us")
    .get_collection("tenant_a")
    .query(...)

Key: Group tenants into namespaces (e.g., by region, tier).

Advantages

✅ Balance between strategies 1 and 2
- Isolation by namespace + by collection
✅ Regional compliance
- EU tenants in the EU namespace (GDPR)
- US tenants in the US namespace
✅ Performance tiers
- Premium tenants in a dedicated namespace (better hardware)
- Free tenants in a shared namespace

Disadvantages

❌ Architectural complexity
- Managing multiple namespaces + collections
❌ Not all DBs support it
- Pinecone: Yes (native namespaces)
- ChromaDB: No (workaround with prefixes)
- Weaviate: Partial (tenants)

When to use

✅ Multi-region deployment (compliance)
✅ Performance tiers (premium vs free)
✅ Mix of large + small tenants

Example: Global SaaS with regional compliance + tiers.

📊 Comparison: Collection vs Metadata vs Namespace

Dimension	Collection per Tenant	Metadata Filtering	Namespace Hybrid
Isolation	✅ Perfect	⚠️ Logical (bug risk)	✅ Perfect
Scale (tenants)	❌ <1K	✅ >10K	⚠️ <5K
Performance	✅ Independent	❌ Shared	✅ Independent
Setup	❌ Complex	✅ Simple	❌ Very complex
Cost (RAM)	❌ High (overhead)	✅ Low	⚠️ Medium
Migration	✅ Easy	❌ Hard	⚠️ Medium
Compliance	✅ Strict	⚠️ Requires audit	✅ Strict

Decision Matrix

Tenants count:
  < 100 → Collection per Tenant
  100-1000 → Collection or Namespace (based on budget)
  > 1000 → Metadata Filtering

Compliance:
  Strict (finance, health) → Collection per Tenant
  Moderate → Namespace
  Relaxed → Metadata Filtering

Budget:
  High → Collection per Tenant (managed service)
  Medium → Namespace (self-hosted)
  Low → Metadata Filtering (single collection)

🔒 Security Considerations

1. Isolation Testing

# Critical test: Verify NO data leakage
def test_tenant_isolation():
    # Insert docs for tenant A and B
    collection.add(
        documents=["Secret A"],
        metadatas=[{"tenant_id": "a"}]
    )
    collection.add(
        documents=["Secret B"],
        metadatas=[{"tenant_id": "b"}]
    )
    
    # Query as tenant A
    results = collection.query(
        query_embeddings=[...],
        where={"tenant_id": "a"},
        n_results=10
    )
    
    # Assert: Must NOT return tenant B docs
    assert all(r['tenant_id'] == 'a' for r in results)

Critical: Run this test in CI/CD (every deploy).

2. Isolation Middleware

# Enforce tenant_id on ALL queries
class TenantMiddleware:
    def query(self, query_embedding, current_user, **kwargs):
        # Inject tenant_id automatically
        where = kwargs.get('where', {})
        where['tenant_id'] = current_user.tenant_id
        
        # Override the where clause (don't trust the client)
        kwargs['where'] = where
        
        return self.db.query(query_embedding, **kwargs)

Key: NEVER trust the client to send the correct tenant_id.

3. Audit Logging

# Log ALL queries with tenant_id
logger.info({
    "action": "query",
    "tenant_id": current_user.tenant_id,
    "user_id": current_user.id,
    "query": query_text,
    "timestamp": time.now()
})

Compliance: SOC2, GDPR require an audit trail.

✅ Comprehension checklist

Verify that you understood this capsule:

What is multi-tenancy?
- Answer: Multiple users/companies share infrastructure, but their data is logically isolated.
What's the main advantage of collection per tenant?
- Answer: Perfect isolation (physical), independent performance, no noisy neighbors.
When to use metadata filtering vs collection per tenant?
- Answer: Metadata filtering for >1000 small tenants. Collection per tenant for <1000 large tenants or strict compliance.
What is the "noisy neighbors" problem?
- Answer: Tenant B's bulk ingestion/queries affect tenant A's latency (if they share a collection).
What critical test to run for security?
- Answer: A test that a tenant A query does NOT return tenant B docs (isolation test).

If you answered 4-5/5 correctly → ✅ Ready for Capsule 05 (Batch Operations)

🔗 Connection with RAG

How does multi-tenancy affect your SaaS RAG?

Case: KnowledgeBase AI (SaaS RAG)

Without multi-tenancy (MVP):

- 10 customers
- Single collection (all mixed together)
- Bug: Customer A sees Customer B docs
- Result: Security breach → Losing clients

With multi-tenancy (Production):

- 500 customers
- Strategy: Metadata filtering (scales well)
- Security: Middleware enforces tenant_id
- Result: SOC2 certified, enterprise-ready

🚀 Next step

You now know search features (filtering, hybrid) and isolation (multi-tenancy). Now you'll learn an operational feature: Batch Operations.

Next capsule: 05 - Batch Operations (Efficient ingestion)

You'll learn:

Why single insert is inefficient (2.7 hours for 1M docs)
Optimal batch size (1000-5000)
Bulk updates and deletes
Rebuild index strategies

Key: Batch operations are essential for ingesting large datasets efficiently.

Reading time: 8-10 minutes
Next: 05-batch-operations.md

← PreviousCapsule 03: Hybrid Search (Keyword + Semantic)Next →Capsule 05: Batch Operations — the hidden cost of naive ingestion

Capsule 04: Multi-tenancy (Data isolation)

🎯 Capsule objective

🏢 What is Multi-tenancy?

Definition

Example: SaaS RAG

🔒 Why is it critical for SaaS RAG?

Problem 1: Data Leakage (Security)

Problem 2: Performance Degradation

Problem 3: Noisy Neighbors

🏗️ Strategy 1: Collection per Tenant

Architecture

Advantages

Disadvantages

When to use

🏗️ Strategy 2: Metadata Filtering (Shared Collection)

Architecture

Advantages

Disadvantages

When to use

🏗️ Strategy 3: Namespace (Hybrid)

Architecture

Advantages

Disadvantages

When to use

📊 Comparison: Collection vs Metadata vs Namespace

Decision Matrix

🔒 Security Considerations

1. Isolation Testing

2. Isolation Middleware

3. Audit Logging

✅ Comprehension checklist

🔗 Connection with RAG

Case: KnowledgeBase AI (SaaS RAG)

🚀 Next step