Module 4: ChromaDB Setup and Configuration

Module 4: ChromaDB Setup and Configuration

Next →

Module description

Time to build! After 3 conceptual modules (Why, How, What features), now you'll build your first vector database with ChromaDB.

This module is 80% hands-on: executable code, real configuration, practical debugging. You'll apply everything you've learned:

  • Configure HNSW with optimal parameters (Module 2)
  • Implement metadata filtering (Module 3)
  • Efficient batch ingestion (Module 3)
  • Basic monitoring (Module 3)

By the end of this module, you'll have:

  • ✅ ChromaDB running locally
  • ✅ A collection configured with optimized HNSW
  • ✅ An ingestion pipeline for 10K-100K documents
  • ✅ Working queries with metadata filtering
  • ✅ Performance benchmarks (latency, throughput)

This module is 80% code, 20% conceptual. Get your IDE ready!

🎯 Module objective

Professional objective:

Implement and configure ChromaDB locally with optimized HNSW, metadata filtering, and efficient batch ingestion, validating performance with real benchmarks (latency <50ms, throughput >1K docs/sec).

Why ChromaDB to start?

  • ✅ Zero setup: Works in Python without an external server
  • ✅ Native HNSW: High accuracy (98%) out-of-the-box
  • ✅ Perfect for MVP: 10K-500K vectors without complexity
  • ✅ Migration path: ChromaDB → Pinecone/Weaviate when you scale

Analogy: ChromaDB is like SQLite for vector databases. Simple, local, perfect for learning and prototyping.

📚 Module content

Capsule 01: Module introduction (you're here)

  • Module objective and philosophy
  • Why ChromaDB to learn
  • Setup requirements and progression

Capsule 02: ChromaDB Installation and Setup

  • Installation (pip install chromadb)
  • Client modes (ephemeral, persistent, client-server)
  • First collection (hello world)
  • Verify a working installation

Capsule 03: Collection Configuration (HNSW Parameters)

  • Configure HNSW (M, efConstruction, efSearch)
  • Distance metrics (cosine, L2, dot product)
  • Collection metadata and settings
  • Validate the optimal configuration

Capsule 04: Metadata Filtering Implementation

  • Add metadata to documents
  • Where clauses (equality, range, logical)
  • Pre-filtering vs post-filtering in ChromaDB
  • Benchmark filtering impact (latency reduction)

Capsule 05: Batch Ingestion Pipeline

  • Single vs batch insert (benchmark)
  • Optimal batch size for ChromaDB (1000-5000)
  • Progress tracking and error handling
  • Concurrent batches (ThreadPoolExecutor)

Capsule 06: Query Optimization

  • Basic query (top-k similarity)
  • Query with metadata filtering
  • Adjust n_results according to accuracy needed
  • Benchmark query performance (p50, p95, p99)

Capsule 07: Persistence and Durability

  • Ephemeral vs persistent client
  • Data directory configuration
  • Backup strategies (simple copy)
  • Recovery and migrations

Capsule 08: Mini-Project - Document Search System

  • Problem: Index 10K Wikipedia articles
  • Complete pipeline (embed → batch insert → query)
  • Metadata filtering by category and date
  • Final benchmark (latency, accuracy, throughput)
  • Summary and transition to Module 5

🔗 Connection with other modules

Prerequisites:

  • Module 1: Why Vector DBs (you understand the need)
  • Module 2: How HNSW works (you'll configure parameters knowingly)
  • Module 3: Essential features (you'll implement filtering, batch ops)

This module prepares you for:

  • Module 5: ChromaDB + full RAG (you'll integrate LLM + vector DB)
  • Modules 6-8: Production (you'll scale, migrate, optimize)

Complete flow:

Module 1-3: Conceptual fundamentals (100% theory)
  ↓
Module 4: ChromaDB Setup ← You're here (80% code)
  ↓
Module 5: ChromaDB + RAG (90% code, full integration)
  ↓
Module 6-8: Production considerations (70% practical)

⏱️ Estimated time

Reading + Code: 90-120 minutes

Breakdown per capsule:

  • Capsule 01: 5 min (introduction)
  • Capsule 02: 12-15 min (installation + setup)
  • Capsule 03: 12-15 min (HNSW configuration)
  • Capsule 04: 12-15 min (metadata filtering)
  • Capsule 05: 15-18 min (batch ingestion)
  • Capsule 06: 10-12 min (query optimization)
  • Capsule 07: 8-10 min (persistence)
  • Capsule 08: 25-30 min (mini-project)

Total: 99-130 minutes

Note: This module requires running code (not just reading). Plan time to experiment.

🎓 What will you learn in this module?

By the end of this module, you'll be able to:

1. Set up ChromaDB correctly

  • ✅ Install ChromaDB (pip)
  • ✅ Choose a client mode (ephemeral vs persistent)
  • ✅ Create collections with an optimal configuration
  • ✅ Debug common errors (dependency issues)

2. Configure HNSW intelligently

  • ✅ Tune M, efConstruction, efSearch according to requirements
  • ✅ Choose a distance metric (cosine, L2, dot product)
  • ✅ Validate the accuracy vs latency trade-off
  • ✅ Benchmark configurations (A/B testing)

3. Implement metadata filtering

  • ✅ Add structured metadata (category, date, tags)
  • ✅ Queries with where clauses (equality, range, logical)
  • ✅ Measure the impact on latency (10x speedup expected)
  • ✅ Debug slow queries (profiling)

4. Optimize batch ingestion

  • ✅ Implement batch insert (1000-5000 batch size)
  • ✅ Progress tracking (tqdm, logging)
  • ✅ Error handling (retry logic)
  • ✅ Concurrent batches (4x speedup)

5. Benchmark performance

  • ✅ Measure latency (p50, p95, p99)
  • ✅ Measure throughput (docs/sec, queries/sec)
  • ✅ Measure accuracy (retrieval recall@10)
  • ✅ Identify bottlenecks (profiling)

💡 Module philosophy

Why hands-on after 3 conceptual modules

You might ask yourself: "Why not start with code from Module 1?"

Answer: Fundamentals first → Informed implementation.

Without fundamentals (typical tutorial):

# Typical tutorial: "Copy this code"
collection = client.create_collection("docs")
collection.add(documents=["Hello"], ids=["1"])
results = collection.query(query_texts=["Hi"], n_results=1)

# ❌ Problem: You don't understand WHY it works, HOW to optimize

With fundamentals (this course):

# With knowledge from Modules 1-3
collection = client.create_collection(
    name="docs",
    metadata={
        # Module 2: Configure HNSW intelligently
        "hnsw:M": 32,  # High accuracy (you understand the trade-off)
        "hnsw:construction_ef": 200,
        "hnsw:search_ef": 100,
        "hnsw:space": "cosine"  # Text embeddings
    }
)

# Module 3: Batch insert (100x faster than single)
collection.add(
    documents=batch_docs,  # 1000 docs
    metadatas=batch_metadata,  # Filtering support
    ids=batch_ids
)

# Module 3: Metadata filtering (10x speedup)
results = collection.query(
    query_texts=["Hi"],
    where={"category": "support"},  # Pre-filter
    n_results=10
)

# ✅ Advantage: You understand each parameter, you can optimize

Key differentiator vs the competition

90% of ChromaDB tutorials:

  • Show basic code without explaining the configuration
  • Don't cover batch ingestion, metadata filtering optimization
  • No benchmarking (you don't know if it's fast or slow)

This module:

  • Code with context (WHY each parameter)
  • Real benchmarks (latency, throughput before/after)
  • Decision framework (when to tune M, efSearch)

🚫 What this module does NOT cover

This module does NOT cover:

❌ LLM integration (that's Module 5)

  • You won't connect the OpenAI API yet
  • You won't build a full RAG system
  • Only an isolated vector database

❌ Production deployment (that's Modules 6-8)

  • No Docker, Kubernetes, cloud deployment
  • Only local development
  • Migration to Pinecone comes later

❌ Advanced features (that's Modules 7-8)

  • No distributed sharding
  • No custom distance metrics
  • No advanced multi-tenancy

Clear scope: This module is about ChromaDB fundamentals done well. Integration and production come later.

✅ Success criteria

You successfully completed this module when:

You can run these commands without errors:

import chromadb

# 1. Setup
client = chromadb.PersistentClient(path="./chroma_db")

# 2. Collection with optimized HNSW
collection = client.create_collection(
    name="test",
    metadata={"hnsw:M": 32, "hnsw:space": "cosine"}
)

# 3. Batch insert
collection.add(
    documents=["doc1", "doc2", "doc3"],
    metadatas=[{"cat": "a"}, {"cat": "b"}, {"cat": "a"}],
    ids=["1", "2", "3"]
)

# 4. Query with filtering
results = collection.query(
    query_texts=["doc1"],
    where={"cat": "a"},
    n_results=2
)

print(results)  # ✅ Should return ["doc1", "doc3"]

You can answer these questions:

  1. ✅ What is the optimal batch size for ChromaDB?

    • Answer: 1000-5000 docs (625K docs/min throughput)

  2. ✅ How to configure HNSW for accuracy >95%?

    • Answer: M=32, efConstruction=200, efSearch=100-200

  3. ✅ When to use ephemeral vs persistent client?

    • Answer: Ephemeral for testing/prototyping, Persistent for development/production

  4. ✅ How to measure if metadata filtering is optimized?

    • Answer: Benchmark latency with/without filtering. Expected: 10x speedup if the filter reduces the search space by 90%

  5. ✅ What is the expected latency for 100K vectors with HNSW?

    • Answer: p50 ~15ms, p95 ~40ms, p99 ~80ms (1536-dim, cosine)

If you answered 4-5/5 correctly AND the code runs without errors → ✅ Module completed

🛠️ Setup Requirements

Software needed

Python:

  • Python 3.8+ (recommended 3.10+)
  • pip (package manager)

Libraries:

pip install chromadb
pip install numpy  # For mock embeddings
pip install tqdm   # Progress bars

Recommended IDE:

  • VS Code with the Python extension
  • Jupyter Notebook (alternative)
  • PyCharm Community Edition

Minimum hardware:

  • RAM: 8 GB (16 GB recommended)
  • Storage: 5 GB free space
  • CPU: Any modern CPU (HNSW is CPU-bound)

Preparation before starting

1. Verify Python version:

python --version  # Must be >= 3.8

2. Create a virtual environment (recommended):

python -m venv venv
source venv/bin/activate  # Mac/Linux
# venv\Scripts\activate   # Windows

3. Install ChromaDB:

pip install chromadb

4. Verify the installation:

import chromadb
print(chromadb.__version__)  # Should print a version (e.g., 0.4.22)

If all the steps work → ✅ Ready to start

📖 How to use this module

Recommended strategy:

  1. Read + run code in parallel

    • Read the capsule on one monitor/window
    • Run the code on another
    • Experiment with parameters

  2. Don't copy code blindly

    • Understand each line (use the comments)
    • Experiment: Change M=16 to M=32, measure the difference
    • Break the code on purpose (learn debugging)

  3. Benchmarking is critical

    • Run benchmarks before/after optimizations
    • Validate that optimizations work (no assumptions)
    • Example: "Batch insert should be 100x faster" → Measure it!

  4. The mini-project is mandatory

    • Capsule 08 is where you consolidate everything
    • Don't skip it: it's where you learn real debugging
    • If it fails, debug it (better learning)

Suggested time:

Option A: Two sessions (recommended)

  • Session 1: Capsules 01-04 (setup, config, filtering) = 45-60 min
  • Session 2: Capsules 05-08 (batch, query, mini-project) = 60-70 min

Option B: One intense session

  • Everything at once = 120-130 min
  • Advantage: Fresh context
  • Disadvantage: Fatigue (a lot of code)

Recommendation: Option A (two sessions with breaks).

🔗 Resources for this module

Official documentation:

  1. ChromaDB Docs - Getting started, API reference
  2. ChromaDB GitHub - Issues, examples

Example notebooks:

  1. ChromaDB Quickstart - Official tutorial
  2. ChromaDB with LangChain - Integration example

Debugging:

  1. ChromaDB Troubleshooting - Common issues
  2. Stack Overflow tag: chromadb

Note: This module is self-contained (you don't need to read external docs). The resources are for going deeper later.

🚀 Ready to start?

Next step:

Go to Capsule 02: ChromaDB Installation and Setup

There you'll learn:

  1. Installation (pip install chromadb)
  2. Client modes (ephemeral vs persistent vs client-server)
  3. First collection (working hello world)
  4. Verify that everything works correctly

Before continuing, make sure you have:

  • ✅ Python 3.8+ installed
  • ✅ A virtual environment activated (recommended)
  • ✅ ChromaDB installed (pip install chromadb)
  • ✅ Your IDE open and ready

If everything is ready → Continue to Capsule 02!

Reading time: 5 minutes
Next: 02-installation-setup.md

← PreviousCapsule 08: Features Comparison and Module SummaryNext →Capsule 02: ChromaDB Installation and Setup