Capsule 03: Collection Configuration — the three parameters almost nobody justifies

Capsule description

When you create a collection in ChromaDB with client.create_collection("docs"), you're silently accepting four decisions the system makes for you: the distance metric (cosine), the indexing algorithm (HNSW), the number of connections per graph node (M=16), and the quality of the index build (construction_ef=100). Those defaults are reasonable — for small datasets and typical use cases. For real production, the defaults are the equivalent of buying a car without choosing the engine: you'll reach your destination, but with factory performance.

This capsule explains the three configurable HNSW parameters (hnsw:space, hnsw:M, hnsw:construction_ef) that define your vector database's performance before you insert the first document. We already covered the distance metric in M03/06 — here we focus on M and construction_ef, which are the two knobs that change the accuracy/memory/latency balance. You'll learn what each one controls, which values to choose for your case, and why you can't change them after inserting data without rebuilding the entire index.

By the end of this capsule, you'll be able to:

• ✅ Explain what each HNSW parameter controls: M, construction_ef, search_ef
• ✅ Choose justified values across three scenarios: prototype, typical production, critical systems
• ✅ Calculate the impact of changing M on RAM consumption (16 → 32 ≈ +50%)
• ✅ Differentiate between build-time parameters (can't be changed later) and runtime ones (can be)
• ✅ Benchmark different configurations to validate the trade-off before committing
• ✅ Anticipate the most expensive mistake: changing M on a collection with data and discovering it requires a full rebuild

Estimated time: 30-35 minutes

The mental model: what does each parameter control?

HNSW builds a layered graph. Each vector is a node. Nodes are connected to other nearby nodes in the vector space. To search, you hop from node to node following the connections, until you find the right cluster.

The three parameters control different aspects of that graph:

┌─────────────────────────────────────────────────────┐
│                                                     │
│  M (build-time): how many connections each node has │
│  ┌─────┐         More connections =                 │
│  │ ●───┤         - Better recall                     │
│  │ ●───┤         - More RAM                          │
│  │ ●───┤         - Slower build                      │
│  └─────┘                                            │
│                                                     │
│  construction_ef (build-time): how exhaustively it  │
│  searches for connections when building the graph   │
│       Higher = better-quality graph                 │
│              + slower build                          │
│                                                     │
│  search_ef (runtime): how exhaustively it searches  │
│  when running a query                               │
│       Higher = better recall + more latency         │
│                                                     │
└─────────────────────────────────────────────────────┘

Critical distinction:

• M and construction_ef are applied when building the graph (when you insert vectors). Once inserted, you can't change them without rebuilding the entire index.
• search_ef is applied when running queries. You can change it dynamically without touching the data.

If you choose M wrong and discover the problem with 1M vectors already inserted, you have to re-embed and re-insert everything. If you choose search_ef wrong, you change it in one line of configuration. That's why M deserves more attention up front.

Parameter 1: hnsw:space (distance metric)

Already covered in detail in M03/06. Summary:

If you're doing RAG with OpenAI or any modern text embedding, use cosine and forget about it. To go deeper, re-read M03/06.

import chromadb

client = chromadb.PersistentClient(path="./chroma_db")
collection = client.create_collection(
    name="docs",
    metadata={"hnsw:space": "cosine"}  # explicit > implicit default
)

Parameter 2: hnsw:M (connections per node)

M is the most impactful parameter. It controls how many connections each node of the HNSW graph has.

What changes with M

M = 8 (low):

• Each node connects to ~8 neighbors
• "Thin" graph: few options to navigate
• Typical recall: 88-92%
• RAM: baseline
• Build time: fast

M = 16 (ChromaDB default):

• ~16 neighbors per node
• "Balanced" graph
• Typical recall: 93-96%
• RAM: +30% vs M=8
• Build time: medium

M = 32 (recommended for production):

• ~32 neighbors per node
• "Dense" graph
• Typical recall: 97-98%
• RAM: +60% vs M=8
• Build time: ~2x M=16

M = 64+ (critical systems):

• Very dense graph, diminishing returns
• Typical recall: 98-99%
• RAM: +120% vs M=8
• Build time: ~4x M=16

The RAM calculation

ChromaDB with HNSW keeps in RAM:

RAM ≈ N × (D × 4 bytes + M × 8 bytes × levels)

Where:
  N = number of vectors
  D = vector dimensions
  4 bytes = float32
  8 bytes = pointer per connection
  levels ≈ 4-6 (graph layers)

For 1M vectors of 1536 dimensions:

For datasets of 100K-1M vectors, the difference between M=16 and M=32 is ~1.5 GB extra. Reasonable for production.

For datasets of 10M+ vectors, the difference becomes significant (15 GB vs 30 GB) and the hosting cost starts to matter.

How to configure it

collection = client.create_collection(
    name="prod_docs",
    metadata={
        "hnsw:space": "cosine",
        "hnsw:M": 32,                # typical production
        "hnsw:construction_ef": 200, # see next section
    }
)

Important: this metadata is set when you create the collection and cannot be changed. If you want to change M, you have to create a new collection and re-insert.

Parameter 3: hnsw:construction_ef (build quality)

construction_ef controls how exhaustively HNSW searches for optimal connections when building the graph.

When you insert a new vector, HNSW has to decide which M neighbors to connect it to. Searching for the "best M neighbors" exhaustively would be O(n) per insertion — too expensive. Instead, HNSW does an approximate search controlled by construction_ef:

• Low construction_ef (50-100): fast but suboptimal search — the graph ends up with "good but not the best" connections
• High construction_ef (200-400): more exhaustive search — the graph ends up with connections closer to optimal

Trade-off:

Recommendation:

• Prototype / small dataset: construction_ef=100 (default). Enough.
• Typical production: construction_ef=200. The quality difference is worth the extra build time (which you only pay once).
• Critical systems / huge datasets: construction_ef=400. You'll build very infrequently, so quality matters more than speed.

Parameter 4: hnsw:search_ef (query quality)

Unlike the previous ones, search_ef is applied at runtime — each query uses the configured value. And it can be changed without touching the data.

# Configure search_ef on the collection
collection = client.get_or_create_collection(
    name="prod_docs",
    metadata={
        "hnsw:space": "cosine",
        "hnsw:M": 32,
        "hnsw:construction_ef": 200,
        "hnsw:search_ef": 50,  # higher than the default 10
    }
)

Trade-off:

Recommendation: production search_ef=50-100. The accuracy/latency sweet spot. The default of 10 is too low for almost any real case.

We cover this in detail in M04/06 (query optimization).

The three canonical configurations

Configuration A: prototype / development

collection = client.create_collection(
    name="dev_docs",
    metadata={
        "hnsw:space": "cosine",
        "hnsw:M": 16,                # default
        "hnsw:construction_ef": 100, # default
        "hnsw:search_ef": 10,        # default
    }
)

When to use: you're learning, dataset <50K vectors, fine recall quality doesn't matter.

Expected performance (50K vectors):

• Build time: ~30 seconds
• Query p95: ~5ms
• Recall@10: ~93%
• RAM: ~400 MB

Configuration B: typical production (recommended for most)

collection = client.create_collection(
    name="prod_docs",
    metadata={
        "hnsw:space": "cosine",
        "hnsw:M": 32,
        "hnsw:construction_ef": 200,
        "hnsw:search_ef": 50,
    }
)

When to use: RAG in production with 100K-5M vectors, a reasonable SLA, accuracy matters.

Expected performance (1M vectors):

• Build time: ~30 minutes (one time only)
• Query p95: ~15ms
• Recall@10: ~97%
• RAM: ~8.5 GB

This is the config you'll use in 80% of cases. Start here.

Configuration C: critical systems (legal, medical, financial)

collection = client.create_collection(
    name="critical_docs",
    metadata={
        "hnsw:space": "cosine",
        "hnsw:M": 64,
        "hnsw:construction_ef": 400,
        "hnsw:search_ef": 200,
    }
)

When to use: a bad retrieval has serious consequences (medical information, legal advice, compliance). You tolerate more latency and RAM for better accuracy.

Expected performance (1M vectors):

• Build time: ~2 hours
• Query p95: ~40ms
• Recall@10: ~99%
• RAM: ~12 GB

Don't start here if you don't need to. It's 4x more expensive in RAM and 4x slower to build vs config B, gaining only 2% extra recall.

Validation with a benchmark before deploying

Don't trust the tables. Measure on your real dataset.

# benchmark_configs.py
import chromadb
from chromadb.utils import embedding_functions
import os
import time
import statistics

openai_ef = embedding_functions.OpenAIEmbeddingFunction(
    api_key=os.getenv("OPENAI_API_KEY"),
    model_name="text-embedding-3-small"
)


def benchmark_config(name, M, construction_ef, search_ef, docs, queries, eval_set):
    """Benchmark an HNSW configuration: build time + query latency + recall."""
    client = chromadb.PersistentClient(path=f"./chroma_bench_{name}")

    # Clean up previous run if it exists
    try:
        client.delete_collection(name)
    except Exception:
        pass

    collection = client.create_collection(
        name=name,
        embedding_function=openai_ef,
        metadata={
            "hnsw:space": "cosine",
            "hnsw:M": M,
            "hnsw:construction_ef": construction_ef,
            "hnsw:search_ef": search_ef,
        }
    )

    # Build time
    build_start = time.perf_counter()
    for batch_start in range(0, len(docs), 200):
        end = min(batch_start + 200, len(docs))
        collection.add(
            documents=docs[batch_start:end],
            ids=[f"doc_{i}" for i in range(batch_start, end)],
        )
    build_time = time.perf_counter() - build_start

    # Query latency (warm-up + measure)
    for q in queries[:5]:
        collection.query(query_texts=[q], n_results=10)

    latencies = []
    for q in queries:
        start = time.perf_counter()
        collection.query(query_texts=[q], n_results=10)
        latencies.append((time.perf_counter() - start) * 1000)
    latencies.sort()
    p50 = latencies[len(latencies) // 2]
    p95 = latencies[int(len(latencies) * 0.95)]

    # Recall@10 over the eval set
    hits = 0
    total = 0
    for item in eval_set:
        result = collection.query(query_texts=[item["query"]], n_results=10)
        retrieved_ids = set(result['ids'][0])
        relevant_ids = set(item["expected_doc_ids"])
        hits += len(retrieved_ids & relevant_ids)
        total += len(relevant_ids)
    recall = hits / total if total else 0

    return {
        "config": name,
        "build_time_seconds": build_time,
        "p50_ms": p50,
        "p95_ms": p95,
        "recall@10": recall,
    }


# Configs to compare
configs = [
    ("A_dev",      16, 100, 10),
    ("B_prod",     32, 200, 50),
    ("C_critical", 64, 400, 200),
]

# Assumes docs, queries, eval_set defined elsewhere
# docs = [...]  # 10K-100K real documents
# queries = [...]  # 50-100 representative queries
# eval_set = [{"query": ..., "expected_doc_ids": [...]}]  # 30+ entries

results = []
for name, M, ef_c, ef_s in configs:
    print(f"\nBenchmarking {name} (M={M}, construction_ef={ef_c}, search_ef={ef_s})...")
    result = benchmark_config(name, M, ef_c, ef_s, docs, queries, eval_set)
    results.append(result)
    print(f"  Build time: {result['build_time_seconds']:.1f}s")
    print(f"  Query p50:  {result['p50_ms']:.1f}ms")
    print(f"  Query p95:  {result['p95_ms']:.1f}ms")
    print(f"  Recall@10:  {result['recall@10']:.2%}")

Typical output (dataset of 100K real vectors):

Benchmarking A_dev (M=16, construction_ef=100, search_ef=10)...
  Build time: 188.4s
  Query p50:  3.2ms
  Query p95:  6.8ms
  Recall@10:  91.5%

Benchmarking B_prod (M=32, construction_ef=200, search_ef=50)...
  Build time: 412.7s
  Query p50:  7.4ms
  Query p95:  14.1ms
  Recall@10:  96.8%

Benchmarking C_critical (M=64, construction_ef=400, search_ef=200)...
  Build time: 1247.3s
  Query p50:  19.8ms
  Query p95:  38.5ms
  Recall@10:  98.9%

Takeaways:

• B_prod gains 5% recall over A_dev at the cost of 2x build time and 2x query latency. Worth it.
• C_critical gains 2% extra recall over B_prod at the cost of 3x build time and 2.5x query latency. Only worth it in critical cases.

Your eval set determines whether the recall differences are significant for your case. If your product tolerates 91% recall (dev), don't pay the prod cost. If you need 98%+ (legal/medical), accept the extra cost.

Traps and common mistakes

Trap 1: changing M on a collection with data

The mistake: you inserted 500K docs with M=16. After reading recommendations, you decide to move to M=32. You modify the collection's metadata.

Symptom: either ChromaDB throws an error, or (worse) it accepts the change but the internal HNSW graph is still built with M=16. The new config applies only to new docs. Result: a collection with an inconsistent graph.

How to prevent it: M and construction_ef are set at creation and can't be changed. If you need to change them, create a new collection with the desired config and re-insert everything (covered in M04/05 batch ingestion).

Trap 2: using config C ("critical") by default

The mistake: "we want the best," so you use M=64, construction_ef=400 from the start.

Symptom: build time 4x slower, RAM 50% higher, latency 2.5x worse — all to gain 2% recall that doesn't show up in your eval set.

How to prevent it: start with config B (production), measure recall over your eval set. Only move up to C if the difference is justified.

Trap 3: accepting search_ef=10 (default) in production

The mistake: you leave everything default, you don't set hnsw:search_ef explicitly. ChromaDB uses 10.

Symptom: queries are super fast (5ms p95), but recall@10 is 88% instead of the 96% you expect from the embedding model. Users report "it doesn't find things I know are there."

How to prevent it: explicitly set hnsw:search_ef=50 for production. The extra latency (10ms vs 5ms) is invisible to the user, but the recall quality is noticeable.

Trap 4: confusing construction_ef with search_ef

The mistake: you think adjusting construction_ef affects queries. You raise construction_ef=400 expecting better recall at runtime.

Symptom: queries don't change — construction_ef only affected the build (which already happened). What you wanted to adjust was search_ef.

How to prevent it: mnemonic — "construction_ef = when building, search_ef = when searching". The first only matters at the initial build; the second on every query.

Trap 5: forgetting that M affects RAM linearly

The mistake: the dataset grows from 100K to 5M. Your config was M=32 for 100K (RAM ~1 GB). You assume that with 5M it'll be ~50 GB, and an instance with 64 GB of RAM holds up.

Reality: besides the vectors themselves, HNSW stores M × levels pointers per node. For 5M with M=32, the HNSW overhead alone (not counting vectors) is ~6 GB extra. Total RAM ~56 GB — within the limit but with no margin.

How to prevent it: monitor ram_usage_pct and plan before hitting limits. Consider dropping to M=16 or migrating to IVF+PQ if memory is the constraint.

Trap 6: not benchmarking, assuming the tables

The mistake: you copy the "config B" values without measuring. You assume recall will be 96-98%.

Reality: recall depends on the embedding model, the dataset distribution, and the real queries. The tables are indicative. Your real system may have recall of 99% (a well-behaved dataset) or 89% (out-of-distribution queries).

How to prevent it: build your own eval set (30-50 queries with labeled docs) and measure before committing.

Applied exercise

Scenario: you're going to deploy a RAG for a medical services company. Characteristics:

• Dataset: 800K clinical guidelines in PDF format (chunked into ~3M chunks)
• Embedding model: OpenAI text-embedding-3-small (1536 dim)
• Constraints:
  • Hardware: an instance with 32 GB RAM available for ChromaDB
  • SLA: p95 query <50ms total (including embedding ~150ms — so retrieval <20ms)
  • Compliance: errors in answers have clinical consequences. Recall@10 minimum 97%.
• Tolerable build time: up to 4 hours (monthly maintenance window)

Your task: choose the HNSW parameters (M, construction_ef, search_ef) and justify each choice with numbers.

collection = client.create_collection(
    name="medical_guidelines",
    embedding_function=openai_ef,
    metadata={
        "hnsw:space": "cosine",
        "hnsw:M": 32,                # RAM margin, latency OK
        "hnsw:construction_ef": 400, # higher-quality build to compensate for a moderate M
        "hnsw:search_ef": 100,       # high recall at runtime
    }
)

Summary and next step

What you learned:

• ChromaDB uses HNSW by default, configurable with four parameters: hnsw:space, hnsw:M, hnsw:construction_ef, hnsw:search_ef.
• M and construction_ef are set when creating the collection and can't be changed without rebuilding the index.
• search_ef is applied at runtime and can be changed without touching the data.
• M controls the graph density: more M → better recall, more RAM, slower build. Default 16, recommended production 32.
• construction_ef controls the build quality: higher → better-quality graph, slower build. Default 100, recommended production 200-400.
• search_ef controls the query quality: default 10 (too low for production), recommended 50-100.
• The three canonical configs (dev, production, critical) cover most cases. Start with production (B), adjust if your eval set justifies it.

Checkpoint: before moving on, you should be able to:

• Differentiate build-time parameters (M, construction_ef) from runtime ones (search_ef).
• Calculate the approximate impact of changing M on RAM consumption for a given dataset.
• Justify the config choice (B prod vs C critical) based on SLA and compliance.

Next capsule: 04 — Metadata Filtering Implementation.

You configured the index engine. Now you'll learn to leverage it: filter results by metadata to reduce the search space 10x when your query allows it. Capsule 04 teaches you the complete syntax of where clauses and how to design the metadata schema so the filters you'll need are easy to express.

Resources

1. ChromaDB — Configuring HNSW Parameters — Official documentation
2. HNSW: Hierarchical Navigable Small World Graphs (paper) — The paper that defines the algorithm, section 4 explains M and ef
3. HNSW Tutorial (Pinecone Learn) — Graph and parameter visualization
4. Comparing HNSW Configurations (Qdrant Blog) — Illustrated trade-offs
5. ANN Benchmarks — Reproducible benchmarks for different configs
6. Tuning HNSW for Production (Weaviate) — Practical case with recommended values

Estimated time: 30-35 minutes Next: 04-metadata-filtering.md