Capsule 06: Distance Metrics — the geometric decision almost nobody justifies

Capsule description

When you created your first collection in ChromaDB, you probably glossed over the metadata={"hnsw:space": "cosine"} parameter or accepted the default without thinking. It's the least visible decision and one of the most expensive to get wrong: it defines what your system considers "similar" and can't be changed without re-embedding the whole dataset.

A distance metric isn't a technical detail. It's a geometric assumption about the embedding space. Cosine assumes direction matters and magnitude doesn't. L2 (euclidean) assumes both matter. Dot product assumes your vectors are already normalized and you're going to optimize for speed. Each assumption is valid in its context and breaks outside it. If your embeddings model recommends cosine and you chose L2 "because it sounded more mathematical", the system works — but recall drops 5-15% and the results rank in the wrong order without it being obvious why.

This capsule gives you the geometric mental model to understand what each metric does, how to choose the right one for your embeddings model, and how to avoid the expensive error: changing a collection's metric in production without re-embedding.

By the end of this capsule you'll be able to:

• ✅ Explain the conceptual difference between cosine, euclidean (L2), and dot product in terms of the geometry of the space
• ✅ Choose the right metric for a given embeddings model (OpenAI, Cohere, image embeddings, custom models)
• ✅ Configure the metric in ChromaDB with hnsw:space and verify that it matches the model's recommendation
• ✅ Identify when dot product beats cosine and why
• ✅ Anticipate the most expensive error: changing a collection's metric without re-embedding
• ✅ Diagnose strangely ranked results as a possible metric mismatch

Estimated time: 30-40 minutes

The mental model: three different geometries of the same space

Imagine two vectors in 2D so it's easy to see. In reality they're 384 or 1536 dimensions, but the logic scales.

       y
       │     ↗ B (3, 4)
       │   ↗
       │ ↗
       │↗  ↗ A (1.5, 2)
       │↗
       │
       ─────────────── x

Vectors A and B point in exactly the same direction — B is twice as long as A, but both go toward the same "corner" of the space. Are they similar?

The answer depends on which metric you use:

Cosine similarity: only direction matters

Cosine measures the angle between two vectors, ignoring magnitude. If two vectors point the same way (zero angle), their cosine similarity is 1, no matter how long they are.

cosine(A, B) = (A · B) / (||A|| × ||B||)

For A=(1.5, 2) and B=(3, 4):
A · B = 1.5×3 + 2×4 = 4.5 + 8 = 12.5
||A|| = √(1.5² + 2²) = √6.25 = 2.5
||B|| = √(3² + 4²) = √25 = 5

cosine(A, B) = 12.5 / (2.5 × 5) = 12.5 / 12.5 = 1.0

Cosine similarity = 1.0 → identical in direction. The metric considers them "equal" for retrieval purposes.

Cosine distance (which is what ChromaDB actually stores): 1 - cosine_similarity. That is, it goes from 0 (identical) to 2 (opposite). In the example: 1 - 1.0 = 0.0.

When magnitude matters and cosine ignores it: in text embeddings, the vector's magnitude is correlated with the length of the text, not with its meaning. A 200-word paragraph and a 50-word paragraph on the same topic produce vectors with similar directions but very different magnitudes. Cosine matches them correctly. L2 considers them "far apart" because of the magnitude difference.

Euclidean (L2): direction AND magnitude matter

L2 measures the straight-line distance between the endpoints of the vectors.

L2(A, B) = √(Σ(A_i - B_i)²)

For A=(1.5, 2) and B=(3, 4):
L2(A, B) = √((1.5-3)² + (2-4)²)
        = √((-1.5)² + (-2)²)
        = √(2.25 + 4)
        = √6.25
        = 2.5

L2 distance = 2.5 → relatively close but not identical. The metric considers them "similar" but distinguishable.

When this difference matters: in image embeddings, the vector's magnitude can be correlated with real visual properties (intensity, contrast). For face recognition, two similar faces can have vectors with the same direction but magnitudes that reflect different lighting — and L2 captures that difference.

Dot product: the geometric bet for speed

Dot product is mathematically the numerator of cosine (without dividing by the magnitudes):

dot(A, B) = A · B = Σ(A_i × B_i)

For A=(1.5, 2) and B=(3, 4):
dot(A, B) = 1.5×3 + 2×4 = 4.5 + 8 = 12.5

Dot product = 12.5 → a number that grows when the vectors are large AND point the same way.

The trick: if all your vectors are normalized (length 1), then ||A|| = ||B|| = 1, and dot(A, B) = cosine(A, B). In that case, dot product is mathematically equivalent to cosine but computes 30-40% faster (it doesn't need the two divisions by magnitude).

The danger: if your vectors are not normalized, dot product returns nonsense results. A large vector pointing anywhere will always "win" because of its magnitude, not its direction.

Which metric to use based on your embeddings model

Here's the table that will save you arguments:

The golden rule

Use the metric the model was trained with. Each embeddings model was trained by optimizing a specific loss function (e.g., contrastive loss with cosine). If you use a different metric at inference time, you're measuring something different from what the model learned to optimize.

How do you verify it? Read the model's official documentation. Hugging Face shows "Cosine Similarity" on the model card if that's the case. OpenAI explicitly says "cosine similarity" in its embeddings guide. If the documentation doesn't say it, look for the original paper — the metric is in the evaluation section.

Practical comparison: the three metrics on the same dataset

Let's see the effect on real results. Same dataset, same query, three collections with three different metrics.

# metrics_experiment.py
import chromadb
from chromadb.utils import embedding_functions
import os

openai_ef = embedding_functions.OpenAIEmbeddingFunction(
    api_key=os.getenv("OPENAI_API_KEY"),
    model_name="text-embedding-3-small"
)

client = chromadb.PersistentClient(path="./chroma_metrics_test")

# Same dataset, three collections with different metrics
docs = [
    "ChromaDB uses HNSW algorithm for approximate nearest neighbor search.",
    "Pinecone is a managed vector database service for production deployments.",
    "Cosine similarity measures the angle between two vectors.",
    "Vector databases are optimized for high-dimensional similarity search.",
    "RAG systems combine retrieval with language model generation.",
]
ids = [f"doc_{i}" for i in range(len(docs))]

# Cosine (recommended for OpenAI)
collection_cosine = client.get_or_create_collection(
    name="metrics_cosine",
    embedding_function=openai_ef,
    metadata={"hnsw:space": "cosine"}
)
collection_cosine.add(documents=docs, ids=ids)

# L2 (NOT recommended for text embeddings, but let's see it)
collection_l2 = client.get_or_create_collection(
    name="metrics_l2",
    embedding_function=openai_ef,
    metadata={"hnsw:space": "l2"}
)
collection_l2.add(documents=docs, ids=ids)

# Inner product (= dot product)
collection_ip = client.get_or_create_collection(
    name="metrics_ip",
    embedding_function=openai_ef,
    metadata={"hnsw:space": "ip"}
)
collection_ip.add(documents=docs, ids=ids)

query = "What algorithm does ChromaDB use for similarity search?"

print("=== COSINE (recommended for OpenAI) ===")
results = collection_cosine.query(query_texts=[query], n_results=3)
for doc, dist in zip(results['documents'][0], results['distances'][0]):
    print(f"  {dist:.3f} | {doc[:60]}...")

print("\n=== L2 (not recommended for OpenAI text embeddings) ===")
results = collection_l2.query(query_texts=[query], n_results=3)
for doc, dist in zip(results['documents'][0], results['distances'][0]):
    print(f"  {dist:.3f} | {doc[:60]}...")

print("\n=== INNER PRODUCT / DOT (when vectors are already normalized) ===")
results = collection_ip.query(query_texts=[query], n_results=3)
for doc, dist in zip(results['documents'][0], results['distances'][0]):
    print(f"  {dist:.3f} | {doc[:60]}...")

Typical output:

=== COSINE (recommended for OpenAI) ===
  0.198 | ChromaDB uses HNSW algorithm for approximate nearest neig...
  0.412 | Vector databases are optimized for high-dimensional simil...
  0.498 | Cosine similarity measures the angle between two vectors...

=== L2 (not recommended for OpenAI text embeddings) ===
  0.629 | ChromaDB uses HNSW algorithm for approximate nearest neig...
  0.918 | Vector databases are optimized for high-dimensional simil...
  0.998 | Cosine similarity measures the angle between two vectors...

=== INNER PRODUCT / DOT ===
  -0.802 | ChromaDB uses HNSW algorithm for approximate nearest neig...
  -0.587 | Vector databases are optimized for high-dimensional simil...
  -0.501 | Cosine similarity measures the angle between two vectors...

Critical observations:

1. The ranking is the same across all three metrics in this example, because the OpenAI embeddings come almost normalized and the documents are different enough that any metric separates them.

2. The absolute distances are completely different:

   • Cosine: range [0, 2]
   • L2: range [0, ∞]
   • Dot product: range [-∞, ∞] (negative because ChromaDB returns the negative of the IP to keep "smaller = more similar")

3. If you apply a threshold (e.g., discard results with distance > 0.5), the threshold has to match the metric. A threshold of 0.5 that works well with cosine is absurd with L2 (it would reject almost everything) or with dot product (it would reject almost nothing, because of the signs).

When the ranking does change between metrics

On larger datasets and with queries near the boundary between two clusters, the three metrics can return different orderings:

# Example where the metrics diverge
docs_difficult = [
    "ChromaDB uses HNSW for approximate nearest neighbor search.",       # Short, technical text
    "ChromaDB is an open-source embedding database designed for AI applications, supporting HNSW indexing, metadata filtering, persistent storage, and integration with popular embedding models including OpenAI, Cohere, and Sentence Transformers, making it suitable for both prototyping and production RAG systems.",  # Long text, more context
]

# Short query
query_short = "HNSW algorithm"

# With cosine: both rank high because the semantic direction is similar
# With L2: the second ranks lower because its vector has a larger magnitude (longer text)
# → observable ranking difference when there's large variation in document lengths

That's why choosing the metric that matches your model matters more when your dataset has texts of very variable lengths.

Configuring the metric in ChromaDB

ChromaDB supports three values for hnsw:space:

# Cosine (default + recommended for text)
collection = client.create_collection(
    name="my_collection",
    metadata={"hnsw:space": "cosine"}
)

# L2 (image embeddings, custom models that require it)
collection = client.create_collection(
    name="my_collection",
    metadata={"hnsw:space": "l2"}
)

# Inner product (normalized vectors, optimization)
collection = client.create_collection(
    name="my_collection",
    metadata={"hnsw:space": "ip"}
)

Verifying the metric of an existing collection

collection = client.get_collection("my_collection")
print(collection.metadata)
# {'hnsw:space': 'cosine'}

If the hnsw:space key doesn't appear, ChromaDB is using the default (l2 before v0.4.0, cosine after). Always verify it before assuming — differences between versions have caused bugs in migrations.

Special cases: distance threshold per metric

When you configure a score_threshold to discard low-quality results, the value depends on the metric:

# Cosine distance: typical range [0, 2], useful threshold 0.4-0.7
def is_relevant_cosine(distance: float) -> bool:
    return distance < 0.5  # 0.5 ≈ "moderately relevant"

# L2 distance: range [0, ∞], depends on the dataset
def is_relevant_l2(distance: float, max_distance: float) -> bool:
    # You need to compute max_distance empirically over your dataset
    return distance < (max_distance * 0.4)

# Inner product (negative): more negative values = more similar
def is_relevant_ip(distance: float) -> bool:
    return distance < -0.5  # More negative = more similar (in ChromaDB)

Recommended practice: run the system over 100-200 queries from your eval set, plot the distribution of distances of the correct chunks vs the incorrect ones, and choose the threshold where they separate. Don't copy thresholds from tutorials — the optimal range depends on your dataset.

Traps and common errors

Trap 1: changing the metric without re-embedding

The error:

# Day 1: you create the collection with L2
collection_v1 = client.create_collection(
    "docs", metadata={"hnsw:space": "l2"}
)
collection_v1.add(documents=thousand_docs, ids=thousand_ids)

# Day 30: someone says "we should use cosine for text embeddings"
# You delete and recreate, thinking only the metric changes
client.delete_collection("docs")
collection_v2 = client.create_collection(
    "docs", metadata={"hnsw:space": "cosine"}
)
# ❌ You forget to re-insert the docs

Symptom A: you forgot to re-insert and the collection is empty.

Symptom B: you re-inserted, but you copy-pasted the old IDs assuming the embeddings would be recomputed. If you saved the precomputed embeddings somewhere and passed them with embeddings=..., the embeddings are the same (computed with the original model) — and the new metric interprets them differently but the data doesn't change.

How to prevent it: the rule is simple — changing hnsw:space requires re-embedding all the documents from the original text. If you only have the embeddings and not the texts, you can't change the metric responsibly.

Trap 2: dot product with non-normalized vectors

The error:

# Embeddings from a custom model that does NOT normalize outputs
custom_embeddings = [
    [0.5, 0.1, 0.2, ...],  # magnitude 0.55
    [5.0, 1.0, 2.0, ...],  # magnitude 5.5 — the same vector scaled 10x
    [0.3, 0.6, 0.1, ...],  # magnitude 0.7
]

collection = client.create_collection(
    "test", metadata={"hnsw:space": "ip"}  # ← inner product
)
collection.add(embeddings=custom_embeddings, documents=docs, ids=ids)

# Query
results = collection.query(query_embeddings=[[1, 1, 1, ...]], n_results=3)

Symptom: the vector with magnitude 5.5 will always "win", regardless of whether it's semantically close to the query. Magnitude dominates over direction.

Why it happens: dot product grows linearly with magnitude. Large vectors get large scores mechanically.

How to prevent it: before using ip, normalize your embeddings:

import numpy as np

def normalize(vec):
    norm = np.linalg.norm(vec)
    return (np.array(vec) / norm).tolist() if norm > 0 else vec

normalized_embeddings = [normalize(e) for e in custom_embeddings]
collection.add(embeddings=normalized_embeddings, documents=docs, ids=ids)

Or simply use cosine, which normalizes implicitly — you lose the small speed advantage but gain robustness.

Trap 3: confusing similarity with distance

The error: you read a tutorial that says "a cosine similarity of 0.85 is very good" and you try to filter with distance > 0.85 in ChromaDB.

Symptom: you filter backwards. The results you should keep are the ones you discard.

Why it happens: ChromaDB returns distance, not similarity.

• Cosine similarity: range [-1, 1], 1 = identical, -1 = opposite
• Cosine distance (what ChromaDB returns): 1 - cosine_similarity, range [0, 2], 0 = identical

A similarity of 0.85 corresponds to a distance of 0.15.

How to prevent it: always read the tool's documentation. ChromaDB documents explicitly: "distances: smaller is more similar". When in doubt, do a sanity check with two identical texts:

collection.add(documents=["test"], ids=["a"])
result = collection.query(query_texts=["test"], n_results=1)
print(result['distances'])  # Should be ~0.0, not ~1.0

Trap 4: using L2 with OpenAI embeddings "because l2 is more standard in ML"

The error: someone with a strong classical ML background joins the team and argues that L2 is "the standard metric". They change the metric from cosine to L2 without changing the embeddings model.

Symptom: recall measured over the eval set drops 5-15%. The results rank differently, sometimes better on short queries and worse on long queries (because of the sensitivity to magnitude).

Why it happens: OpenAI's embeddings were optimized with a cosine-based loss function. Using L2 at inference time compares geometries the model never learned to produce.

How to prevent it: paste the official documentation. OpenAI says cosine. ChromaDB default cosine. Cohere says cosine. If the model doesn't explicitly say "L2", don't use L2.

Trap 5: assuming ChromaDB uses the metric you expect

The error: you created the collection without passing metadata={"hnsw:space": ...}, assuming the default was cosine. But you're using an old version of ChromaDB where the default was L2.

Symptom: queries that in another system ranked A→B→C rank here B→A→C. Inexplicable until you check the collection's metadata.

How to prevent it: always specify the metric explicitly when creating collections. Don't rely on defaults across versions.

# ❌ Relying on the default
collection = client.create_collection("docs")

# ✅ Explicit
collection = client.create_collection(
    "docs", metadata={"hnsw:space": "cosine"}
)

Trap 6: comparing distances across collections with different metrics

The error: you have two collections, one with cosine and another with L2 (say, one for text and another for images in a multimodal app). You want to "rank them jointly" the results from both.

Symptom: you compare a distance of 0.4 (cosine, decently similar) with 4.5 (L2, depends on the dataset) and the ranking makes no sense.

Why it happens: the metrics have different ranges and semantics. They're not directly comparable.

How to prevent it: if you need to combine results from multiple retrievals, normalize the scores to a common range [0, 1]:

def normalize_scores(distances, metric):
    """Normalizes distances to similarity [0, 1] where 1 is most similar."""
    if metric == "cosine":
        # Cosine distance [0, 2] → similarity [1, 0]
        return [1 - (d / 2) for d in distances]
    elif metric == "l2":
        # L2 [0, ∞] requires empirical normalization
        max_d = max(distances)
        return [1 - (d / max_d) for d in distances]
    elif metric == "ip":
        # Inner product (negative in Chroma) → similarity
        # Assumes normalized embeddings, range [-1, 0]
        return [1 + d for d in distances]  # from [-1, 0] to [0, 1]

This is a partial solution. For robust combination of retrievals, consider techniques like Reciprocal Rank Fusion (RRF), which ignores the absolute scores and combines rankings — covered in guide #8 (Advanced RAG).

Applied exercise

Scenario: you're reviewing a coworker's code who is building a multimodal search system for a fashion company. The system lets you search products by (a) text description and (b) reference image. Your coworker hands you this snippet:

# search_service.py
from chromadb import PersistentClient
from chromadb.utils import embedding_functions

client = PersistentClient(path="./chroma_fashion")

# For text embeddings: use text-embedding-3-small from OpenAI
text_ef = embedding_functions.OpenAIEmbeddingFunction(
    api_key=API_KEY, model_name="text-embedding-3-small"
)
products_text = client.create_collection(
    "products_text",
    embedding_function=text_ef,
    metadata={"hnsw:space": "ip"}  # ← decided this
)

# For image embeddings: use CLIP via custom embedding function
products_image = client.create_collection(
    "products_image",
    embedding_function=clip_ef,  # custom function that calls CLIP
    metadata={"hnsw:space": "l2"}  # ← decided this
)

# Function to combine results
def hybrid_search(text_query: str, image_path: str, top_k: int = 10):
    text_results = products_text.query(query_texts=[text_query], n_results=top_k)
    image_results = products_image.query(query_images=[image_path], n_results=top_k)

    # Combines the rankings by summing the distances (?)
    combined = {}
    for doc_id, dist in zip(text_results['ids'][0], text_results['distances'][0]):
        combined[doc_id] = combined.get(doc_id, 0) + dist
    for doc_id, dist in zip(image_results['ids'][0], image_results['distances'][0]):
        combined[doc_id] = combined.get(doc_id, 0) + dist

    return sorted(combined.items(), key=lambda x: x[1])[:top_k]

Question: identify the three conceptual errors about distance metrics in this code and explain how to fix each one.

# Option A (simpler): use cosine
products_text = client.create_collection(
    "products_text",
    embedding_function=text_ef,
    metadata={"hnsw:space": "cosine"}
)

# Option B (if you want IP's speed): normalize the embeddings explicitly
# before inserting them. But ChromaDB doesn't expose this easily when you use
# embedding_function — for that you'd have to pre-compute embeddings,
# normalize them, and insert with embeddings=... instead of documents=...
# More work, marginal in speed. Use cosine.

products_image = client.create_collection(
    "products_image",
    embedding_function=clip_ef,
    metadata={"hnsw:space": "cosine"}  # ← change to cosine
)

def hybrid_search(text_query: str, image_path: str, top_k: int = 10):
    text_results = products_text.query(query_texts=[text_query], n_results=top_k * 2)
    image_results = products_image.query(query_images=[image_path], n_results=top_k * 2)

    # RRF: combines rankings, not scores
    K = 60  # typical RRF constant
    rrf_scores = {}

    for rank, doc_id in enumerate(text_results['ids'][0]):
        rrf_scores[doc_id] = rrf_scores.get(doc_id, 0) + 1 / (K + rank)

    for rank, doc_id in enumerate(image_results['ids'][0]):
        rrf_scores[doc_id] = rrf_scores.get(doc_id, 0) + 1 / (K + rank)

    return sorted(rrf_scores.items(), key=lambda x: -x[1])[:top_k]

Summary and next step

What you learned:

• Distance metrics aren't interchangeable — each one assumes a different geometry of the embedding space.
• Cosine: ignores magnitude, measures the angle. Default for text embeddings (OpenAI, Cohere, Sentence Transformers).
• L2 (euclidean): considers direction and magnitude. Appropriate for traditional image embeddings and face recognition.
• Dot product (ip in ChromaDB): mathematically equivalent to cosine when the vectors are normalized, ~30% faster. Fails silently with non-normalized vectors.
• ChromaDB returns distance, not similarity. Cosine distance goes from 0 (identical) to 2 (opposite). Be careful when copying thresholds from tutorials that mix the concepts.
• The metric is chosen based on the embeddings model's official documentation, not personal preference. OpenAI says cosine, CLIP says cosine, FaceNet says L2.
• Changing a collection's metric in production requires re-embedding from the original text — precomputed embeddings are tied to the metric the model was trained with.

Checkpoint: before moving on, you should be able to:

• Explain to a coworker the conceptual difference between cosine and L2 in terms of "what each one ignores".
• Justify the metric choice for a given model by citing documentation, not intuition.
• Identify the three most expensive distance-metric errors (changing without re-embedding, dot product without normalizing, summing distances across different metrics).

Next capsule: 07 — Observability and monitoring for vector databases.

You just learned to configure the metric that defines what your system considers "similar". But in production, how do you know if the system is working well according to that metric? What quality metrics should you monitor when the dataset grows, the queries change, or the embeddings model is updated?

Capsule 07 teaches you what to measure, how to measure it, and what thresholds trigger action. It's the operational complement to the technical decisions you made up to here.

Resources

1. OpenAI Embeddings — Distance metrics — Official documentation recommending cosine
2. ChromaDB — Configuring HNSW Distance Metric — Official configuration
3. CLIP paper — Learning Transferable Visual Models From Natural Language Supervision — Section 2.3 documents the use of cosine similarity
4. Sentence Transformers — Choosing the Right Distance Metric — Recommendations per model
5. Reciprocal Rank Fusion (original paper) — For combining results across metrics
6. Pinecone — Cosine vs Dot Product — Technical comparison with benchmarks
7. Vector Norms and Distances — 3Blue1Brown — Geometric visualization of the metrics

Estimated time: 30-40 minutes Next: 07-observability-monitoring.md