Capsule 02: Pre-filter vs post-filter — the architectural decision that defines the whole module

Capsule description

There are two ways to apply metadata filtering. They sound equivalent — in the end, both return only the documents that match the filter. But architecturally they're completely different, with dramatic consequences for latency, recall and security. This capsule is the decision that defines the rest of the module: when it's safe to post-filter (rarely) and why pre-filter is the mandatory default in production.

You can state the executive summary in one sentence: pre-filter in 99% of cases. But understanding the "why" with data lets you defend the decision to a Tech Lead, identify the 1% of cases where post-filter applies, and diagnose problems when someone implements post-filter without knowing better.

By the end of this capsule you'll be able to:

• ✅ Explain the operational difference between pre-filter and post-filter in a vector DB
• ✅ Quantify the impact on latency and recall with reproducible benchmarks
• ✅ Identify the three cases where post-filter does make sense (they're rare)
• ✅ Implement guardrails that forbid queries without the mandatory filters
• ✅ Design a staged fallback that relaxes optional filters without breaking isolation
• ✅ Anticipate the pathological case: filters that shrink the subset below n_results

Estimated time: 25-30 minutes

The insight: where you apply the filter defines how fast and how safe it is

Visually:

                    PRE-FILTER (recommended)

  Query + filter  ─→  Vector DB (the engine)
                          │
                          ├─→ 1. Apply the filter over the corpus
                          │    (reduces from 5M to 100K vectors)
                          │
                          ├─→ 2. Search for similarity over the 100K
                          │
                          └─→ 3. Return the top-K
                              ↓
                          Results guaranteed to come from the filtered subset


                    POST-FILTER (dangerous)

  Query (no filter)  ─→  Vector DB (the engine)
                              │
                              ├─→ 1. Search for similarity over the 5M
                              │
                              └─→ 2. Return the top-100
                                  ↓
                              The app filters in code:
                              candidates = [d for d in top-100 if d.tenant_id == 'acme']
                                  ↓
                              Top-K (may be <K if few pass the filter)

The immediate differences:

1. Pre-filter: the vector DB does the filtering. It's index-aware pre-filtering.
2. Post-filter: the app filters afterwards. The vector DB never knew about the filter.

The consequences compound along every dimension.

Quantitative comparison with benchmarks

# benchmark_filtering.py
import chromadb
import time
from chromadb.utils import embedding_functions
import os

openai_ef = embedding_functions.OpenAIEmbeddingFunction(
    api_key=os.getenv("OPENAI_API_KEY"),
    model_name="text-embedding-3-small",
)
client = chromadb.PersistentClient(path="./chroma_filter_test")
collection = client.get_collection("multi_tenant_corpus", embedding_function=openai_ef)

# Assume a corpus of 100K docs spread across 50 tenants
# tenant_id: ~2K docs each

query = "How do I configure authentication?"
target_tenant = "acme_corp"


# Pre-filter
def search_pre_filter(query: str):
    start = time.perf_counter()
    results = collection.query(
        query_texts=[query],
        n_results=5,
        where={"tenant_id": target_tenant},  # ← the filter goes IN the query
    )
    elapsed = (time.perf_counter() - start) * 1000
    return elapsed, len(results["ids"][0])


# Post-filter
def search_post_filter(query: str):
    start = time.perf_counter()
    # Fetch more candidates to leave room for the post-filter
    results = collection.query(query_texts=[query], n_results=100)

    # Filter manually
    filtered = [
        (doc, meta)
        for doc, meta in zip(results["documents"][0], results["metadatas"][0])
        if meta.get("tenant_id") == target_tenant
    ][:5]
    elapsed = (time.perf_counter() - start) * 1000
    return elapsed, len(filtered)


# Run it 50 times and compare
pre_latencies = []
pre_results_count = []
post_latencies = []
post_results_count = []

for _ in range(50):
    lat, n = search_pre_filter(query)
    pre_latencies.append(lat)
    pre_results_count.append(n)

    lat, n = search_post_filter(query)
    post_latencies.append(lat)
    post_results_count.append(n)


def percentile(arr, p):
    arr_sorted = sorted(arr)
    return arr_sorted[int(len(arr_sorted) * p / 100)]


print("Pre-filter:")
print(f"  Latency p50: {percentile(pre_latencies, 50):.1f} ms")
print(f"  Latency p95: {percentile(pre_latencies, 95):.1f} ms")
print(f"  Average results: {sum(pre_results_count) / len(pre_results_count):.1f}")

print("\nPost-filter:")
print(f"  Latency p50: {percentile(post_latencies, 50):.1f} ms")
print(f"  Latency p95: {percentile(post_latencies, 95):.1f} ms")
print(f"  Average results: {sum(post_results_count) / len(post_results_count):.1f}")

Typical output (100K corpus, a tenant with 2K docs):

Pre-filter:
  Latency p50: 28 ms
  Latency p95: 45 ms
  Average results: 5.0   ← always returns 5 (they exist in the subset)

Post-filter:
  Latency p50: 185 ms
  Latency p95: 245 ms
  Average results: 3.7   ← sometimes <5 (there weren't enough in the top-100)

The readings:

1. Latency 6x worse with post-filter. The vector DB searches the entire corpus, and then you throw away 98% of the results.
2. Poor recall with post-filter. If only 2 of the top-100 belong to the tenant, you return 2 docs instead of 5. Recall collapses.
3. Pre-filter is stable. It always returns n_results or close to it, with consistent latency.

The pathological case: a very small subset

If the filter leaves a tiny subset compared to n_results, both techniques lose:

Corpus:           100K docs
Filter:           {"tenant_id": "tiny_tenant", "type": "tutorial", "language": "es"}
Matching subset:  20 docs

Pre-filter top-K=5:  searches the 20 → top-5 OK (returns 5)
Pre-filter top-K=50: searches the 20 → top-20 (there aren't 50 in the subset)

Post-filter top-100: searches 100K; the subset's 20 docs rarely make the top-100
                     because there are 99,980 docs competing
                     result: 1-3 docs (not 5)

The operational conclusion: if the subset is small and the filter is restrictive, pre-filter works perfectly and post-filter fails catastrophically.

The three cases where post-filter DOES apply

There are real situations where post-filter is the right choice. They're rare but legitimate:

Case 1: the engine doesn't support the filter you need

Some vector DB backends have limited filters. For example, if you need to filter on a complex expression (SUBSTRING(metadata.description, 1, 5) = "ABC") and your engine only supports equality and ranges, you'll have to post-filter.

# The engine doesn't allow SUBSTRING
results = collection.query(query_texts=[q], n_results=200)

# Post-filter with custom logic
filtered = [
    doc for doc in results["documents"][0]
    if doc.startswith("ABC")  # a complex condition
]

When it applies: rarely. Most modern vector DBs (ChromaDB, Pinecone, Weaviate, Qdrant) support expressive filters.

Case 2: a filter whose result is dynamic and changes per request

Sometimes the filter depends on information you only know AFTER the retrieval. For example: discarding docs based on something computed over the document's body.

# You need to compute something from the content to decide whether to filter
results = collection.query(query_texts=[q], n_results=50)

filtered = []
for doc in results["documents"][0]:
    if compute_complex_score(doc, user_context) > threshold:
        filtered.append(doc)

return filtered[:5]

When it applies: secondary ranking or dynamic filters. Even so, consider whether it can be pre-computed and stored as metadata.

Case 3: a prototype with a small dataset (<10K)

For very small datasets, post-filter's extra latency is negligible, and it lets you test filter logic quickly without touching the schema.

# Prototype: 5K docs, post-filter latency is ~50ms
# Not worth the effort of re-indexing to add structured metadata
results = collection.query(query_texts=[q], n_results=100)
filtered = [...]  # manual filter

When it applies: only in exploration. For production, always migrate to pre-filter.

Guardrails: turning isolation into a platform rule

In multi-tenant systems, the tenant_id filter has to be mandatory — not optional, not a convention. Implement guardrails:

# secure_query.py
class TenantIsolationError(Exception):
    pass


def secure_query(
    collection,
    query_text: str,
    tenant_id: str,           # ← mandatory in the signature
    additional_filters: dict = None,
    n_results: int = 5,
):
    """
    A safe wrapper that NEVER allows queries without a tenant_id.
    Any code calling secure_query MUST pass a tenant_id.
    """
    if not tenant_id:
        raise TenantIsolationError(
            "tenant_id is mandatory. Searching without a tenant filter is not allowed."
        )

    # Build the `where` with the mandatory tenant_id + the optional filters
    where = {"tenant_id": tenant_id}
    if additional_filters:
        where = {"$and": [
            {"tenant_id": tenant_id},
            additional_filters,
        ]}

    return collection.query(
        query_texts=[query_text],
        n_results=n_results,
        where=where,
    )


# Correct usage
results = secure_query(
    collection,
    "How do I authenticate?",
    tenant_id=current_user.tenant_id,
    additional_filters={"language": "en"},
)


# Incorrect usage: TenantIsolationError
results = secure_query(collection, "How do I authenticate?", tenant_id=None)
# raises TenantIsolationError

Bonus: code review enforcement. Add a custom linter that catches direct collection.query(...) calls that don't go through secure_query(). An example with a pre-commit hook:

# pre_commit_check.py
import re
import sys
from pathlib import Path

DIRECT_QUERY_PATTERN = re.compile(r'collection\.query\(')
SAFE_FUNCTION = "secure_query"

for py_file in Path("src").rglob("*.py"):
    content = py_file.read_text()
    if DIRECT_QUERY_PATTERN.search(content) and SAFE_FUNCTION not in content:
        print(f"⚠️  Direct collection.query() in {py_file}. Use secure_query() instead.")
        sys.exit(1)

This stops a new dev from accidentally calling collection.query(...) with no filter.

The staged fallback: relaxing optional filters without breaking security

Sometimes your filter is so restrictive that it doesn't return enough results. The solution is to relax it progressively, but NEVER to relax the security filter.

def search_with_progressive_fallback(
    collection,
    query: str,
    tenant_id: str,           # ← MANDATORY, never relaxed
    optional_filters: dict,
    n_results: int = 5,
    min_results: int = 3,
):
    """
    Searches with a staged fallback. Relaxes the optional filters if there aren't
    enough results, but NEVER relaxes tenant_id.
    """
    # Relaxation order: from the most specific to the most general
    fallback_order = [
        optional_filters,                                          # full
        {k: v for k, v in optional_filters.items() if k != "tags"},  # without tags
        {k: v for k, v in optional_filters.items() if k not in {"tags", "category"}},
        {},                                                          # tenant_id only
    ]

    for filter_set in fallback_order:
        # Build the `where`: tenant_id + the remaining filters
        where = {"tenant_id": tenant_id}
        if filter_set:
            where = {"$and": [
                {"tenant_id": tenant_id},
                filter_set,
            ]}

        results = collection.query(
            query_texts=[query],
            n_results=n_results,
            where=where,
        )

        if len(results["ids"][0]) >= min_results:
            return results, filter_set  # also return which filters survived

    # If even tenant_id alone finds nothing, return empty
    return {"ids": [[]], "documents": [[]]}, {}


# Usage
results, used_filters = search_with_progressive_fallback(
    collection,
    "OAuth2 setup",
    tenant_id="acme",
    optional_filters={"category": "auth", "tags": "oauth2", "language": "es"},
)

print(f"Results: {len(results['ids'][0])}")
print(f"Filters used in the end: {used_filters}")
# E.g. if there was no tags="oauth2", the filters used = {"category": "auth", "language": "es"}

The key pattern: tenant_id goes into EVERY step of the fallback. No exceptions. If a new dev tries to add a fallback that drops it, code review must reject it.

Traps and common mistakes

Trap 1: post-filter as the solution "because it's easier"

The mistake: a new team doesn't want to deal with a metadata schema, so they post-filter.

The symptom: latency 5-10x worse than with pre-filter. Variable, unpredictable recall. At scale, the system collapses.

How to prevent it: pre-filter is the industry standard. Post-filter only in documented, exceptional cases.

Trap 2: an ultra-restrictive filter with no fallback

The mistake:

where = {"tenant_id": "X", "category": "Y", "language": "Z", "version": "v3", "author": "John"}

The symptom: queries that combined all the filters return zero results. The user sees "I found nothing" when there are obviously docs on the topic.

How to prevent it: the staged fallback (shown above). Only tenant_id is mandatory; the rest get relaxed if needed.

Trap 3: forgetting tenant_id in internal queries

The mistake: admin scripts / cron jobs that run direct queries without going through secure_query. They assume "I'm admin, I don't need to filter."

The symptom: admin logs get accidental access to cross-tenant data. The audit fails.

How to prevent it: secure_query is mandatory in ALL code, including internal scripts. If you need a cross-tenant query, use an explicit cross_tenant_admin_query function that requires elevated permissions.

Trap 4: post-filter with n_results equal to the target

The mistake:

# You want 5 docs after the filter
results = collection.query(query_texts=[q], n_results=5)  # ❌ only 5 candidates
filtered = [d for d in results if d.tenant == "X"]  # you can end up with 0

The symptom: it very frequently returns <5 docs because the top-5 didn't belong to the right tenant.

How to prevent it: if you have to post-filter (a rare case), use n_results = top_k * 20 to leave margin. But better: pre-filter.

Trap 5: validating a tenant_id from a badly sanitized string

The mistake:

where = {"tenant_id": request.user.tenant_id}  # tenant_id comes from user input

If tenant_id is user-controllable (e.g. a URL parameter), an attacker can change it.

How to prevent it: tenant_id must come from the authenticated backend, NEVER from user input. A validated JWT token, a session, etc.

Trap 6: pre-filter but badly indexed metadata

The mistake: you turn on pre-filter, but the tenant_id field isn't in the index. ChromaDB scans it linearly — slower than post-filter.

The symptom: latency with pre-filter is worse than with no filter at all.

How to prevent it: verify that ChromaDB/Pinecone actually indexes the metadata fields you use in filters. Covered in capsule 03 (schema design).

Applied exercise

Scenario: you're an AI Engineer at a support SaaS. The stack:

• 50 customers (tenants)
• Each customer has 500-50K documents (some small, some large)
• Current system: a manual post-filter in the app's code
• The symptoms:
  • p95 latency = 320ms
  • Tickets reporting "I saw another company's document in my results"
  • Complaints: "the bot can't find things I know I have"

Your job:

1. Diagnose the problems and connect each one to the current approach.
2. Design the migration to pre-filter.
3. Estimate the latency and security impact.

# Step 1: verify that tenant_id is indexed in ChromaDB
# (capsule 03 covers how to verify this)
all_metadatas = collection.get(include=["metadatas"])
unique_tenants = set(m["tenant_id"] for m in all_metadatas["metadatas"])
print(f"Tenants in the corpus: {len(unique_tenants)}")  # should be 50

# Step 2: implement secure_query as the mandatory wrapper
def secure_query(query: str, tenant_id: str, **kwargs):
    if not tenant_id:
        raise TenantIsolationError("tenant_id is mandatory")
    where = {"tenant_id": tenant_id}
    if kwargs.get("additional_filters"):
        where = {"$and": [where, kwargs["additional_filters"]]}
    return collection.query(query_texts=[query], where=where, n_results=kwargs.get("n_results", 5))

# Step 3: refactor every retrieval call-site
# - Search for `collection.query(` in the code
# - Replace it with `secure_query(`
# - Make sure tenant_id is always passed

# Step 4: add a pre-commit hook that fails if someone calls collection.query directly

# Step 5: isolation tests
def test_tenant_isolation():
    """Verify that it NEVER crosses data."""
    for tenant_a in test_tenants[:5]:
        for tenant_b in test_tenants[5:10]:
            results = secure_query("test query", tenant_id=tenant_a)
            for meta in results["metadatas"][0]:
                assert meta["tenant_id"] == tenant_a, f"Leak: {meta} in a query from {tenant_a}"

Current system (post-filter):
  - Search over 1.5M docs: ~250ms
  - Filter in code: ~5ms
  - Handle the "not enough results" case + retry: ~50ms (sometimes)
  - Total p95: 320ms

System with pre-filter:
  - Filter down to ~30K docs (the average tenant): ~5ms
  - Search over 30K docs: ~25ms
  - Total p95: ~40ms

Gain: 320ms → 40ms (-87%)

Current system (post-filter):
  Some small tenants (500 docs): the global top-100 sometimes doesn't include their docs
  Average recall@5: ~65%

System with pre-filter:
  Each tenant searches their own subset → recall guaranteed from the subset
  Average recall@5: ~85% (depends on the retrieval's quality, not the filter's)

Summary and next step

What you learned:

• Pre-filter applies the filter IN the query to the vector DB. Post-filter applies it AFTERWARDS, in code.
• Pre-filter is 5-10x faster and guarantees isolation. Post-filter is bug-prone and has poor recall.
• Three cases where post-filter DOES apply: an engine without support, dynamic logic that can't be pre-computed, a small prototype. Exceptional.
• In multi-tenant, tenant_id must be mandatory. Wrappers like secure_query enforce it.
• The staged fallback relaxes optional filters without breaking isolation.
• The main trap: post-filter "because it's easier". The cost at scale is enormous.
• Guardrails: a pre-commit hook that forbids direct collection.query(), plus automated isolation tests.

Checkpoint: before moving on, you should be able to:

• Explain the two structural differences between pre and post filter (latency + recall).
• Implement secure_query with a mandatory tenant_id.
• Design a staged fallback that does NOT relax security.

Next capsule: 03 — Designing the metadata schema.

You know pre-filter is the choice. But pre-filter only works if your metadata is well indexed and designed for the filters you'll need. Capsule 03 covers how to design the schema from the start so you don't have to re-index 6 months later because you're missing a field.

Resources

1. ChromaDB — Where Clauses — Pre-filter syntax
2. Pinecone — Metadata Filtering Best Practices — A comparison of patterns
3. OWASP — Multi-Tenancy Security — Isolation risks
4. GDPR Article 32 — Security of Processing — Compliance and isolation
5. LangChain — Self Querying Retriever — Auto-building filters from queries
6. Anthropic — Contextual Retrieval — Complementary patterns

Estimated time: 25-30 minutes Next: 03-designing-the-metadata-schema.md