Module 4: Proximity Search

7. Capstone Exercise: Designing a Search Strategy

Next →

Exercise overview

This is the capstone exercise for Module 4. Here you're going to apply everything you've seen: given a use case (dataset size, latency and precision requirements), you'll decide which index to use, which parameters to configure, and you'll justify your decisions.

Part 1: Use cases

For each case, decide:

  1. Which index should you use? (Brute force, HNSW, IVF, IVF-PQ)
  2. Which parameters should you configure?
  3. What precision and latency do you expect?

Case 1: A personal blog

Context:

  • 1,000 articles
  • Semantic search on the blog
  • Low traffic (10 queries/day)

Requirements:

  • Latency: < 500ms is acceptable
  • Precision: High preferred
  • Budget: Minimal (cheap hosting)
See solution

Index: Brute force

Justification:

  • 1,000 vectors → brute force is ~10ms (very fast)
  • No investment in a complex index
  • 100% precision guaranteed
  • Low traffic → it doesn't need extreme optimization

Implementation: A simple vector database or even in-memory computations.

Case 2: An e-commerce site with a mid-sized catalog

Context:

  • 200,000 products
  • Semantic product search
  • Medium traffic (1,000 queries/day)

Requirements:

  • Latency: < 100ms
  • Precision: ~98%
  • Budget: Medium
See solution

Index: HNSW

Parameters:

ef_construction = 200
M = 32
ef_search = 100

Expected:

  • Latency: ~20-30ms
  • Precision: ~98%
  • Memory: ~600K vector-equivalents (3x)

Justification:

  • 200K vectors → the sweet spot for HNSW
  • Latency and precision meet the requirements
  • Moderate infrastructure cost

Vector database: Weaviate or Qdrant (HNSW by default)

Case 3: A company with massive documentation

Context:

  • 50 million internal documents
  • Semantic search for employees
  • High traffic (100,000 queries/day)

Requirements:

  • Latency: < 50ms
  • Precision: ~95% is acceptable
  • Memory budget: Limited
See solution

Index: IVF with Product Quantization (IVF-PQ)

Parameters:

nlist = 7000 (approximately sqrt(50M))
nprobe = 15
PQ compression = 8x

Expected:

  • Latency: ~30-40ms
  • Precision: ~94-96%
  • Memory: ~6.25M vector-equivalents (8x compression)

Justification:

  • 50M vectors → HNSW would be extremely expensive in memory
  • IVF-PQ compresses 8x → economically viable
  • ~95% precision is acceptable (the "missed" ones are marginal)

Vector database: FAISS or Milvus (full IVF-PQ support)

Part 2: Tuning the parameters

The scenario:

You have an HNSW system with:

Dataset: 5M vectors
ef_construction = 200
M = 32
ef_search = 100

Current results:

  • Latency: 25ms (average)
  • Precision: 98%

Question 2.1:

A user complains that search is "slow" (they expect < 15ms). What do you tune?

See solution

Adjustment: Reduce ef_search

ef_search = 50  (down from 100)

Expected:

  • Latency: ~15ms ✅
  • Precision: ~96% (a slight drop)

Justification: Fewer nodes explored → faster, slightly less precise. An acceptable trade-off if 96% is enough.

Question 2.2:

A user reports that some relevant results don't show up. What do you tune?

See solution

Adjustment: Increase ef_search

ef_search = 200  (up from 100)

Expected:

  • Latency: ~40ms (slower)
  • Precision: ~99% ✅

Justification: More nodes explored → a greater chance of finding all the top-K. The user prefers precision over speed.

Part 3: Scalability

Question 3.1:

Your system currently has 1M vectors with HNSW. You estimate growing to 10M in 6 months. What do you plan?

See reflection

Option A: Stay on HNSW

  • Advantage: The same precision (~99%)
  • Disadvantage: Memory grows 10x (expensive)
  • Requires: An infrastructure upgrade

Option B: Migrate to IVF-PQ

  • Advantage: Memory grows less (compression)
  • Disadvantage: Precision drops to ~95%
  • Requires: A full rebuild of the index

Decision: It depends on budget vs precision requirements.

Recommendation: Start planning a migration to IVF if the budget is limited.

Question 3.2:

You currently insert 10,000 new vectors/day. How do you handle inserts?

See reflection

With HNSW:

Option 1 (incremental): Insert directly
- Problem: The quality of the graph degrades over time
- Solution: A periodic rebuild (e.g. weekly)

Option 2 (batch): Accumulate the inserts, rebuild 1x/week
- Advantage: Optimal graph quality
- Disadvantage: New vectors aren't available immediately

With IVF:

Insert directly (reassign to a cluster)
- Easier than with HNSW
- Periodic re-clustering (e.g. monthly)

The typical decision: A weekly batch with HNSW (a balance of quality and freshness).

Part 4: Debugging

The problem: High latency (200ms vs the expected 30ms)

Possible causes:

  1. ef_search is too high
  2. The dataset is much larger than expected
  3. Cold start (the first query after a deployment)
  4. The index isn't loaded in memory (it's reading from disk)

How do you investigate?

See guide

Step 1: Check the dataset size

If the dataset grew from 1M to 10M → it's expected to be slower

Step 2: Review the configuration

ef_search = 500 → too high, reduce it to 100

Step 3: Warm-up

Run dummy queries after a deployment

Step 4: Monitor memory

If the index doesn't fit in RAM → it swaps to disk → slow
Solution: Upgrade the RAM or use IVF-PQ (compression)

Exercise summary

What you did:

  1. ✅ You chose indexes for 3 use cases
  2. ✅ You tuned the parameters based on the requirements
  3. ✅ You planned for scalability (1M → 10M)
  4. ✅ You debugged common problems

The intuition, consolidated:

"There's no perfect index for everything. The decision depends on size, latency, precision and budget. HNSW is the sweet spot for most cases (100K-10M); IVF-PQ is for massive scale."

Module 4 conclusion

Congratulations on completing Module 4: Proximity Search. 🎉

What you achieved:

  1. ✅ You understand the problem of searching across millions of vectors
  2. ✅ You know kNN conceptually
  3. ✅ You can compare indexes (exact vs approximate)
  4. ✅ You understand HNSW (hierarchical graphs)
  5. ✅ You understand IVF (clustering)
  6. ✅ You can decide which index to use for a given case
  7. ✅ You can tune the parameters to optimize

The module's key intuition:

Vector databases don't do exhaustive search (slow). They use approximate indexes (HNSW, IVF) that are 1000x faster with ~95-99% precision. The trade-off (speed vs perfection) is acceptable in production.

Next module: Module 5: Keyword vs Semantic Search — Comparing traditional and vector search, and when to use each.

← Previous6. Trade-offs and Decisions: Choosing the Right IndexNext →1. Module Introduction: Keyword vs Semantic Search