8. Capstone Exercise: Designing a Search System

Exercise overview

This is the capstone exercise for Module 6. Here you're going to design a complete search system for a specific case, making decisions about chunking, embeddings, the vector DB, ranking, and metrics.

The case: An academic search engine

Context:
A university wants an internal search engine for 100K scientific papers (PDF) across multiple disciplines (CS, physics, biology, mathematics).

Requirements:

1. Typical queries:

   • Conceptual: "optimization methods in machine learning"
   • Exact: "Hinton's 2012 paper"
   • Exploratory: "the latest advances in quantum computing"

2. Features:

   • Search by author, year, discipline
   • Suggest related papers (diversity)
   • Prefer recent papers (a time boost)

3. Constraints:

   • Latency < 1 second
   • A limited budget (prefer local models if possible)
   • High precision (academics are demanding users)

The task: Design the architecture

For each component, decide:

1. Chunking strategy

2. Embedding model

3. Vector database

4. Search strategy

1. Detect the query pattern:
   Does it contain "[author]'s [year] paper"?
   → Exact filter: author + year + semantic search

   Does it contain a discipline ("in CS")?
   → Filter: discipline=CS + semantic search

   Otherwise:
   → Hybrid search (α=0.7, favoring semantic)

2. Run the search (top-100)

3. MMR (λ=0.6, diversify by discipline and author)

4. Time boost (papers < 2 years old: 1.3x)

5. Return the top-20

5. Ranking strategy

Keyword + semantic fusion
→ RRF with k=60

λ = 0.6
Diversify by:
- Discipline (not 20 CS papers if the query is general)
- Author (not 20 papers from the same group)

Papers < 2 years old: 1.3x
Papers 2-5 years old: 1.1x
Papers > 5 years old: 1.0x

6. Query processing latency

1. Embedding the query (local SPECTER): 50ms
2. kNN search (Weaviate HNSW): 80ms
3. MMR (diversifying 100 → 20): 100ms
4. Total: ~230ms

7. Evaluation metrics

Architecture diagram (the solution)

┌─────────────────────────────────────────────────┐
│              100K PAPERS (PDF)                  │
└────────────┬────────────────────────────────────┘
             │
             v
    ┌────────────────┐
    │  Hierarchical  │ (By section)
    │    Chunking    │
    └────────┬───────┘
             │
             v
    ┌────────────────┐
    │  SPECTER       │ (Local embeddings)
    │  768D          │
    └────────┬───────┘
             │
             v
    ┌─────────────────────────────────────┐
    │  WEAVIATE (self-hosted)             │
    │  - HNSW index                       │
    │  - Metadata: author, year, discipline│
    └────────┬────────────────────────────┘
             │
             │
    ┌────────┴────────┐
    │                 │
    v                 v
┌─────────┐    ┌──────────────┐
│ Keyword │    │   Semantic   │
│  BM25   │    │   (cosine)   │
└────┬────┘    └──────┬───────┘
     │                │
     └────────┬───────┘
              │
              v
     ┌────────────────┐
     │   RRF Fusion   │
     └────────┬───────┘
              │
              v
     ┌────────────────┐
     │      MMR       │ (λ=0.6, diversity)
     └────────┬───────┘
              │
              v
     ┌────────────────┐
     │   Time Boost   │ (< 2 years: 1.3x)
     └────────┬───────┘
              │
              v
     ┌────────────────┐
     │     Top-20     │
     └────────────────┘

Exercise summary

What you did:

1. ✅ You designed a chunking strategy (hierarchical)
2. ✅ You chose an embedding model (local SPECTER)
3. ✅ You selected a vector DB (Weaviate)
4. ✅ You defined a search strategy (hybrid + MMR)
5. ✅ You established a ranking approach (RRF + MMR + boost)
6. ✅ You estimated the latency (200-300ms)
7. ✅ You defined the metrics (Precision, NDCG, diversity)

The intuition, consolidated:

"Designing a search system requires informed decisions at every component: chunking, embeddings, the vector DB, ranking. There's no single solution; it depends on the requirements (latency, budget, precision) and the type of queries."

Module 6 conclusion

Congratulations on completing Module 6: Designing Search Systems. 🎉

What you achieved:

1. ✅ You understand end-to-end architecture (indexing, query processing)
2. ✅ You can design an indexing pipeline (chunking, embeddings)
3. ✅ You can optimize query processing (kNN, reranking)
4. ✅ You can implement advanced ranking (RRF, MMR, boost)
5. ✅ You can evaluate quality (precision, recall, MRR, NDCG)
6. ✅ You can analyze case studies (RAG, e-commerce, support)

The module's key intuition:

An effective search system integrates multiple techniques (chunking, embeddings, ANN, ranking, filters) into a coherent architecture. The design decisions depend on specific requirements: latency, budget, the type of queries, and the expected precision.

Next module: Module 7: RAG (Retrieval-Augmented Generation) — Full integration with LLMs.