Module 8: The Capstone Final Project
2. Stage 1: The Project's Requirements and Constraints
Overview
Before designing, you need to understand the problem thoroughly. This stage covers the functional requirements, the non-functional ones, the constraints, and the target users.
The complete case: An academic RAG system
Organizational context
The client: A public university (20K students, 1K professors)
The current problem:
- Students search for papers in an old system (keyword only)
- Many failed searches ("I couldn't find anything useful")
- Average search time: 15-30 minutes per paper
- Researchers duplicate effort (they can't find prior work)
The vision:
An intelligent chatbot that answers questions and suggests relevant papers instantly.
Functional requirements
FR1: Conceptual semantic search
User: "papers about deep learning applied to medicine"
The system:
→ Finds papers even if they do NOT contain exactly "deep learning" or "medicine"
→ Understands synonyms: "neural networks", "healthcare", "medical imaging"
→ Returns the top-20 most relevant papers
Success metric: Precision@10 > 0.75
FR2: Exact search (author, title, year)
User: "María González's 2021 thesis"
The system:
→ Exact filter: author="María González" AND year=2021
→ Returns only theses (not papers, not other documents)
Success metric: Recall@5 = 1.0 (it finds every exact match)
FR3: Q&A over the content
User: "What methodology does Smith's NLP paper use?"
The system:
→ Finds Smith's paper
→ Extracts the methodology section
→ The LLM generates a summary: "The paper uses a transformer-based model..."
Success metric: Cited answers (with a source) in 90% of cases
FR4: Suggesting related papers (diversity)
The user searches for: "optimization algorithms"
The system returns:
- Paper A: Gradient descent (optimization)
- Paper B: Genetic algorithms (optimization)
- Paper C: Simulated annealing (optimization)
- Paper D: Applications in robotics (a related field)
→ Diversity: multiple approaches + a related field
Success metric: Diversity score (disciplines) ≥ 3 in the top-20
FR5: Metadata filters
Available filters:
- Author
- Year (range: 2000-2024)
- Department (CS, Physics, Biology, Math, etc.)
- Type (paper, thesis, study guide)
User: "CS papers about AI after 2020"
The system:
→ Filter: department=CS AND topic~"AI" AND year>=2020
Non-functional requirements
NFR1: Latency
Target: < 2 seconds (p95)
An acceptable breakdown:
- Query embedding: < 200ms
- kNN search: < 100ms
- Reranking: < 300ms
- LLM generation: < 1500ms
Total: 2100ms
NFR2: Scalability
Initial dataset: 100K documents
Growth: +5K documents/year
The system must support:
- 200K documents without latency degradation
NFR3: Availability
Uptime target: 99.5% (SLA)
→ Maximum downtime: 3.65 hours/month
NFR4: Cost
Budget: $2000/month
Expected queries: 50K/month
(20K students × 2.5 queries/month on average)
Constraints
Constraint 1: Private data (it can't leave the university)
→ Do NOT use external APIs for embeddings (OpenAI, Cohere)
→ Option: A local model (Sentence-BERT)
Exception: The LLM can be an external API (GPT-4) if the query data isn't sensitive.
Constraint 2: A limited budget ($2000/month)
→ Prioritize cost-effective solutions
→ Self-host where possible
Constraint 3: A small team (2 engineers)
→ Avoid very complex solutions (multiple services)
→ Prioritize managed services (less maintenance)
Constraint 4: Documents in multiple formats
Formats:
- 60% PDFs (papers)
- 30% LaTeX source (theses)
- 10% DOCX (study guides)
→ The system must parse every format
Target users
Profile 1: Undergraduate students (50% of queries)
Typical queries:
- "papers about [a course topic]"
- "study guides for [a subject]"
- "a summary of [a concept]"
Expectations:
- Low latency (< 2s)
- Results that are easy to understand
Profile 2: Researchers (40% of queries)
Typical queries:
- "papers about [a very specific topic]"
- "work by [a specific author]"
- "the methodology of [a paper]"
Expectations:
- High precision (Precision@10 > 0.80)
- Diversity in the results
Profile 3: Professors (10% of queries)
Typical queries:
- "material for [a course]"
- "recent papers about [a topic]"
Expectations:
- Advanced filters (year, department)
The dataset's data
Statistics:
Total documents: 100K
Distribution by type:
- Papers (journals): 60K
- Theses (masters/PhD): 25K
- Study guides: 15K
Distribution by department:
- CS: 30K
- Physics: 20K
- Biology: 20K
- Math: 15K
- Others: 15K
Distribution by year:
- 2000-2010: 20K
- 2011-2020: 50K
- 2021-2024: 30K
Average size:
- A paper: 8 pages (~10K tokens)
- A thesis: 80 pages (~100K tokens)
- A guide: 20 pages (~25K tokens)
Total tokens: ~3.5B tokens
(60K papers × 10K + 25K theses × 100K + 15K guides × 25K
= 600M + 2,500M + 375M ≈ 3,475M)
A summary of the critical requirements
Functional:
- ✅ Conceptual semantic search
- ✅ Exact search (metadata)
- ✅ Q&A with an LLM
- ✅ Diversity in the results
Non-functional:
- ✅ Latency < 2s (p95)
- ✅ Cost < $2000/month
- ✅ Scalable to 200K docs
Constraints:
- ✅ Local embeddings (private data)
- ✅ A limited budget
- ✅ A small team
Your task (Stage 1)
Now that you know the requirements and constraints, analyze:
-
Which requirements conflict with each other?
(E.g. latency < 2s vs local embeddings) -
Which constraints drive the technical decisions?
(E.g. private data → NO OpenAI for embeddings) -
What trade-offs do you anticipate?
(E.g. precision vs latency vs cost)
Next stage: 03-architecture-design.md — Designing the system's complete architecture.