Module 8: Capstone RAG Project with ChromaDB

Module description

This module consolidates everything learned in the guide into a final project: a complete, testable, and deployable RAG system. The goal is not just that it works, but that it has engineering quality and is ready for your portfolio.

What makes this module different?

In modules 1-7 you learned concepts and techniques separately. Here you integrate everything into an end-to-end flow:

Modules 1-2: Why vector databases and how they work internally → you choose ChromaDB on solid grounds.
Module 3: Metadata filtering, hybrid search, multi-tenancy → you use them to filter by source.
Modules 4-5: ChromaDB hands-on, batch operations → you implement ingestion at scale.
Module 6: Decision matrix to choose a DB → you justify ChromaDB in this context.
Module 7: Production considerations → you apply observability, scaling, and hardening.

It's not new theoretical learning. It's practical integration: connecting the pieces, making design decisions, writing production-ready code, and documenting it for future evolution.

Concept map by module

So you can see concretely what you apply from each module:

Module	Key concept	Where you'll use it in the project
1	Why a vector DB vs SQL/NoSQL	README, architecture justification
1	RAG needs fast semantic search	Retrieval pipeline design
2	HNSW, approximate indexes	ChromaDB uses HNSW by default; you understand why it's fast
2	Recall vs latency trade-off	Choice of top_k and score_threshold
3	Metadata filtering	Filters by doc_id, source in queries
3	Batch operations	Ingestion in batches of 1K-2K
4	ChromaDB CRUD, PersistentClient	The entire vector store
4	Similarity search with metadata	`/search` and `/ask` endpoints
5	Pinecone/Weaviate comparison	Decision to use ChromaDB now, migrate later
6	Decision matrix (cost, scale, self-hosted)	Justification in documentation
7	Observability, scaling, migrations	Logs, health check, Docker

This mapping helps you not "forget" what you learned and reinforces the theory-practice connection.

Module goal

Build an end-to-end RAG solution with:

Ingestion for 1,000+ documents: a robust pipeline with batches, consistent metadata, and validation.
Retrieval with metadata filters: top-k, where clauses, and handling of low scores.
Generation with citations: answers grounded in evidence, verifiable sources.
REST API for external consumption: FastAPI with Swagger/ReDoc, stable contracts.
Basic testing and observability: automated tests, metrics, and traceability.

The result will be a system you can show in a portfolio, deploy with Docker, and use as a direct base for Guide #8 (Advanced RAG Techniques).

Connection with Guide #8 (Advanced RAG)

This project is designed to be reusable. When you move on to Guide #8:

The same ingestion flow will adapt to Pinecone or Weaviate with minimal changes.
The modular architecture (ingestion ↔ retrieval ↔ generation) makes swapping components easier.
The API contract (/ask, sources, trace_id) will be preserved; what changes is the vector store.

Thinking about evolution from the design stage saves you from rewriting code later.

Concrete reusability example

In Guide #8 you'll work with Pinecone. The change would be roughly:

# Current (ChromaDB)
from chromadb import PersistentClient
client = PersistentClient(path="./chroma")
collection = client.get_collection("rag_docs")
results = collection.query(query_embeddings=[embedding], n_results=5)

# Guide #8 (Pinecone)
from pinecone import Pinecone
pc = Pinecone()
index = pc.Index("rag-index")
results = index.query(vector=embedding, top_k=5, include_metadata=True)

The retrieval logic (embed query, run the search, filter by score) is the same. Only the client changes. If you abstract the vector store behind an interface, migrating is a matter of swapping one implementation.

Capsule structure

Capsule	Content	Focus
01	Project introduction	What you integrate, success criteria, work approach
02	Architecture and design	Components, data flow, design decisions
03	Ingestion pipeline	1,000+ docs, chunking, embeddings, ChromaDB
04	Retrieval + Generation + API	Endpoints, `/ask`, citations, FastAPI
05	Testing and evaluation	Unit, integration, quality, performance
06	Observability and deployment	Logs, metrics, Docker
07	Final hardening	Security, rate limiting, error handling
08	Project delivery	Production-ready checklist, documentation

Module success criteria

Upon completing the project, you should be able to state:

Stable ingestion pipeline for 1,000+ documents (reasonable time, no massive errors).
/ask endpoint that returns grounded answers, sources, and an explicit fallback when there is not enough context.
Minimal set of automated tests (unit + integration) that pass reproducibly.
Reproducible deployment with Docker (an image that brings up the API + ChromaDB with a persistent volume).
Evidence of basic observability: structured logs, latency metrics, and a trace_id in responses.

Recommended work approach

1. Design the architecture first

Before writing code, define:

Components and responsibilities.
Contracts between components (what passes between ingestion → retrieval → generation).
Externalized configuration (environment variables, not hardcoded values).

2. Build a functional vertical slice

Implement a minimal flow that works end to end:

Ingestion of 10-20 documents.
Manual retrieval.
An /ask endpoint that returns an answer.

Validate that the flow makes sense before scaling.

3. Harden with tests, metrics, and hardening

Once functional, add:

Automated tests.
Error handling and limits.
Logs and metrics for diagnosis.

4. Document decisions to enable evolution

In a README or DESIGN.md, record:

Why you chose ChromaDB (vs others).
Why chunk_size=512, batch_size=1000, etc.
Dependencies and versions.

Mindset: don't seek perfection in v1

A frequent mistake is trying to make the first version have everything: exhaustive tests, rate limiting, advanced metrics, perfect documentation. That dilutes the focus.

Recommended approach:

v1: A functional end-to-end pipeline. Ingestion of 1K docs, /ask that answers, Docker that brings it up. Clean but not over-designed code.
v1.1: Minimal tests that show the core works. A health check, one /ask integration test.
v1.2: Basic observability: logs with a trace_id, latency metrics if you have time.
v2 (optional): Rate limiting, alerts, migration to Pinecone for Guide #8.

Prioritize verifiable functionality over future complexity.

Suggested route (approximate timeline)

Day / Session	Capsules	Deliverable
1	01, 02	Design on paper, documented decisions
2	03	Ingestion pipeline working with 100+ docs
3	04	API with `/ask` and `/search` working
4	03 (scale)	Ingestion of 1,000+ docs verified
5	05, 06	Tests, Docker, basic observability
6	07, 08	Hardening, checklist, delivery

Adjust to your pace. The critical thing is to have the vertical slice (02→03→04) working before hardening.

What you need to start

Technical prerequisites:

Python 3.10+
ChromaDB installed (module 4)
OpenAI account (for embeddings and generation)
FastAPI and the guide's dependencies (modules 4-5)

Prior knowledge:

Having completed modules 1-7 of this guide (or equivalent).
Familiarity with REST APIs and the structure of a Python project.

Pre-project exercises

The following exercises prepare you before coding:

Exercise 1: Map modules to components

List which concept from each module (1-7) you'll use in the project. Example:

Module 1 (why a vector DB) → justification in the README.
Module 4 (ChromaDB CRUD) → add/query operations.
etc.

Solution: Create a table or list with module ↔ project component. This forces you to recall and connect prior learning.

Exercise 2: Define the `/ask` contract

Before implementing, write the JSON response you want for /ask. Include: answer, sources, confidence, trace_id. What will you do if no documents are retrieved?

Solution: Define a schema like:

{
  "answer": "string",
  "sources": [
    {"doc_id": "id", "title": "string", "score": 0.82}
  ],
  "confidence": 0.0,
  "trace_id": "uuid",
  "fallback_reason": null
}

If there is no evidence: answer = an explicit message "I don't have enough information", sources = [], confidence = 0, fallback_reason = "insufficient retrieval".

Exercise 3: Ingestion time estimate

If you process 1,000 documents with:

Chunking: ~100 ms per doc (approx).
Embeddings: batch of 100, ~1 s per batch (OpenAI).
ChromaDB add: batch of 1000, ~0.2 s per batch.

Estimate the approximate total time. Where is the bottleneck?

Solution: Order of magnitude:

Chunking: 100 docs × 100 ms = 10 s (parallelizable).
Embeddings: 10 batches × 1 s = 10 s (limited by the API).
ChromaDB: 1 batch × 0.2 s = 0.2 s.

The typical bottleneck is embedding generation (API calls). That's why batches are used and, in production, you consider caching or local models.

Exercise 4: Production checklist (upfront)

Before implementing, review the production checklist from module 7. Mark which items you'll apply in this minimal project and which you'll leave for a second iteration.

Solution: Example prioritization:

Now: Basic logs, health check, environment variables, Docker.
Later: Advanced rate limiting, metrics with Prometheus, alerts.

Exercise 5: Reusability for Guide #8

List 3 changes you would have to make if tomorrow you migrate from ChromaDB to Pinecone. Where is the vector store coupled in your design?

Solution: Typical changes:

Replace the ChromaDB client with the Pinecone client.
Adapt the ID and metadata format (Pinecone has its own restrictions).
Change the collections configuration to indexes.

If you separated a vector_store module or similar with an abstract interface, the change concentrates there. If ChromaDB is scattered across the code, the migration will be costly.

Exercise 6: Prioritizing success criteria

The success criteria are: (1) ingestion of 1K+ docs, (2) /ask with sources and fallback, (3) minimal tests, (4) Docker, (5) observability. In what order would you implement them and why?

Solution: Suggested order:

/ask with sources and fallback — It's the heart of the product. Without this, there's no RAG.
Ingestion of 1K+ docs — You need data to test real retrieval. You can start with 100 and scale.
Docker — It makes reproducibility and deployment easier. Relatively fast to add.
Minimal tests — They validate you don't break anything when changing. One /ask integration test is enough for v1.
Observability — Logs and trace_id have high impact with low effort; metrics can wait.

Pre-development troubleshooting

"I'm not clear on where to start"

Prioritize: 1) architecture on paper/diagram, 2) a minimal flow (10 docs, 1 endpoint), 3) scale and harden. Don't try to make everything perfect in the first iteration.

"I'm afraid the project is too big"

The minimum scope is: ingestion of 1,000 docs, a functional /ask, Docker that brings everything up. Tests and observability can be basic. Reduce features before reducing core quality.

"I don't know if my design is correct"

Review capsule 02 (Architecture). The key criteria: separated components, explicit contracts, externalized configuration. If you meet those, the design is reasonable. Perfection is not the goal; coherence is.

"I want to use another DB instead of ChromaDB"

For this module, use ChromaDB: it's the focus of the guide, has no API cost, and is easy to install. When you move to Guide #8, you'll migrate to Pinecone or another. The design exercise prepares you for that change.

"Do I have to implement everything from scratch?"

You can reuse code from modules 4-5 (ChromaDB, batch ingestion). Adapt and extend; don't start from scratch on what you've already seen. The value is in the integration and the production decisions.

Common mistakes when starting

Mistake	Why it happens	How to avoid it
Starting to code without a design	Anxiety to "see something working"	Spend 30-60 min on capsule 02 before touching code
Over-designing the abstraction	Fear of coupling, experience with large projects	For 1K docs and a single vector store, a simple interface is enough; don't build 5 layers of abstraction
Ignoring the fallback	Assuming retrieval will always find something	Implement it from day 1: "I don't have information" when retrieval is empty
Hardcoding API keys	Initial speed	Use `python-dotenv` and `.env` from the first commit; add `.env` to `.gitignore`
Not documenting decisions	"I'll remember it"	Write 2-3 paragraphs in the README or DESIGN.md; in 2 weeks you'll have forgotten it

Summary

This module consolidates what you learned in modules 1-7 into a complete RAG system.
The project is integration, not new theoretical content: you connect ingestion, retrieval, generation, and the API.
The result is portfolio-worthy: deployable, documented, and testable.
The design is reusable: a direct base for Guide #8 (Advanced RAG) with Pinecone or another DB.
Focus on architecture first, then a minimal flow, then hardening.
Success criteria: ingestion of 1K+ docs, /ask with sources and fallback, tests, Docker, basic observability.
Document design decisions to enable future evolution and migration.

Additional resources

FastAPI Documentation
ChromaDB Documentation
Docker Documentation
RAG Paper (Lewis et al.) — "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks"
LangChain Text Splitters — RecursiveCharacterTextSplitter and alternatives
OpenAI Embeddings API
Vector Databases Fundamentals README — Full context of the guide
Production RAG Considerations — Checklist and patterns

Reading checklist

Before moving to capsule 02, verify that you can answer:

What do you integrate from each module (1-7) in this project?
Why is the design reusable for Guide #8?
What are the 5 success criteria?
In what order would you implement: design, vertical slice, tests, Docker?
What to do when retrieval returns 0 documents?

If you can answer the 5 questions, you're ready to design the architecture.

What this module is not

To avoid incorrect expectations:

It's not a complete LangChain tutorial: We use LangChain only for text splitters; the rest is our own code.
It's not large-scale production deployment: Docker and basic observability yes; K8s, load balancers, and HA are out of scope.
It's not model fine-tuning: We use pre-trained embeddings and an LLM.
It's not exhaustive RAG evaluation: There will be basic tests and metrics; not a complete evaluation framework.

The scope is deliberate: a functional, clean, and extensible system, not a complete enterprise product.

Next steps

After reading this capsule, move to 02-project-architecture.md to define the design. Don't jump straight to the code: the 20-30 minutes you invest in architecture will save you hours of refactoring. If you already have experience with RAG or FastAPI, you can review capsule 02 more quickly, but make sure you're clear on the 5 components and the contracts between them before implementing.

Estimated time: 15-20 minutes
Next: 02-project-architecture.md

← PreviousCapsule 08: Project - Production Readiness Checklist Next →Capsule 02: Project Architecture