Module 7: Production with Pinecone — the migration from "working demo" to "24/7 service"
Module 7: Production with Pinecone — the migration from "working demo" to "24/7 service"
Module description
Up to now you've built an advanced RAG: strategic chunking (M02), query optimization (M03), re-ranking (M04), hybrid search (M05), metadata filtering with multi-tenant isolation (M06). All of it running on a local ChromaDB. It works, the benchmarks confirm the quality, the isolation tests pass.
But there's a leap between "it works on my laptop with 100K docs" and "it works in production with 10M docs serving 50 customers 24/7". That leap is what this module covers.
A local ChromaDB is excellent for iterating. But at scale, you run into: latency that grows with the corpus, RAM that isn't enough, no replication for high availability, no auto-backup, no SLA. Those problems don't get solved by optimizing code — they get solved by migrating to a managed vector DB built for scale. Pinecone is the industry standard.
By the end of this module you'll be able to:
- ✅ Decide when to migrate to Pinecone (on objective criteria, not on hype)
- ✅ Configure a serverless Pinecone index with the right dimensions and metric
- ✅ Implement the ChromaDB → Pinecone migration without losing data or breaking the queries
- ✅ Use Pinecone's namespaces as native multi-tenant isolation
- ✅ Adapt M06's metadata filtering to Pinecone's syntax
- ✅ Compute the real total cost (TCO) and compare it with staying on ChromaDB
Estimated module time: 4-5 hours (8 capsules).
The transition from local to managed
Before (local, with ChromaDB):
Laptop / a single server
┌─────────────────────────┐
│ App + ChromaDB │
│ Same process │
│ Shared RAM │
│ No replication │
│ Manual backups │
└─────────────────────────┘
Pros: fast to iterate, free, total control
Cons: doesn't scale, no HA, no SLA
After (production, with Pinecone):
Your app Pinecone (managed)
┌──────────────┐ ┌──────────────────────┐
│ Application │ ────HTTP─→ │ Distributed cluster │
│ (Pinecone │ (50ms) │ - Auto-scaling │
│ client) │ │ - Replication │
│ │ ←─────── │ - Auto backups │
└──────────────┘ │ - SLA 99.95% │
└──────────────────────┘
Pros: scales to billions, HA, SLA, no ops burden
Cons: recurring cost, vendor lock-in, network latency
The change is NOT just the SDK. It involves:
- Designing the namespace schema (how you organize your tenants).
- Adapting your filters to Pinecone's syntax.
- Rethinking the batch ingestion strategy for large volumes.
- Operations: monitoring, backups, costs.
- The actual migration, with minimal downtime.
When you SHOULD migrate to Pinecone
| Symptom or requirement | Migrate? |
|---|---|
| Corpus < 500K docs, growing slowly | ❌ No, ChromaDB is fine |
| Corpus > 1M docs and growing | ⚠️ Start evaluating |
| Corpus > 10M docs | ✅ Yes, clearly |
| You need an SLA with measurable uptime (99.9%+) | ✅ Yes |
| Your team can't operate databases | ✅ Yes (managed removes all that burden) |
| Multiple instances of the app need to share the index | ✅ Yes (or ChromaDB in server mode) |
| Strict compliance requiring audit logs and disaster recovery | ✅ Yes |
| Unpredictable traffic spikes | ✅ Yes (auto-scaling) |
| The budget is severely constrained | ❌ Stay on ChromaDB as long as you can |
A practical rule: most products end up migrating. The question is when, not whether.
Module map
| Capsule | Topic | Time |
|---|---|---|
| 01 (you're here) | Module introduction | 10-15 min |
| 02 | ChromaDB's limits in production | 25-30 min |
| 03 | Setting up serverless Pinecone | 30 min |
| 04 | The ChromaDB → Pinecone migration | 35 min |
| 05 | Namespaces as native multi-tenant isolation | 30 min |
| 06 | Metadata filtering in Pinecone (the new syntax) | 30 min |
| 07 | Benchmarking + cost calculation | 30 min |
| 08 | Capstone project — a Production RAG System | 60 min |
Total: 4-5 hours. This module is operational — more about infrastructure than about algorithms.
Connection with the previous modules
What you already built:
├─ M01: A working RAG pipeline
├─ M02-M03: Retrieval quality (chunking, query opt)
├─ M04: Re-ranking
├─ M05: Hybrid search (semantic + BM25 + RRF)
└─ M06: Metadata filtering with tenant isolation
What changes with Pinecone:
├─ The vector DB backend: ChromaDB → Pinecone
├─ Multi-tenancy: workspace_id in metadata → namespaces
├─ Hybrid search: local rank_bm25 → Pinecone sparse vectors (native BM25)
├─ Metadata filter: where clauses → a similar but different filter syntax
└─ Operations: your infra → Pinecone's managed service
What does NOT change:
├─ Re-ranking (the local cross-encoder keeps running)
├─ Query optimization (transparent to the backend)
├─ The metadata schema (with minor adjustments)
└─ The eval set and the metrics
The key point: module 7 is infrastructure, not algorithms. Retrieval quality doesn't depend on Pinecone vs ChromaDB — it depends on how you built M02-M06. Pinecone solves the operational problems of running that same quality at scale.
Pinecone's approximate cost
To size it up:
| Pinecone plan | Approximate cost | When |
|---|---|---|
| Starter (free) | $0 | Up to 100K vectors, a single project. Ideal for evaluating. |
| Standard (serverless) | $50-300/month | Typical production, 1-10M vectors |
| Enterprise | $1K+/month | 10M+ vectors, SLA 99.99%, dedicated support |
Comparison with self-hosted ChromaDB:
ChromaDB on your own VM (10M vectors):
- A VM with 32GB RAM, SSD: $200-400/month
- The engineering cost to maintain it (10-15 hrs/month): $500-1500/month
- No SLA, downtime until you fix it yourself
- Effective total: $700-2000/month
Pinecone Standard (10M vectors):
- $300-500/month
- 0 engineering hours to maintain
- SLA 99.95%
- Total: $300-500/month
The typical reading: for >5M vectors, or for teams with no dedicated DevOps, Pinecone is cheaper in TCO.
The module's limits
- ❌ We don't cover deploying your own Kubernetes — autoscaling your app is generic, not RAG-specific.
- ❌ We don't cover advanced multi-region disaster recovery — Pinecone Enterprise manages it, and it's out of scope.
- ❌ We don't cover enterprise cost optimization — volume discounts are negotiated with Pinecone, not computed.
- ❌ We don't cover alternative vector DBs (Qdrant, Weaviate, Milvus) — the concepts transfer but the syntax changes. Pinecone is the most widely used one, so we cover it in detail.
Prerequisites before starting
Make sure you have:
- ✅ A working RAG pipeline with M02-M06 implemented
- ✅ A Pinecone account (the free Starter is enough to learn)
- ✅ The
PINECONE_API_KEYenvironment variable configured - ✅ Your own eval set to validate that the migration doesn't degrade quality
- ✅ Management sign-off for the recurring cost
Technical setup
pip install pinecone-client>=4.0# .env
PINECONE_API_KEY=...
PINECONE_INDEX_NAME=production-rag
OPENAI_API_KEY=...Self-check before moving on
Before starting capsule 02, make sure you can answer:
- Why is migrating to Pinecone NOT optional for systems at scale?
- Which components of the pipeline change when you migrate, and which stay?
- When is it NOT worth migrating (a small corpus, a team with no room for a recurring cost)?
Answers
-
A local ChromaDB doesn't scale beyond a certain point: limited RAM, no replication, no SLA, manual operations. At scale (10M+ vectors), those problems are show-stoppers. Pinecone solves them with a managed layer.
-
Changes: the vector DB backend (ChromaDB → Pinecone), the isolation (a metadata filter → namespaces), hybrid search (rank_bm25 → Pinecone sparse vectors), the filter syntax. Stays: re-ranking (the local cross-encoder), query optimization (transparent), the metadata schema (with minor adjustments), the eval set, generation with the LLM.
-
When the corpus is <500K docs and grows slowly, when you have a dedicated DevOps team to operate ChromaDB, when the budget is severely constrained (Pinecone costs $50-500/month), when you're at the MVP/prototype stage and the migration is premature. For 80% of products, migrating makes sense once the corpus passes 1M docs.
Next step: capsule 02
The next capsule gets into the detail: what exactly breaks with ChromaDB at scale? You'll see the concrete symptoms that signal it's time to migrate, the benchmarks that justify the decision, and the objective criteria for not migrating prematurely.
Resources
- Pinecone Documentation — The complete official documentation
- Pinecone — Serverless Indexes — To get started
- Pinecone — Multitenancy — Namespaces
- Pinecone Pricing — Current costs
- Vector Databases in Production (Pinecone Blog) — An operational guide
- ChromaDB vs Pinecone Comparison — For the decision
Estimated time: 10-15 minutes Next: 02-chromadb-limits-in-production.md