Capsule 03: Managed vs Self-Hosted

🎯 Capsule objective

Understand the most important operational trade-off when choosing a vector database: managed service or your own infrastructure? And build a decision framework you can apply to any provider.

This decision is not technical in isolation — it's a business decision that affects delivery speed, long-term costs, data control, and team load. A 3-person team that chooses self-hosted can end up spending 40% of its time on operations instead of product. An enterprise team that chooses managed can face unexpected costs when scaling or compliance constraints it didn't anticipate.

In this capsule you'll break down the advantages, risks, and hidden costs of both models. The goal is not for one to be "better" — it's for you to know which one reduces the most risk for your specific context of team, budget, and product stage.

By the end of this capsule:

• ✅ You'll explain the concrete advantages and risks of managed and self-hosted
• ✅ You'll calculate TCO (Total Cost of Ownership) for both models
• ✅ You'll identify signs that you chose the wrong model
• ✅ You'll apply a decision framework with weighted criteria

Estimated time: 20-25 minutes

🏢 Managed model: "You build RAG, they operate the DB"

What does managed mean?

In the managed model, the provider takes care of:

• Infrastructure provisioning (servers, storage, networking)
• Automatic scaling (vertical and horizontal)
• Backups and disaster recovery
• Software updates and security patches
• Infrastructure monitoring and alerts
• High availability (multi-AZ, failover)

You just interact with the API: create indexes, insert vectors, run queries.

Main managed providers

Example: Managed setup with Pinecone

from pinecone import Pinecone, ServerlessSpec

pc = Pinecone(api_key="YOUR_API_KEY")

pc.create_index(
    name="rag-production",
    dimension=1536,
    metric="cosine",
    spec=ServerlessSpec(
        cloud="aws",
        region="us-east-1"
    )
)

index = pc.Index("rag-production")

index.upsert(
    vectors=[
        {"id": "doc_1", "values": [0.1, 0.2, ...], "metadata": {"source": "api_docs"}}
    ],
    namespace="v1"
)

results = index.query(
    vector=[0.1, 0.2, ...],
    top_k=10,
    namespace="v1",
    include_metadata=True
)

Time from zero to a working query: ~5 minutes (signup + API key + code).

Example: Managed setup with Qdrant Cloud

from qdrant_client import QdrantClient
from qdrant_client.models import VectorParams, Distance

client = QdrantClient(
    url="https://your-cluster-id.aws.cloud.qdrant.io:6333",
    api_key="YOUR_API_KEY"
)

client.create_collection(
    collection_name="rag-production",
    vectors_config=VectorParams(size=1536, distance=Distance.COSINE)
)

Time from zero to a working query: ~10 minutes (signup + create cluster + API key + code).

Advantages of the managed model

Risks of the managed model

Hidden cost: the "lock-in tax"

Scenario: A startup grows from 500K to 5M vectors in 12 months

Month 1-6:   Pinecone serverless = ~$70/month     ← "Dirt cheap"
Month 7-12:  Pinecone serverless = ~$400/month    ← "Still reasonable"
Month 13-18: Pinecone serverless = ~$1,500/month  ← "How much?!"

Cost of migrating to self-hosted in month 13:
- 2-3 weeks of engineering time
- Downtime risk during migration
- Rewriting code that uses the Pinecone SDK
- Performance testing on new infra

"Lock-in tax" = keep paying $1,500/month because migrating
costs more short-term than staying.

🖥️ Self-hosted model: "You operate everything"

What does self-hosted mean?

In the self-hosted model, you are responsible for:

• Server provisioning (VMs, Kubernetes, bare metal)
• Vector database configuration (Docker, Helm charts)
• Manual or semi-automatic scaling (replicas, shards)
• Backups and disaster recovery (cron jobs, scripts, snapshots)
• Version updates (testing, rolling updates)
• Monitoring and alerts (Prometheus, Grafana, custom dashboards)
• Network security (firewalls, TLS, access control)

Main self-hosted providers

Example: Self-hosted deploy with Qdrant (Docker)

# docker-compose.yml
version: '3.8'
services:
  qdrant:
    image: qdrant/qdrant:latest
    ports:
      - "6333:6333"  # REST API
      - "6334:6334"  # gRPC
    volumes:
      - qdrant_data:/qdrant/storage
    environment:
      QDRANT__SERVICE__GRPC_PORT: 6334
    restart: unless-stopped
    deploy:
      resources:
        limits:
          memory: 4G

volumes:
  qdrant_data:

docker compose up -d

Time from zero to a working query: ~15 minutes (Docker + compose + code).

Example: Self-hosted deploy with Milvus (Docker Compose)

# docker-compose.yml (Milvus standalone)
version: '3.8'
services:
  etcd:
    image: quay.io/coreos/etcd:v3.5.5
    environment:
      ETCD_AUTO_COMPACTION_MODE: revision
      ETCD_AUTO_COMPACTION_RETENTION: "1000"
    volumes:
      - etcd_data:/etcd

  minio:
    image: minio/minio:RELEASE.2023-03-20T20-16-18Z
    environment:
      MINIO_ACCESS_KEY: minioadmin
      MINIO_SECRET_KEY: minioadmin
    command: minio server /minio_data
    volumes:
      - minio_data:/minio_data

  milvus:
    image: milvusdb/milvus:v2.4-latest
    command: ["milvus", "run", "standalone"]
    environment:
      ETCD_ENDPOINTS: etcd:2379
      MINIO_ADDRESS: minio:9000
    ports:
      - "19530:19530"
      - "9091:9091"
    depends_on:
      - etcd
      - minio
    volumes:
      - milvus_data:/var/lib/milvus

volumes:
  etcd_data:
  minio_data:
  milvus_data:

Time from zero: ~30-45 minutes (3 services, connectivity debugging).

Critical observation: Qdrant requires 1 container. Milvus requires 3 (+ Pulsar in distributed mode = 4). This difference in operational complexity is fundamental for small teams.

Advantages of the self-hosted model

Risks of the self-hosted model

Hidden cost: the "ops tax"

Scenario: A team of 4 engineers operates Qdrant self-hosted

Infra (3 nodes, 8GB RAM each):
  AWS EC2: $150/month × 3 = $450/month

Operations (monthly average):
  Routine maintenance:          8 hours/month
  Incidents (1 every 2 months): 4 hours/month averaged
  Updates:                      4 hours/month
  Monitoring/alerts:            4 hours/month
  Total:                        20 hours/month

Engineering cost ($80/hour):    20 × $80 = $1,600/month

"Ops tax" total: $450 + $1,600 = $2,050/month

vs. Qdrant Cloud managed: ~$600-900/month

Conclusion: For this team, self-hosted ends up MORE expensive
when you include the team's time.

📊 TCO: How to Calculate the Real Cost

TCO formula

TCO = Direct Cost + Operational Cost + Risk Cost

Where:
  Direct Cost       = Managed service OR infra (VMs, storage, networking)
  Operational Cost  = Team hours × cost/hour
  Risk Cost         = Incident probability × estimated impact

Comparative TCO table (6 months, 1M vectors, 50 QPS)

But... the cost crossover exists

                Monthly cost ($)
                    │
              3000  │              Self-hosted ────────
                    │             /
              2000  │            /        Managed ──────────────
                    │           /        /
              1000  │──────────/────────/
                    │        ↑
                    │   Crossover point
                    │   (~5-10M vectors)
                    └──────────────────────────────── Vectors
                    1M      5M      10M     50M

From ~5-10M vectors with high traffic onward,
self-hosted starts to be cheaper (if the team
already has operational expertise).

TCO decision rule

def recommend_model(vectors_millions, has_devops, months_deadline):
    """Simplified managed vs self-hosted decision framework."""

    if months_deadline <= 2:
        return "MANAGED — the priority is time-to-market"

    if not has_devops:
        return "MANAGED — without an ops team, self-hosted is high risk"

    if vectors_millions < 1:
        return "MANAGED — at this scale infra cost is similar, ops isn't justified"

    if vectors_millions > 10 and has_devops:
        return "SELF-HOSTED — at this scale, infra savings exceed ops cost"

    return "EVALUATE — gray zone, do a detailed TCO calculation"


print(recommend_model(vectors_millions=0.5, has_devops=False, months_deadline=1))
# → "MANAGED — the priority is time-to-market"

print(recommend_model(vectors_millions=15, has_devops=True, months_deadline=6))
# → "SELF-HOSTED — at this scale, infra savings exceed ops cost"

print(recommend_model(vectors_millions=3, has_devops=True, months_deadline=4))
# → "EVALUATE — gray zone, do a detailed TCO calculation"

🧭 Decision Framework: 5 Weighted Criteria

The process

1. Define the 5 criteria.
2. Assign a weight to each one (1-5) based on your project's priority.
3. Score each model (1-5) for each criterion.
4. Multiply weight × score.
5. Add up totals and compare.

Full example

Context: A 6-person startup, RAG for technical support, 800K vectors, no dedicated DevOps, needs a 99.9% SLA.

Result: Managed wins by 23 points → the recommendation is clear.

Another example: Enterprise with a platform team

Context: A 200-person company, an 8-person platform team, 20M vectors, strict GDPR requirements.

Result: Self-hosted wins by 26 points → GDPR and scale dominate the decision.

⚠️ Signs That You Chose the Wrong Model

Signs that self-hosted was a bad idea

🚩 The team spends > 30% of its time operating the DB
🚩 You've gone 3+ months without shipping new product features
🚩 You've had > 2 serious incidents in 6 months
🚩 The team can't take vacations without operational risk
🚩 Version updates are postponed indefinitely

Action: Evaluate migrating to managed. Calculate how much the migration costs vs. how much you lose in undelivered features.

Signs that managed was a bad idea

🚩 The monthly cost grew > 3x in the last 6 months
🚩 You need a feature the provider doesn't support (and it's not on the roadmap)
🚩 Compliance requires on-premise data and the provider doesn't offer it
🚩 The provider's latency won't drop below your SLA and you can't tune it
🚩 The provider had outages that affected your committed SLA

Action: Plan a migration to self-hosted. Define a runway (how many months you can keep paying) and use that time to prepare the infra.

🔄 Hybrid Strategy: The Best of Both Worlds

Pattern: "Start Managed, Graduate to Self-Hosted"

Phase 1 (Month 1-6): Managed
├── Validate the product fast
├── Iterate on RAG without operational load
└── Define success metrics

Phase 2 (Month 7-9): Prepare migration
├── Evaluate 12-month TCO
├── Prototype self-hosted in staging
└── Document operations runbooks

Phase 3 (Month 10+): Self-hosted
├── Migrate with dual-write (both active)
├── Validate latency and accuracy
└── Cut traffic gradually

Code: Abstraction to ease migration

from abc import ABC, abstractmethod
from typing import Any


class VectorStore(ABC):
    """Common interface to ease migration between providers."""

    @abstractmethod
    def upsert(self, id: str, vector: list[float], metadata: dict) -> None:
        pass

    @abstractmethod
    def query(self, vector: list[float], top_k: int, filters: dict | None = None) -> list[dict]:
        pass

    @abstractmethod
    def delete(self, id: str) -> None:
        pass


class PineconeStore(VectorStore):
    def __init__(self, index_name: str, api_key: str):
        from pinecone import Pinecone
        self.index = Pinecone(api_key=api_key).Index(index_name)

    def upsert(self, id: str, vector: list[float], metadata: dict) -> None:
        self.index.upsert(vectors=[{"id": id, "values": vector, "metadata": metadata}])

    def query(self, vector: list[float], top_k: int, filters: dict | None = None) -> list[dict]:
        results = self.index.query(vector=vector, top_k=top_k, filter=filters, include_metadata=True)
        return [{"id": m.id, "score": m.score, "metadata": m.metadata} for m in results.matches]

    def delete(self, id: str) -> None:
        self.index.delete(ids=[id])


class QdrantStore(VectorStore):
    def __init__(self, collection_name: str, host: str = "localhost", port: int = 6333):
        from qdrant_client import QdrantClient
        self.client = QdrantClient(host=host, port=port)
        self.collection = collection_name

    def upsert(self, id: str, vector: list[float], metadata: dict) -> None:
        from qdrant_client.models import PointStruct
        self.client.upsert(
            collection_name=self.collection,
            points=[PointStruct(id=id, vector=vector, payload=metadata)]
        )

    def query(self, vector: list[float], top_k: int, filters: dict | None = None) -> list[dict]:
        results = self.client.query_points(
            collection_name=self.collection, query=vector, limit=top_k
        )
        return [{"id": p.id, "score": p.score, "metadata": p.payload} for p in results.points]

    def delete(self, id: str) -> None:
        from qdrant_client.models import PointIdsList
        self.client.delete(collection_name=self.collection, points_selector=PointIdsList(points=[id]))


# Usage: switching provider = changing one line
store: VectorStore = PineconeStore("rag-index", api_key="...")
# store: VectorStore = QdrantStore("rag-index", host="qdrant-server")

store.upsert("doc_1", [0.1, 0.2, ...], {"source": "api_docs"})
results = store.query([0.1, 0.2, ...], top_k=5)

Benefit: The abstraction layer lets you migrate from Pinecone to Qdrant by changing only the instantiation — without touching the RAG code.

🔧 Choice troubleshooting

1. "We don't know if we'll have high scale"

Cause: Product uncertainty — normal in early phases.

Solution: Choose based on current needs and define an explicit review trigger. Example: "We re-evaluate when we exceed 1M vectors OR when the monthly cost exceeds $500". Without a trigger, the re-evaluation doesn't happen.

2. "We have budget but no operational experience"

Cause: The team is strong in development but weak in DevOps/SRE.

Solution: Managed reduces risk in this configuration. The cost of the service will be lower than the cost of incidents and time lost operating infra without experience. If you want to build operational capacity, do it in staging first, not in production.

3. "We want full control from day 1"

Cause: An engineering-culture stance or compliance requirements.

Solution: Validate that the team can sustain incidents and on-call rotations before committing. Do a "fire drill": simulate a DB outage on a Friday at 5pm. If they have no runbook or available person, self-hosted is high risk.

4. "The CTO wants self-hosted, the VP Product wants managed"

Cause: Different stakeholders optimize for different variables.

Solution: Use the weighted criteria table (previous section). Have both assign weights to the 5 criteria BEFORE discussing options. Usually, aligning weights resolves 80% of the debate.

5. "We started managed and now the cost is unsustainable"

Cause: Unanticipated growth + pricing that scales with usage.

Solution: Plan a migration with dual-write (write to both systems in parallel for 2-4 weeks). Compare latency and accuracy before cutting over. Don't migrate under cost pressure without validating that self-hosted meets your SLAs.

✏️ Exercises

Exercise 1: TCO table for your project

Fill in this table with real or estimated data from your project:

Exercise 2: Weighted criteria

Use the 5-criteria framework for your current project:

Exercise 3: Identify warning signs

Read each scenario and decide whether the sign indicates the chosen model was wrong:

1. A managed team spends $2,000/month but only uses 15% of the capacity.
2. A self-hosted team hasn't updated the Qdrant version in 8 months.
3. A managed team has p95 latency of 200ms but its internal SLA is 100ms.
4. A self-hosted team has 0 incidents in 6 months with documented runbooks.

Exercise 4: Design an abstraction layer

Extend the VectorStore class (from the hybrid strategy section) to add a health_check() method that returns the connection state. Implement it for at least one provider.

from abc import ABC, abstractmethod


class VectorStore(ABC):
    @abstractmethod
    def upsert(self, id: str, vector: list[float], metadata: dict) -> None:
        pass

    @abstractmethod
    def query(self, vector: list[float], top_k: int, filters: dict | None = None) -> list[dict]:
        pass

    @abstractmethod
    def delete(self, id: str) -> None:
        pass

    @abstractmethod
    def health_check(self) -> dict:
        """Return connection state and basic metrics."""
        pass


class QdrantStore(VectorStore):
    def __init__(self, collection_name: str, host: str = "localhost", port: int = 6333):
        from qdrant_client import QdrantClient
        self.client = QdrantClient(host=host, port=port)
        self.collection = collection_name

    def health_check(self) -> dict:
        try:
            info = self.client.get_collection(self.collection)
            return {
                "status": "healthy",
                "vectors_count": info.vectors_count,
                "points_count": info.points_count,
                "segments": len(info.segments) if hasattr(info, 'segments') else "N/A"
            }
        except Exception as e:
            return {"status": "unhealthy", "error": str(e)}

    def upsert(self, id, vector, metadata):
        pass  # (implemented in the previous section)

    def query(self, vector, top_k, filters=None):
        pass

    def delete(self, id):
        pass

Exercise 5: Re-evaluation trigger

Define for your project:

1. A quantitative trigger (number of vectors, monthly cost, QPS).
2. A qualitative trigger (incidents, team, compliance).
3. An action plan for when the trigger fires.

🔗 Connection with the project (Decision Tree)

This capsule feeds your decision tree directly with a key decision node:

Does your team have DevOps/SRE capacity?
├── NO → Managed (Pinecone, Weaviate Cloud, Qdrant Cloud)
│         └── Budget > $500/month? → Pinecone / Qdrant Cloud
│         └── Budget < $500/month? → Free tiers / ChromaDB
└── YES → Scale > 5M vectors?
          ├── YES → Self-hosted (Milvus or Qdrant)
          └── NO → Evaluate TCO managed vs self-hosted

In the next capsule you'll add the axis of features for RAG (metadata filtering, hybrid search, multi-tenancy) to complete the decision criteria.

📝 Summary

• Managed prioritizes speed and simplicity in exchange for control and potential vendor lock-in.
• Self-hosted gives full control but demands a team with real operational capacity.
• The TCO includes service + team hours + incident risk — not just the price of the service.
• For teams without DevOps or in an early phase, managed reduces overall risk.
• For scale > 5-10M vectors with a platform team, self-hosted can be 3-5x cheaper.
• The hybrid strategy (start managed → graduate to self-hosted) mitigates the risks of both models.
• An abstraction layer over the provider's SDK eases future migration.
• Define explicit re-evaluation triggers — without a trigger, the review doesn't happen.

📚 Additional resources

• The Twelve-Factor App — Principles for cloud-native apps
• Google SRE Book — Fundamentals of production operations
• Pinecone Pricing Calculator — Managed cost estimation
• Qdrant Cloud Pricing — Tier comparison
• Zilliz Cloud Pricing — Managed Milvus pricing
• Weaviate Cloud Pricing — WCS tiers and free sandbox
• AWS EC2 Pricing — To estimate self-hosted costs
• Vector DB Comparison (Superlinked) — Up-to-date comparison

Reading time: 20-25 minutes Next: 04-rag-feature-comparison.md