Capsule 06: Recommendations by Scenario

Capsule description

So far you have criteria, weights, and a scoring matrix. But a matrix without context is just a spreadsheet. In this capsule you're going to apply the complete framework to four realistic scenarios that cover the typical spectrum of an AI engineer: from the solo developer who wants to validate an idea, to the company with strict regulations and large teams.

The goal isn't to give you "the right answer" (there isn't one), but to show you how the same criteria produce different recommendations when the constraints change. In the end, you'll have reusable templates you can adapt to any scenario you face in your career.

Each recommendation includes numerical justification, explicit trade-offs, and re-evaluation triggers. So when someone asks you "why did you choose X?", you can answer with data, not opinions.

Why scenarios matter more than benchmarks

A benchmark tells you that Qdrant is 15% faster than Weaviate on a synthetic 1M-vector dataset. That's useful. But it doesn't tell you whether Qdrant is better for you. Your team, your budget, your deadlines, and your regulations change the answer completely.

Scenarios capture what benchmarks can't:

┌─────────────────────────────────────────────────┐
│              BENCHMARK                          │
│  "Qdrant: 45ms p95 @ 1M vectors"              │
│                                                 │
│  ✗ Doesn't say if your team can operate it     │
│  ✗ Doesn't say if it fits your budget          │
│  ✗ Doesn't say if it meets compliance          │
│  ✗ Doesn't say if it scales with your growth   │
└─────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────┐
│              SCENARIO                           │
│  "Startup of 3 devs, 200K vectors,             │
│   $300/month, no DevOps, MVP in 8 weeks"       │
│                                                 │
│  ✓ Filters out unfeasible options              │
│  ✓ Prioritizes real criteria                   │
│  ✓ Produces an actionable recommendation       │
│  ✓ Defines when to re-evaluate                 │
└─────────────────────────────────────────────────┘

Recommendation-by-scenario framework

Before analyzing each scenario, here's the process you'll follow:

from dataclasses import dataclass, field
from typing import Optional


@dataclass
class Scenario:
    name: str
    team_size: int
    has_devops: bool
    vector_count_now: int
    vector_count_12m: int
    budget_monthly_usd: int
    latency_p95_ms: int
    compliance_required: bool
    timeline_weeks: int
    description: str = ""


@dataclass
class Recommendation:
    scenario: Scenario
    primary: str
    alternative: str
    scores: dict
    decisive_criteria: list
    risks: list
    reevaluation_trigger: str


def recommend_for_scenario(
    scenario: Scenario,
    weights: dict,
    providers: dict[str, dict],
) -> Recommendation:
    """
    Apply the scoring matrix to a specific scenario
    and generate a recommendation with justification.
    """
    scores = {}
    for provider_name, provider_scores in providers.items():
        total = 0
        max_total = 0
        for criterion, weight in weights.items():
            total += weight * provider_scores.get(criterion, 0)
            max_total += weight * 1.0
        scores[provider_name] = round((total / max_total) * 100, 1)

    ranked = sorted(scores.items(), key=lambda x: x[1], reverse=True)

    top_criteria = sorted(weights.items(), key=lambda x: x[1], reverse=True)
    decisive = [c[0] for c in top_criteria[:3]]

    return Recommendation(
        scenario=scenario,
        primary=ranked[0][0],
        alternative=ranked[1][0],
        scores=scores,
        decisive_criteria=decisive,
        risks=_identify_risks(scenario, ranked[0][0], providers[ranked[0][0]]),
        reevaluation_trigger=_suggest_trigger(scenario),
    )


def _identify_risks(scenario, provider, scores):
    risks = []
    for criterion, score in scores.items():
        if score < 0.5:
            risks.append(f"{criterion}: low score ({score})")
    if scenario.vector_count_12m > scenario.vector_count_now * 5:
        risks.append("Growth >5x in 12 months")
    return risks


def _suggest_trigger(scenario):
    if scenario.timeline_weeks <= 8:
        return "Re-evaluate on MVP completion"
    elif scenario.vector_count_12m > 1_000_000:
        return "Re-evaluate on reaching 500K vectors"
    return "Re-evaluate quarterly"

Scenario 1: Solo Developer — Idea validation

Context

solo_dev = Scenario(
    name="Solo Developer - Idea validation",
    team_size=1,
    has_devops=False,
    vector_count_now=10_000,
    vector_count_12m=100_000,
    budget_monthly_usd=50,
    latency_p95_ms=500,
    compliance_required=False,
    timeline_weeks=4,
    description="Indie developer validating whether RAG adds value to their product",
)

Weights adjusted to the scenario

When you're a solo developer, setup speed and cost dominate everything else:

weights_solo = {
    "setup_speed": 5,       # you need something running today
    "cost_control": 5,      # personal budget
    "ops_simplicity": 5,    # there's no support team
    "documentation": 4,     # you'll learn on your own from docs
    "scale": 2,             # not a priority yet
    "latency": 2,           # 500ms is acceptable for an MVP
    "compliance": 1,        # doesn't apply
}

Provider scoring

providers_solo = {
    "ChromaDB_local": {
        "setup_speed": 1.0,
        "cost_control": 1.0,
        "ops_simplicity": 0.9,
        "documentation": 0.8,
        "scale": 0.4,
        "latency": 0.7,
        "compliance": 0.3,
    },
    "Pinecone_starter": {
        "setup_speed": 0.9,
        "cost_control": 0.7,
        "ops_simplicity": 1.0,
        "documentation": 0.9,
        "scale": 0.8,
        "latency": 0.9,
        "compliance": 0.5,
    },
    "Qdrant_cloud_free": {
        "setup_speed": 0.8,
        "cost_control": 0.9,
        "ops_simplicity": 0.8,
        "documentation": 0.7,
        "scale": 0.7,
        "latency": 0.85,
        "compliance": 0.4,
    },
}

result = recommend_for_scenario(solo_dev, weights_solo, providers_solo)

Result

Recommendation

The scores are a technical tie (Pinecone 85.4 vs ChromaDB 84.2, just 1.2 points apart). At that distance, the tie-breaker decides: for a solo developer, zero cost and instant setup tip the balance toward ChromaDB.

Primary: ChromaDB local — setup in 5 minutes, zero infra cost, ideal for fast iteration.

Alternative: Pinecone Starter — if you'd rather not manage anything and you accept the free tier with limits.

Explicit trade-offs

Re-evaluation trigger

Re-evaluate when: (a) you exceed 50K active vectors, (b) you need concurrent queries from multiple users, or (c) you validate product-market fit and decide to scale.

Scenario 2: Small startup — Product under construction

Context

startup = Scenario(
    name="Small startup - Product under construction",
    team_size=4,
    has_devops=False,
    vector_count_now=200_000,
    vector_count_12m=1_500_000,
    budget_monthly_usd=500,
    latency_p95_ms=300,
    compliance_required=False,
    timeline_weeks=12,
    description="Team of 4 devs building a SaaS with RAG as a core feature",
)

Adjusted weights

The balance changes: now scale matters because you expect real growth, and latency starts to be a differentiator:

weights_startup = {
    "scale": 4,             # expected 7x growth
    "latency": 4,           # UX differentiator
    "ops_simplicity": 5,    # no dedicated DevOps
    "cost_control": 4,      # limited runway
    "setup_speed": 3,       # you have 12 weeks
    "documentation": 3,     # the team can invest in learning
    "compliance": 1,        # doesn't apply yet
}

Scoring and result

providers_startup = {
    "Pinecone_standard": {
        "scale": 0.9, "latency": 0.9, "ops_simplicity": 1.0,
        "cost_control": 0.6, "setup_speed": 0.9,
        "documentation": 0.9, "compliance": 0.5,
    },
    "Qdrant_cloud": {
        "scale": 0.85, "latency": 0.85, "ops_simplicity": 0.8,
        "cost_control": 0.8, "setup_speed": 0.7,
        "documentation": 0.7, "compliance": 0.5,
    },
    "Weaviate_cloud": {
        "scale": 0.85, "latency": 0.8, "ops_simplicity": 0.85,
        "cost_control": 0.7, "setup_speed": 0.75,
        "documentation": 0.8, "compliance": 0.5,
    },
}

Recommendation

Primary: Pinecone Standard — maximum ops simplicity, proven scale, lets the team focus on the product.

Alternative: Qdrant Cloud — better cost/control ratio, more tuning options if the team invests time.

Trade-offs

Re-evaluation trigger

Re-evaluate when: (a) the managed cost exceeds 15% of the total cloud budget, (b) you need features the managed option doesn't support (custom indexes, on-prem), or (c) you raise a round and can hire DevOps.

Scenario 3: Mid-size team — Growing SaaS

Context

midsize = Scenario(
    name="Mid-size team - Growing SaaS",
    team_size=15,
    has_devops=True,
    vector_count_now=5_000_000,
    vector_count_12m=20_000_000,
    budget_monthly_usd=3000,
    latency_p95_ms=150,
    compliance_required=False,
    timeline_weeks=16,
    description="Team with an SRE, multi-tenant SaaS, 99.9% SLA",
)

Adjusted weights

With a DevOps team, operational simplicity drops in priority. Scale, latency, and SLA dominate:

weights_midsize = {
    "scale": 5,             # 20M vectors in 12 months
    "latency": 5,           # p95 < 150ms is aggressive
    "multi_tenancy": 5,     # multi-tenant SaaS
    "sla_guarantee": 4,     # 99.9% uptime
    "ops_simplicity": 3,    # they have an SRE
    "cost_control": 3,      # reasonable budget
    "compliance": 2,        # not strict but it matters
}

Scoring and result

providers_midsize = {
    "Pinecone_enterprise": {
        "scale": 0.9, "latency": 0.85, "multi_tenancy": 0.9,
        "sla_guarantee": 0.95, "ops_simplicity": 1.0,
        "cost_control": 0.5, "compliance": 0.7,
    },
    "Qdrant_self_hosted": {
        "scale": 0.85, "latency": 0.9, "multi_tenancy": 0.7,
        "sla_guarantee": 0.7, "ops_simplicity": 0.5,
        "cost_control": 0.9, "compliance": 0.8,
    },
    "Weaviate_enterprise": {
        "scale": 0.85, "latency": 0.8, "multi_tenancy": 0.85,
        "sla_guarantee": 0.85, "ops_simplicity": 0.8,
        "cost_control": 0.6, "compliance": 0.75,
    },
    "Milvus_self_hosted": {
        "scale": 0.95, "latency": 0.85, "multi_tenancy": 0.6,
        "sla_guarantee": 0.6, "ops_simplicity": 0.4,
        "cost_control": 0.85, "compliance": 0.7,
    },
}

Recommendation

Primary: Pinecone Enterprise — guaranteed SLA, native multi-tenancy, frees the SRE team for other problems.

Alternative: Weaviate Enterprise — good balance between managed and control, native hybrid search that can differentiate the product.

12-month cost analysis

Self-hosted looks cheaper on infra, but the operation hours make it more expensive in total TCO.

Re-evaluation trigger

Re-evaluate when: (a) the managed cost exceeds $5K/month, (b) you need custom sharding by region, or (c) your SLA rises to 99.99%.

Scenario 4: Enterprise — Strict compliance and control

Context

enterprise = Scenario(
    name="Enterprise - Strict compliance",
    team_size=50,
    has_devops=True,
    vector_count_now=50_000_000,
    vector_count_12m=200_000_000,
    budget_monthly_usd=15000,
    latency_p95_ms=100,
    compliance_required=True,
    timeline_weeks=24,
    description="Regulated company, sensitive data, self-hosted mandatory, external audits",
)

Adjusted weights

Compliance and control are non-negotiable. Everything else is optimized around those constraints:

weights_enterprise = {
    "compliance": 5,        # non-negotiable
    "data_residency": 5,    # data in a specific jurisdiction
    "scale": 5,             # 200M vectors
    "latency": 4,           # p95 < 100ms
    "audit_trail": 4,       # external audits
    "ops_simplicity": 2,    # dedicated team available
    "cost_control": 2,      # ample budget
}

Scoring and result

providers_enterprise = {
    "Qdrant_self_hosted": {
        "compliance": 0.9, "data_residency": 1.0, "scale": 0.85,
        "latency": 0.9, "audit_trail": 0.7, "ops_simplicity": 0.5,
        "cost_control": 0.85,
    },
    "Weaviate_self_hosted": {
        "compliance": 0.85, "data_residency": 1.0, "scale": 0.8,
        "latency": 0.8, "audit_trail": 0.7, "ops_simplicity": 0.55,
        "cost_control": 0.8,
    },
    "Milvus_self_hosted": {
        "compliance": 0.8, "data_residency": 1.0, "scale": 0.95,
        "latency": 0.85, "audit_trail": 0.6, "ops_simplicity": 0.4,
        "cost_control": 0.8,
    },
    "Pinecone_enterprise": {
        "compliance": 0.6, "data_residency": 0.4, "scale": 0.9,
        "latency": 0.85, "audit_trail": 0.8, "ops_simplicity": 1.0,
        "cost_control": 0.5,
    },
}

Recommendation

Primary: Qdrant Self-hosted — best compliance + performance balance, modern API, active community for support.

Alternative: Weaviate Self-hosted — native hybrid search is an advantage if the use case needs it, good enterprise deployment documentation.

Why Pinecone does NOT work here

Despite being excellent at ops and SLA, Pinecone fails the non-negotiable criteria of this scenario:

Re-evaluation trigger

Re-evaluate when: (a) Pinecone or managed providers offer on-prem/dedicated deployment, (b) data residency regulations change, or (c) scale exceeds the current cluster's capabilities.

Scenario comparison table

Recommendation by dominant variable

Sometimes you don't fit a template scenario. In that case, look for your dominant variable:

By budget

By team size

By compliance

Recommendation anti-patterns

Mistakes you see repeatedly in real teams:

Anti-pattern 1: "We chose by benchmark"

❌ "Milvus is the fastest on ANN Benchmarks, let's go with Milvus"

✓ "Milvus has better raw throughput, but our team of 3 can't
   operate a distributed cluster. Pinecone gives us 85% of the performance
   with 0% of the ops overhead."

Anti-pattern 2: "We chose for the distant future"

❌ "Someday we'll have 100M vectors, so we need Milvus now"

✓ "We have 50K vectors today, we'll reach 500K in 12 months. ChromaDB
   covers that. We'll migrate when we hit 1M and have the budget
   for managed."

Anti-pattern 3: "Open source is free"

❌ "Qdrant is open source = $0"

✓ "Self-hosted Qdrant costs $200/month in infra + 15hrs/month of operation.
   At $50/hr = $950/month total. Qdrant Cloud costs $180/month with 2hrs/month
   of ops = $280/month total."

Anti-pattern 4: "The CTO already decided"

❌ "The CTO wants Pinecone because they saw it at a conference"

✓ "We respect the preference, but here's the matrix: Pinecone score
   72 vs Qdrant 84 for our scenario. The difference is in cost
   and compliance. Shall we review the weights together?"

Troubleshooting

"My recommendations change every week"

This happens when the weights aren't anchored to real constraints. Freeze criteria and weights per quarterly cycle. If a stakeholder wants to change a weight, they must justify which constraint changed. Put the weights in a shared document with a last-reviewed date and a next scheduled review.

"A provider wins by a very small margin (less than 3 points)"

Don't choose by tenths. When two options are less than 5 points apart, apply the simplicity tie-breaker: choose the option that requires the fewest operational changes for your current team. If they're still tied, choose the one with the lowest exit cost.

"I can't justify the alternative to the team"

Explicitly include the criterion where the primary option is weaker. For example: "Pinecone is our primary recommendation (score 84), but if the managed cost exceeds $X/month, self-hosted Qdrant (score 81) saves us 40% with a trade-off of 15hrs/month of ops."

"The scenario doesn't fit any of the four"

Blend the weights of the closest scenarios. A team of 8 devs with partial compliance takes the startup's weights but raises compliance to 4. You don't need an exact scenario: you need weights that reflect your real constraints.

"The business changed: we went from startup to enterprise"

Don't redo everything. Update only the weights that changed (compliance rises, cost_control drops), recalculate scores, and document the delta. If the ranking changes, plan the migration with a realistic timeline.

Exercises

Exercise 1: Customize your scenario

Create a Scenario for your current situation (real or projected). Define the 7 mandatory fields and calculate scores for at least 3 providers.

my_scenario = Scenario(
    name="EdTech startup - AI tutoring platform",
    team_size=3,
    has_devops=False,
    vector_count_now=80_000,
    vector_count_12m=400_000,
    budget_monthly_usd=200,
    latency_p95_ms=400,
    compliance_required=False,
    timeline_weeks=10,
)

weights = {
    "setup_speed": 4,
    "cost_control": 5,
    "ops_simplicity": 5,
    "scale": 3,
    "latency": 3,
    "documentation": 4,
    "compliance": 1,
}

providers = {
    "ChromaDB_local": {
        "setup_speed": 1.0, "cost_control": 1.0, "ops_simplicity": 0.9,
        "scale": 0.5, "latency": 0.7, "documentation": 0.8, "compliance": 0.3,
    },
    "Qdrant_cloud": {
        "setup_speed": 0.8, "cost_control": 0.8, "ops_simplicity": 0.8,
        "scale": 0.8, "latency": 0.85, "documentation": 0.7, "compliance": 0.5,
    },
    "Pinecone_starter": {
        "setup_speed": 0.9, "cost_control": 0.6, "ops_simplicity": 1.0,
        "scale": 0.9, "latency": 0.9, "documentation": 0.9, "compliance": 0.5,
    },
}

result = recommend_for_scenario(my_scenario, weights, providers)
# Pinecone leads by score (84.4) over ChromaDB (82.4), but with that
# tiny difference the cost + setup tie-breaker favors ChromaDB
# for a small team; Qdrant Cloud (77.8) grows well if you reach 400K in 12 months.

Exercise 2: Change the weights and observe the effect

Take Scenario 2 (Startup) and make three weight variations. Record how the ranking changes.

# Variation A: prioritize cost above all
weights_cost_first = {
    "scale": 3, "latency": 3, "ops_simplicity": 4,
    "cost_control": 5, "setup_speed": 3,
    "documentation": 2, "compliance": 1,
}
# Result: Pinecone still #1 (82.9), but Qdrant (77.6) narrows the gap on cost

# Variation B: prioritize latency and SLA
weights_perf_first = {
    "scale": 4, "latency": 5, "ops_simplicity": 3,
    "cost_control": 2, "setup_speed": 2,
    "documentation": 2, "compliance": 1,
}
# Result: Pinecone stays #1 (86.3) and widens its lead on its latency score

# Variation C: prioritize flexibility and control
weights_control_first = {
    "scale": 4, "latency": 3, "ops_simplicity": 2,
    "cost_control": 4, "setup_speed": 2,
    "documentation": 3, "compliance": 3,
}
# Result: Pinecone still #1 (79.5); Qdrant (75.0) closes in on control/cost

# Conclusion: with this data Pinecone wins all three variations, but the weights
# change the distance to #2 (and in tighter scenarios they would move the winner).
# Document WHY each weight has its value.

Exercise 3: Spot the anti-pattern

Read these three justifications and point out which anti-pattern each one commits:

1. "We chose Milvus because it supports 1 billion vectors and we have 50K."
2. "Weaviate is open source so it's free."
3. "Our investor told us to use Pinecone."

Exercise 4: Design a re-evaluation trigger

For each scenario (1-4), define three concrete metrics that would trigger a re-evaluation of the decision.

Exercise 5: Matrix for your Decision Questionnaire

Project connection: Take three of the questions from your Decision Questionnaire (capsule 08) and assign how each possible answer modifies the matrix weights.

def adjust_weights_from_questionnaire(base_weights: dict, answers: dict) -> dict:
    """Adjust weights based on questionnaire answers."""
    adjusted = base_weights.copy()

    # Question: "How many vectors will you handle in 12 months?"
    if answers.get("vectors_12m", 0) > 1_000_000:
        adjusted["scale"] = max(adjusted.get("scale", 3), 5)

    # Question: "Do you have a DevOps team?"
    if not answers.get("has_devops", False):
        adjusted["ops_simplicity"] = max(adjusted.get("ops_simplicity", 3), 5)

    # Question: "Do you need compliance (HIPAA, SOC2, etc.)?"
    if answers.get("compliance_required", False):
        adjusted["compliance"] = 5
        adjusted["data_residency"] = 5

    return adjusted


base = {
    "scale": 3, "latency": 3, "ops_simplicity": 3,
    "cost_control": 3, "compliance": 1,
}
answers = {
    "vectors_12m": 2_000_000,
    "has_devops": False,
    "compliance_required": True,
}
result = adjust_weights_from_questionnaire(base, answers)
# {'scale': 5, 'latency': 3, 'ops_simplicity': 5,
#  'cost_control': 3, 'compliance': 5, 'data_residency': 5}

Summary

• Benchmarks measure isolated performance; scenarios capture real constraints like team, budget, and compliance.
• For a solo developer: prioritize setup speed and zero cost (ChromaDB local).
• For startups: prioritize ops simplicity and predictable cost (managed cloud).
• For mid-size with an SRE: optimize SLA and multi-tenancy (enterprise managed).
• For a regulated enterprise: compliance and data residency are non-negotiable (self-hosted).
• Always document trade-offs and re-evaluation triggers: today's decision is not permanent.
• The most common anti-patterns are choosing by benchmark, for the distant future, by sticker price, or by authority.
• Your Decision Questionnaire (capsule 08) will automate this process of weights → scoring → recommendation.

Additional resources

• Decision Matrix Method — Wikipedia
• Pinecone Pricing — Managed cost reference
• Qdrant Cloud Pricing — Open source vs cloud comparison
• Weaviate Pricing — Tiers and features by level
• Milvus Documentation — Enterprise self-hosted deployment
• Total Cost of Ownership for Cloud — AWS TCO framework
• Martin Fowler — Two Hard Things — Naming and cache invalidation apply to decisions
• Google SRE Book — Decision Making — How Google makes infrastructure decisions

Estimated time: 25-30 minutes
Next: 07-real-world-decision-cases.md