Module 6: Decision Matrix for AI Engineers

Capsule 07: Real-World Decision Cases

Capsule description

In the previous capsule you applied the matrix to template scenarios. Now you're going to see how it works in complete cases with characters, changing constraints, and decisions that aren't obvious. The difference: here there's no "right answer from the start". Each case includes moments of uncertainty, incomplete data, and business pressure — exactly as it happens in reality.

The goal is for you to practice the complete decision process: from requirements gathering to the documented justification, passing through the uncomfortable moments where the matrix tells you one thing and your intuition tells you another. By the end of this capsule, you'll have seen the framework survive four scenarios with real constraints and you'll know how to apply it to your own project.

These cases are designed to connect directly with your Decision Questionnaire (capsule 08). Every decision you'll see here could have been guided by the questionnaire you'll build.


Structure of each case

Each case follows this flow:

┌─────────────────────────────────────────┐
│  1. CONTEXT                             │
│     Who, what, when, with what          │
├─────────────────────────────────────────┤
│  2. INITIAL REQUIREMENTS                │
│     What the team thinks it needs        │
├─────────────────────────────────────────┤
│  3. REAL REQUIREMENTS                   │
│     What they discover on digging deeper │
├─────────────────────────────────────────┤
│  4. MATRIX APPLIED                      │
│     Weights, scores, ranking            │
├─────────────────────────────────────────┤
│  5. COMPLICATION                        │
│     Something changes or goes wrong     │
├─────────────────────────────────────────┤
│  6. FINAL DECISION                      │
│     With documented justification       │
├─────────────────────────────────────────┤
│  7. RESULT AT 6 MONTHS                  │
│     What happened afterward?            │
└─────────────────────────────────────────┘

Case A: HealthTech startup — RAG for medical documentation

1. Context

Company: MediSearch, an 8-person startup (5 devs, 1 PM, 1 designer, 1 founder).

Product: An intelligent medical-literature search engine for general practitioners. Doctors ask questions in natural language and receive answers with citations from papers and clinical guidelines.

Current state: A working prototype with OpenAI embeddings + brute-force search in NumPy. It works with 15K documents but the investor wants a demo with 500K+ for the next round.

Timeline: 10 weeks to the investor demo.

2. Initial requirements (what the team says)

from dataclasses import dataclass


@dataclass
class MediSearchRequirements:
    vectors_now: int = 15_000
    vectors_demo: int = 500_000
    vectors_12m: int = 2_000_000
    latency_target_ms: int = 200
    budget_monthly: int = 400
    team_devs: int = 5
    has_devops: bool = False
    timeline_weeks: int = 10
    compliance: str = "none for now"


initial_reqs = MediSearchRequirements()

3. Real requirements (what they discover)

In week 2, the investor's legal team asks: "Is the medical data encrypted in transit and at rest?"

It turns out the papers aren't patient data, but the doctors' queries can indeed contain sensitive information (symptoms of specific patients). This changes everything:

@dataclass
class MediSearchRealRequirements(MediSearchRequirements):
    query_data_sensitive: bool = True
    encryption_at_rest: bool = True
    encryption_in_transit: bool = True
    data_residency: str = "LATAM or US"
    audit_queries: bool = True


real_reqs = MediSearchRealRequirements()

4. Matrix applied

weights = {
    "setup_speed": 4,       # demo in 10 weeks
    "scale": 4,             # 500K for the demo, 2M in 12 months
    "latency": 4,           # <200ms for smooth UX
    "encryption": 5,        # queries with sensitive data
    "data_residency": 3,    # preferably LATAM/US
    "ops_simplicity": 5,    # no DevOps
    "cost_control": 3,      # limited but flexible budget
}

providers = {
    "Pinecone": {
        "setup_speed": 0.9, "scale": 0.9, "latency": 0.9,
        "encryption": 0.9, "data_residency": 0.7,
        "ops_simplicity": 1.0, "cost_control": 0.6,
    },
    "Qdrant_Cloud": {
        "setup_speed": 0.8, "scale": 0.85, "latency": 0.85,
        "encryption": 0.8, "data_residency": 0.8,
        "ops_simplicity": 0.8, "cost_control": 0.8,
    },
    "Weaviate_Cloud": {
        "setup_speed": 0.75, "scale": 0.8, "latency": 0.8,
        "encryption": 0.85, "data_residency": 0.75,
        "ops_simplicity": 0.85, "cost_control": 0.7,
    },
    "ChromaDB_local": {
        "setup_speed": 1.0, "scale": 0.4, "latency": 0.7,
        "encryption": 0.3, "data_residency": 1.0,
        "ops_simplicity": 0.7, "cost_control": 1.0,
    },
}


def score_provider(weights: dict, scores: dict) -> float:
    total = sum(weights[k] * scores.get(k, 0) for k in weights)
    max_total = sum(weights.values())
    return round((total / max_total) * 100, 1)


results = {name: score_provider(weights, scores)
           for name, scores in providers.items()}
# Pinecone: 86.4, Qdrant Cloud: 81.4, Weaviate Cloud: 79.5, ChromaDB: 69.3
ProviderScoreNotes
Pinecone86.4Best at ops + encryption
Qdrant Cloud81.4Good cost/features balance
Weaviate Cloud79.5Solid, hybrid search bonus
ChromaDB local69.3Fails on encryption and scale

5. Complication

Week 4: the CTO discovers that Pinecone Standard doesn't include query logs for the audit trail. The investor's legal team wants to be able to demonstrate that they can audit which queries were made. Options:

  • A: Upgrade to Pinecone Enterprise (out of budget: ~$1500/month)
  • B: Implement audit logging at the application layer (3-5 days of development)
  • C: Switch to Qdrant Cloud, which allows more logging control

6. Final decision

The team chooses Pinecone Standard + audit logging in the application (option B):

decision_doc = {
    "provider": "Pinecone Standard",
    "justification": (
        "Highest score (86.4). The audit gap is resolved with logging middleware "
        "in the app (3 days of dev, doesn't block the demo)."
    ),
    "alternative": "Qdrant Cloud (81.4) — if Pinecone raises prices post-demo",
    "risks": [
        "Vendor lock-in: migration will cost ~2 weeks if we switch",
        "Cost scaling: $400/month today → $800-1200/month at 2M vectors",
        "App-layer audit: more fragile than native audit",
    ],
    "mitigations": [
        "Abstraction with a generic interface for the vector store",
        "Monthly cost review with an alert at $600/month",
        "Integration tests for the audit middleware",
    ],
    "reevaluation": "Post-demo (week 12) or if cost > $600/month",
}

7. Result at 6 months

  • Successful demo, they raised a seed round.
  • Pinecone works well at 800K vectors, cost $280/month.
  • The audit middleware had 1 bug (queries not logged on timeout) that they fixed in sprint 8.
  • They plan to evaluate Qdrant Cloud in Q3 when they hit 2M vectors and the cost rises.

Lesson: The matrix got it right. The audit complication was resolved with engineering, not a provider change. The key was having the alternative documented.


Case B: Regulated FinTech — Transaction embeddings

1. Context

Company: PayGuard, a fintech with 40 employees, 12 in engineering.

Product: A fraud-detection system that uses embeddings of transaction patterns to find anomalies similar to known frauds.

Current state: A legacy system with Elasticsearch + cosine similarity. It works but with 2-3 second latency per query. The ML team wants a native vector DB.

Timeline: 16 weeks (gradual migration, no big bang).

2. Initial requirements

@dataclass
class PayGuardRequirements:
    vectors_now: int = 8_000_000
    vectors_12m: int = 30_000_000
    latency_target_ms: int = 50    # fraud detection is time-critical
    budget_monthly: int = 5_000
    team_devs: int = 12
    has_devops: bool = True         # platform team of 3
    has_sre: bool = True
    timeline_weeks: int = 16
    compliance: str = "PCI-DSS, SOC2"
    data_residency: str = "US mandatory"
    multi_region: bool = True       # DR mandatory

3. Real requirements

The compliance team adds constraints in week 3:

@dataclass
class PayGuardRealRequirements(PayGuardRequirements):
    encryption_at_rest: str = "AES-256 with our own key management"
    network_isolation: bool = True   # VPC peering or private link
    audit_immutable: bool = True     # non-modifiable logs
    pen_test_required: bool = True   # vendor must allow pen testing
    data_deletion_sla: str = "72 hours max"
    vendor_soc2_type2: bool = True   # provider certification

4. Matrix applied

weights = {
    "latency": 5,           # <50ms is critical for fraud
    "scale": 5,             # 30M vectors
    "compliance_pci": 5,    # non-negotiable
    "network_isolation": 5, # non-negotiable
    "multi_region": 4,      # DR mandatory
    "encryption_custom": 4, # our own key management
    "ops_simplicity": 3,    # they have a team
    "cost_control": 3,      # reasonable budget
}

providers = {
    "Qdrant_self_hosted": {
        "latency": 0.9, "scale": 0.85, "compliance_pci": 0.9,
        "network_isolation": 1.0, "multi_region": 0.7,
        "encryption_custom": 0.9, "ops_simplicity": 0.5,
        "cost_control": 0.85,
    },
    "Pinecone_enterprise": {
        "latency": 0.85, "scale": 0.9, "compliance_pci": 0.7,
        "network_isolation": 0.7, "multi_region": 0.85,
        "encryption_custom": 0.5, "ops_simplicity": 1.0,
        "cost_control": 0.5,
    },
    "Milvus_self_hosted": {
        "latency": 0.85, "scale": 0.95, "compliance_pci": 0.85,
        "network_isolation": 1.0, "multi_region": 0.8,
        "encryption_custom": 0.85, "ops_simplicity": 0.4,
        "cost_control": 0.8,
    },
    "Weaviate_self_hosted": {
        "latency": 0.8, "scale": 0.8, "compliance_pci": 0.85,
        "network_isolation": 1.0, "multi_region": 0.7,
        "encryption_custom": 0.8, "ops_simplicity": 0.55,
        "cost_control": 0.8,
    },
}

results = {name: score_provider(weights, scores)
           for name, scores in providers.items()}
ProviderScoreNotes
Qdrant Self-hosted84.4Best compliance + latency
Milvus Self-hosted83.7Best scale, more complex
Weaviate Self-hosted80.3Solid but lower latency
Pinecone Enterprise75.4Fails on critical compliance

5. Complication

Week 6: the platform team runs a PoC with self-hosted Qdrant on Kubernetes. They discover that:

  • Qdrant works excellently on single-node (42ms p95 with 1M test vectors)
  • Multi-region replication requires manual configuration and isn't as mature as they expected
  • The platform team of 3 is already at 90% capacity with other services

The CTO presents two paths:

Path A: Qdrant self-hosted
  ✓ Perfect compliance
  ✓ Excellent latency
  ✗ Multi-region complex → they need to hire 1 more SRE ($12K/month)
  ✗ Timeline extends 4-6 weeks

Path B: Milvus self-hosted
  ✓ More mature multi-region (Milvus distributed)
  ✓ Scale to 30M proven in production by others
  ✗ More complex ops (Pulsar/Kafka dependency)
  ✗ Slightly higher latency (55ms p95 in the PoC)

6. Final decision

The team chooses Qdrant self-hosted + hiring an SRE:

decision_doc = {
    "provider": "Qdrant Self-hosted (Kubernetes)",
    "justification": (
        "Highest score (84.4). The 42ms latency meets the 50ms SLA with "
        "margin. Multi-region is solved with app-level replication + "
        "Qdrant snapshots across regions. The cost of 1 additional SRE ($12K/month) "
        "is justified against the operational overhead of Milvus with Pulsar."
    ),
    "alternative": "Milvus Self-hosted — if Qdrant's multi-region doesn't scale",
    "migration_plan": [
        "Weeks 1-4: Deploy Qdrant in staging, migrate the embeddings pipeline",
        "Weeks 5-8: Migrate 20% of traffic (non-critical transactions)",
        "Weeks 9-12: Migrate 80% of traffic with a fallback to Elasticsearch",
        "Weeks 13-16: 100% migrated, decommission Elasticsearch vector search",
    ],
    "risks": [
        "Multi-region snapshot lag: max 30 min of delay acceptable for DR",
        "Hiring an SRE takes 4-8 weeks: team lead covers the interim",
        "Qdrant version upgrades: test in staging before production",
    ],
    "reevaluation": "Quarterly, or if p95 latency exceeds 45ms in production",
}

7. Result at 6 months

  • Qdrant in production with 12M vectors, 38ms p95.
  • Multi-region with snapshots every 15 minutes (acceptable for DR).
  • SRE hired in week 10, already operating the cluster.
  • Total cost: $2,800/month (infra) + $12K/month (SRE) = $14,800/month.
  • Comparison: Pinecone Enterprise would have cost ~$8K/month but didn't pass compliance.

Lesson: Sometimes the "more expensive in headcount" option is the only viable one when compliance is non-negotiable. The cost of NOT meeting PCI-DSS is existential for a fintech.


Case C: E-commerce — Product recommendations at scale

1. Context

Company: StyleFind, a fashion e-commerce with 200 employees, 35 in tech.

Product: A "Find Similar" recommendation system where users upload photos or describe garments and the system finds similar products in the catalog.

Current state: A recommendation model based on collaborative filtering. They want to add visual similarity with image embeddings (CLIP).

Timeline: 20 weeks (launch for Black Friday).

2. Requirements

  • Vectors: 0 today → 2M at launch → 5M in 12 months
  • Latency: p95 < 100ms (demanding e-commerce UX)
  • Budget: $2,000/month
  • Team: 35 devs with DevOps + 4 ML engineers
  • Timeline: 20 weeks (Black Friday launch)
  • Peak: Black Friday 20x normal traffic (50 → 1,000 qps)
  • Embeddings: CLIP 512D (text + image)

3. Real requirements — the Black Friday factor

The infra team raises a constraint nobody had mentioned:

Normal traffic:     50 queries/second
Black Friday:       1,000 queries/second (20x)
Cyber Monday:       800 queries/second (16x)
Rest of the year:   50-100 queries/second

This changes the weights dramatically. The database needs aggressive auto-scaling or provisioned capacity for peaks:

weights = {
    "scale": 5,             # 5M vectors
    "latency_at_peak": 5,   # <100ms even at 1000 qps
    "auto_scaling": 5,      # 20x peaks non-negotiable
    "multimodal_support": 4, # CLIP embeddings
    "ops_simplicity": 4,    # large team but they don't want to dedicate ML engineers to ops
    "cost_efficiency": 4,   # paying for constant peaks is wasteful
    "setup_speed": 3,       # 20 weeks is reasonable
    "compliance": 1,        # doesn't apply
}

4. Matrix applied

providers = {
    "Pinecone_standard": {
        "scale": 0.9, "latency_at_peak": 0.85, "auto_scaling": 0.9,
        "multimodal_support": 0.8, "ops_simplicity": 1.0,
        "cost_efficiency": 0.5, "setup_speed": 0.9, "compliance": 0.5,
    },
    "Qdrant_cloud": {
        "scale": 0.85, "latency_at_peak": 0.85, "auto_scaling": 0.7,
        "multimodal_support": 0.85, "ops_simplicity": 0.8,
        "cost_efficiency": 0.8, "setup_speed": 0.8, "compliance": 0.5,
    },
    "Weaviate_cloud": {
        "scale": 0.8, "latency_at_peak": 0.8, "auto_scaling": 0.75,
        "multimodal_support": 0.9, "ops_simplicity": 0.85,
        "cost_efficiency": 0.7, "setup_speed": 0.75, "compliance": 0.5,
    },
    "Milvus_on_k8s": {
        "scale": 0.95, "latency_at_peak": 0.9, "auto_scaling": 0.85,
        "multimodal_support": 0.8, "ops_simplicity": 0.4,
        "cost_efficiency": 0.85, "setup_speed": 0.5, "compliance": 0.5,
    },
}

results = {name: score_provider(weights, scores)
           for name, scores in providers.items()}
ProviderScoreNotes
Pinecone Standard82.7Auto-scaling + ops simplicity
Qdrant Cloud79.7Good overall balance
Weaviate Cloud78.4Best native multimodal
Milvus on K8s76.5Best scale + cost, worst ops

5. Complication — The cost of peaks

The finance team runs the numbers:

ProviderNormal monthBlack Friday monthAnnual estimate
Pinecone$450$3,200$8,150
Qdrant Cloud$280$1,800$4,880
Milvus K8s$600 (incl. ops)$900$7,500

Pinecone is the most expensive annualized, but Milvus requires K8s expertise the ML team doesn't have. The team debates whether to hire infrastructure or pay the premium.

6. Final decision

Hybrid architecture: Qdrant Cloud + pre-warming for peaks

Provider: Qdrant Cloud — Score 79.7, close to the top. The deciding factor is annual TCO: $4,880 vs $8,150 for Pinecone (-40%).

Peak architecture:

  • Normal: Qdrant Cloud standard tier, 2M vectors
  • Pre-peak: Manual scale-up 48h before Black Friday / Cyber Monday
  • Peak: Provisioned capacity 3x to absorb 1000 qps
  • Post-peak: Scale down 72h after the event

Alternative: Pinecone Standard — if pre-warming turns out to be operationally expensive.

Risks: Manual pre-warming requires an events calendar; unplanned peaks cause latency until scaling.

Re-evaluation: Post-Black Friday: did pre-warming work? How many unplanned peaks were there?

7. Result at 6 months

  • Black Friday: pre-warming worked. 85ms p95 at 950 qps. A spike to 1,100 qps caused 120ms p95 for 3 minutes.
  • Actual annual cost: $5,200 (slightly over estimate due to growing storage).
  • The ML team didn't have to learn K8s, they stayed focused on improving CLIP embeddings.
  • They decided to keep Qdrant Cloud and improve the pre-warming process with automatic alerts.

Lesson: The highest-scoring option isn't always the best when annual TCO and the team's skills weigh in. An "80% automatic + 20% manual" solution can be more pragmatic than a "100% automatic and 2x more expensive" one.


Case D: AI Consultancy — Multiple clients, multiple needs

1. Context

Company: DataPulse, an AI consultancy with 15 people.

Special situation: They don't build a single product. They have 6-8 simultaneous clients, each with different needs. They need a vector DB strategy that works across projects.

2. The consultancy's unique problem

ClientVectorsComplianceBudgetTimeline
A50KNoLow4 weeks
B3MYesHigh16 weeks
C200KNoMedium8 weeks
D800KNoMedium12 weeks

They can't use a different provider for each client — the team doesn't have the bandwidth to master 4 different technologies.

3. Adapted matrix — a "versatility" criterion

weights_consultancy = {
    "versatility": 5,       # must work for diverse clients
    "learning_curve": 5,    # the team must be productive fast on any project
    "managed_and_self": 4,  # able to offer both models
    "scale_range": 4,       # 50K to 3M without changing tech
    "cost_predictability": 4,  # able to quote clients with confidence
    "documentation": 4,     # the team rotates between projects
    "community": 3,         # support when the team is busy
}

providers = {
    "Qdrant": {
        "versatility": 0.9, "learning_curve": 0.8,
        "managed_and_self": 1.0, "scale_range": 0.85,
        "cost_predictability": 0.8, "documentation": 0.75,
        "community": 0.8,
    },
    "Weaviate": {
        "versatility": 0.85, "learning_curve": 0.75,
        "managed_and_self": 1.0, "scale_range": 0.8,
        "cost_predictability": 0.75, "documentation": 0.8,
        "community": 0.75,
    },
    "Pinecone": {
        "versatility": 0.7, "learning_curve": 0.9,
        "managed_and_self": 0.3, "scale_range": 0.9,
        "cost_predictability": 0.6, "documentation": 0.9,
        "community": 0.7,
    },
    "ChromaDB": {
        "versatility": 0.6, "learning_curve": 1.0,
        "managed_and_self": 0.4, "scale_range": 0.5,
        "cost_predictability": 0.9, "documentation": 0.8,
        "community": 0.7,
    },
}

results = {name: score_provider(weights_consultancy, scores)
           for name, scores in providers.items()}
ProviderScoreNotes
Qdrant84.5Managed + self-hosted, wide scale range
Weaviate81.6Very close, hybrid search bonus
Pinecone72.1Managed only = limited for clients with compliance
ChromaDB70.7Doesn't scale for large clients

4. Complication — Client B needs self-hosted

Client B (fintech, 3M vectors, compliance) signs a contract. They need self-hosted and the team has never operated Qdrant in production. Three options:

  • A: Learn self-hosted Qdrant (2-3 weeks of ramp-up) — consistent but delays
  • B: Use self-hosted Weaviate (prior experience) — fast but breaks standardization
  • C: Subcontract DevOps for the Qdrant deployment ($5K one-time) — keeps the standard without delay

5. Final decision

Option C: Subcontract DevOps for the first self-hosted Qdrant deployment, with knowledge transfer to the internal team.

Strategy: Standardize on Qdrant (cloud + self-hosted).

ClientDeployment
Client A (50K, no compliance)Qdrant Cloud free tier
Client B (3M, compliance)Qdrant self-hosted in the client's AWS
Client C (200K, no compliance)Qdrant Cloud starter
Client D (800K, no compliance)Qdrant Cloud standard

Investment: DevOps contractor $5K one-time + 2 internal 4h workshops + a deployment runbook.

Re-evaluation: Every 6 months, or when a client requires features Qdrant doesn't support.

6. Result at 6 months

  • 4 clients running on Qdrant (2 cloud, 1 self-hosted, 1 cloud migrated from ChromaDB).
  • The team became a Qdrant expert: deployment, tuning, monitoring.
  • Standardization reduced new-dev onboarding from 2 weeks to 3 days.
  • A new client asked for hybrid search → they briefly evaluated Weaviate but solved it with Qdrant sparse vectors.

Lesson: For consultancies, the dominant variable is versatility + the team's learning curve. Technological consistency across projects saves more money than optimizing each project individually.


Common patterns across the four cases

  1. Compliance trumps all: When compliance is a requirement, eliminate managed-only options immediately (Case B). Don't waste time evaluating what you can't use.

  2. TCO beats sticker price: In all 4 cases, the "cheapest" provider by direct price was never the cheapest in TCO. Always add operation hours and risk.

  3. Complications test the decision: Every decision will face a complication. The framework doesn't prevent it but gives you tools to respond rationally (re-score, don't throw everything away).

  4. A documented alternative saves you: Having a documented alternative with a score gives the team confidence: "if X fails, we have Y ready with a gap of only N points".


Troubleshooting

"My real case doesn't look like any of these four"

It doesn't need to match exactly. Extract the weights from the closest case and adjust the 2-3 criteria that are different. The framework's structure is the same: define constraints → score → choose → document → re-evaluate.

"I built the matrix but my boss doesn't accept the result"

The problem isn't the matrix, it's that your boss has implicit criteria that aren't in the matrix. Ask them: "Which criterion would you add and with what weight?" If they say "brand trust", add it with weight 3 and recalculate. If the result changes, that was the real constraint.

"I don't have a benchmark to score providers"

Use three sources: (1) the provider's official documentation, (2) public benchmarks (ANN Benchmarks, technical blogs), (3) a 2-3 day PoC with representative data. You don't need perfection: you need reasonable scores you can defend and revisit later.

"The complication that came up invalidates the whole decision"

It rarely invalidates everything. Re-score only the affected criteria and recalculate. If the ranking doesn't change, the decision stands. If it changes, you have the documented alternative to activate without panic.

"The team wants to switch providers every 3 months"

Set an explicit switching cost in the matrix (migration_cost as a weighted criterion). When someone proposes switching, the migration cost enters the calculation and typically anchors the current decision unless the benefit is very clear.


Exercises

Exercise 1: Analyze a case and challenge the decision

Take Case A (MediSearch) and argue WHY the decision should have been different. Use the matrix but with weights that reflect a different priority.

Solution

If you prioritize long-term cost over speed to demo:

weights_alternative = {
    "setup_speed": 2,       # low priority (no longer urgent post-demo)
    "scale": 5,             # 2M vectors is the real goal
    "latency": 4,
    "encryption": 5,
    "data_residency": 4,    # rises: LATAM matters for the target market
    "ops_simplicity": 3,    # drops: the team can learn
    "cost_control": 5,      # rises: startup with limited runway
}

# With these weights the gap closes: Pinecone 82.9 vs Qdrant 81.6 (technical tie).
# In that zone the long-term cost tie-breaker favors Qdrant Cloud:
# - Better cost at 2M vectors ($180/month vs $800+ Pinecone)
# - Open source = lower vendor lock-in
# - Data residency in more regions

# Counter-argument: in week 10 with the investor demo,
# arriving 2 weeks late to learn Qdrant could cost
# the investment round. Pinecone's cost ($400/month)
# is insignificant compared to losing $500K in funding.

There's no "wrong" answer — there are different priorities.

Exercise 2: Build your own case

Write a complete case following the 7-step structure. Use your current company/project or invent a realistic one.

Guide structure
my_case = {
    "context": {
        "company": "___",
        "product": "___",
        "current_state": "___",
        "timeline": "___",
    },
    "initial_requirements": {
        "vectors": "___",
        "latency": "___",
        "budget": "___",
        "team": "___",
    },
    "real_requirements": "What did you discover that you didn't know at the start?",
    "matrix": {
        "weights": {},
        "providers": {},
        "scores": {},
    },
    "complication": "What happened that you didn't expect?",
    "decision": {
        "primary": "___",
        "justification": "___",
        "alternative": "___",
        "risks": [],
    },
    "result_6m": "How do you project it will turn out?",
}

Quality criterion: your case must include at least one complication that forces a partial re-evaluation of the decision.

Exercise 3: Cross-scoring

Take the 4 providers from Case C (StyleFind) and score them using the weights from Case B (PayGuard). Does the ranking change? Why?

Solution
# PayGuard weights applied to StyleFind providers
weights_payguard = {
    "latency": 5, "scale": 5, "compliance_pci": 5,
    "network_isolation": 5, "multi_region": 4,
    "encryption_custom": 4, "ops_simplicity": 3, "cost_control": 3,
}

# StyleFind's providers have NO scores for compliance_pci,
# network_isolation, etc. This demonstrates that:
# 1. The managed providers (Pinecone, Qdrant Cloud) get
#    very low scores on compliance and network isolation
# 2. The ranking flips completely
# 3. Only the self-hosted options survive

# Lesson: the weights change EVERYTHING. There's no absolute
# "best provider". There's the best provider FOR YOUR CONTEXT.

Exercise 4: Complications map

For each case (A-D), identify an additional complication that could have occurred and how it would have affected the decision.

Example solution
  • Case A: OpenAI raises embedding prices 3x → they need a local model → embeddings of a different dimension → does the vector DB support resizing without re-indexing?
  • Case B: A PCI audit reveals that Qdrant has no formal certification → document compensating controls → 3 extra weeks of compliance.
  • Case C: Black Friday traffic is 30x instead of 20x → insufficient pre-warming → they consider a hot-spare on another provider.
  • Case D: A large client wants to use Pinecone (already has an enterprise contract) → keep Qdrant + Pinecone → breaks standardization.

Exercise 5: Decision Questionnaire connection

Project connection: Take the 4 cases and extract the 5 most important questions an automated questionnaire should ask to classify any new case into one of the 4 scenarios.

Solution

The 5 critical questions:

  1. "How many vectors in 12 months?" → <100K = Case A, 1M-10M = Case C, 10M+ = Case B
  2. "Compliance (PCI, HIPAA, SOC2)?" → No = managed viable, PCI/HIPAA = self-hosted mandatory
  3. "Do you have DevOps/SRE?" → No = ops_simplicity weight 5, Yes = can drop to 2-3
  4. "Timeline?" → <4 weeks = setup_speed weight 5, >12 weeks = can drop
  5. "Multiple clients/projects?" → Yes = versatility weight 5 (Case D pattern)

Exercise 6: Decision post-mortem

Choose one of the 4 cases and write a 12-month post-mortem: What changed? Does the decision still hold? What would you do differently?

Example solution — Case A post-mortem

Context: We chose Pinecone Standard (score 86.4), alternative: Qdrant Cloud (81.4).

What happened: Months 1-6 perfect. Successful demo, round raised. In month 9, 1.8M vectors, cost $520/month. In month 11, 2.3M vectors, cost $720/month. The audit middleware failed silently for 3 days in month 8.

Did it hold? Partially. Pinecone was correct for the first 9 months. In month 10, the cost crossed the trigger ($600/month) and we activated the Qdrant Cloud evaluation.

What would we do differently? (1) Audit logging with more tests from day 1. (2) A more aggressive cost trigger ($500/month). (3) A migration script kept up to date for the alternative.


Summary

  • Every real case has complications the matrix alone doesn't predict, but the framework gives you structure to respond without panic.
  • The documented alternative is as important as the primary choice: it gives you a plan B with data, not intuition.
  • Non-negotiable compliance eliminates options before scoring — don't waste time evaluating what's unfeasible.
  • Real TCO (service + ops + risk) always beats "sticker price" as a decision criterion.
  • For consultancies and multi-project teams, technological consistency (a single stack) tends to win over individual optimization.
  • Re-evaluation triggers turn a static decision into a living process that adapts.
  • Your Decision Questionnaire (capsule 08) will automate the classification you did manually in these 4 cases.
  • The best decision isn't the one that's always right, but the one you can justify, defend, and change with data when the context evolves.

Additional resources


Estimated time: 30-35 minutes
Next: 08-project-decision-questionnaire.md