6. Final Decision Matrix — Document Your Decision

Description

In this capsule you'll integrate the decision matrices from Module 1 (strategies) and Module 7 (platforms) into a final applied decision: the decision matrix v_final. It's not a theoretical exercise — it's the documentation of WHY your AI system is deployed where it is, with what strategy, and what trade-offs you accepted. This document is the justification you present in an interview, send to your tech lead, or consult when you need to re-evaluate.

Context: In Module 1, you built a decision framework. In Module 7, you extended it with platform criteria. Now you apply it. The difference between v2 (M7) and v_final (M8) is that v_final includes the real deployment experience: what worked, what didn't, and what you'd change.

The Evolution of the Decision Matrix

From framework to applied decision

Module 1 — v1: Theoretical framework
├── 4 strategies evaluated (Local, Serverless, Managed, Self-hosted)
├── 5 base dimensions (Cost, Complexity, Scalability, Control, Time-to-deploy)
├── Decision matrix with hypothetical weights
└── Result: "For my case, the recommended strategy is [X]"

Module 7 — v2: Extended with platforms
├── Chosen strategy + specific platform
├── Platform criteria (free tier, DX, deploy flow)
├── Render vs Railway vs Fly.io vs AWS comparison
└── Result: "Within [strategy X], I choose [platform Y]"

Module 8 — v_final: Applied and validated
├── Executed decision: strategy + platform with evidence
├── REAL trade-offs (not theoretical) of the deployment
├── What worked, what didn't, what I'd change
├── Re-evaluation conditions with real metrics
└── Result: "My system is on [platform Y] because [justification with data]"

v_final is the most valuable document because it has real data, not estimates.

Structure of the Decision Matrix v_final

Complete template

# Decision Matrix v_final — [AI System Name]

**Date:** [Date]
**Author:** [Your name]
**System:** [Name of your AI app]
**Production URL:** [Real URL]

---

## 1. Final Decision

### Chosen strategy: [Managed/Serverless/Local/Self-hosted]
### Chosen platform: [Railway/Render/Fly.io/AWS]

**In one sentence:** [Why this combination is right for your case]

---

## 2. Project Context (Updated)

| Data | Value when deciding (M1) | Current value (M8) |
|------|----------------------|-------------------|
| Stage | [MVP/Growth/Scale] | [Current] |
| Users | [Estimate] | [Real or current estimate] |
| Requests/day | [Estimate] | [Real or current estimate] |
| Budget | [Range] | [Real spend] |
| Team | [Size] | [Current size] |
| Latency target | [Value] | [Measured value] |

---

## 3. Weighted Decision Matrix

### Criteria and Weights

| # | Criterion | Weight | Weight justification |
|---|----------|------|----------------------|
| 1 | [Criterion] | [N] | [Why this weight] |
| 2 | [Criterion] | [N] | [Why this weight] |
| ... | ... | ... | ... |
| | **Total** | **100** | |

### Evaluation (1-5)

| Criterion (Weight) | Local | Serverless | Managed | Self-hosted |
|-----------------|:-----:|:----------:|:-------:|:-----------:|
| [C1] ([P1]) | [score] | [score] | [score] | [score] |
| ... | ... | ... | ... | ... |
| **WEIGHTED TOTAL** | **[X]** | **[X]** | **[X]** | **[X]** |

### Platform Evaluation (within the chosen strategy)

| Criterion | Render | Railway | Fly.io | AWS |
|----------|:------:|:-------:|:------:|:---:|
| Free tier | [1-5] | [1-5] | [1-5] | [1-5] |
| Deploy simplicity | [1-5] | [1-5] | [1-5] | [1-5] |
| DX (developer experience) | [1-5] | [1-5] | [1-5] | [1-5] |
| Scaling options | [1-5] | [1-5] | [1-5] | [1-5] |
| Pricing transparency | [1-5] | [1-5] | [1-5] | [1-5] |
| Docker support | [1-5] | [1-5] | [1-5] | [1-5] |
| **TOTAL** | **[X]** | **[X]** | **[X]** | **[X]** |

---

## 4. Detailed Justification

### Why [strategy] and not the others

[3-5 paragraphs explaining the decision with concrete data]

### Why [platform] and not the others

[2-3 paragraphs with specific reasons]

### Accepted trade-offs

| Trade-off | What we gain | What we sacrifice |
|-----------|-------------|-------------------|
| [Trade-off 1] | [Benefit] | [Cost] |
| [Trade-off 2] | [Benefit] | [Cost] |
| [Trade-off 3] | [Benefit] | [Cost] |

---

## 5. Post-Deployment Validation

### What worked as expected
- ✅ [Aspect 1]
- ✅ [Aspect 2]

### What was different from expected
- ⚠️ [Aspect 1 — what we expected vs what happened]
- ⚠️ [Aspect 2]

### What we would change
- [Change 1 — why]
- [Change 2 — why]

---

## 6. Re-evaluation Conditions

| Trigger | Metric | Action |
|---------|---------|--------|
| [Trigger 1] | [Specific metric] | [What to do] |
| [Trigger 2] | [Specific metric] | [What to do] |
| [Trigger 3] | [Specific metric] | [What to do] |

**Scheduled review:** [Date, who, what to evaluate]

Complete Example: DocuSearch AI

Decision Matrix v_final applied

# Applied project data
project_context = {
    "name": "DocuSearch AI",
    "type": "RAG",
    "strategy_chosen": "Managed",
    "platform_chosen": "Railway",
    "url": "https://docusearch-ai.railway.app",
    "deploy_date": "2026-03-08",
}

# Criteria with weights (from M1, validated in M8)
criteria_final = {
    "Total monthly cost": {
        "weight": 20,
        "justification": "Budget <$50/month, it's a hard constraint",
    },
    "Operational complexity": {
        "weight": 25,
        "justification": "Team of 2, we can't dedicate hrs/week to ops",
    },
    "Time-to-deploy": {
        "weight": 20,
        "justification": "We iterate on prompts daily, fast deploy = better system",
    },
    "Scalability": {
        "weight": 10,
        "justification": "150 internal users, we don't expect 10x in 3 months",
    },
    "Control": {
        "weight": 10,
        "justification": "Internal but unregulated data, medium control is enough",
    },
    "Memory (ChromaDB)": {
        "weight": 15,
        "justification": "ChromaDB needs ~2GB of persistent RAM",
    },
}

# Final evaluation with real data
evaluation_final = {
    "Total monthly cost": {
        "Local (VPS)":     (4, "$24/month VPS + $15 APIs = $39"),
        "Serverless":      (2, "Stateless ChromaDB doesn't work on Lambda"),
        "Managed (Rwy)":   (4, "$5/month Railway + $15 APIs = $20"),
        "Self-hosted":     (1, "EC2 $30 + ops time = $200+"),
    },
    "Operational complexity": {
        "Local (VPS)":     (2, "Requires maintaining the VPS, updates, SSL"),
        "Serverless":      (3, "AWS manages infra but Lambda debugging is complex"),
        "Managed (Rwy)":   (5, "Git push and done, Railway manages everything"),
        "Self-hosted":     (1, "All manual, requires DevOps expertise"),
    },
    "Time-to-deploy": {
        "Local (VPS)":     (3, "5-15 min with CI/CD"),
        "Serverless":      (4, "2-5 min with SAM"),
        "Managed (Rwy)":   (5, "2 min from git push"),
        "Self-hosted":     (1, "20+ min with the whole process"),
    },
    "Scalability": {
        "Local (VPS)":     (2, "Limited vertical"),
        "Serverless":      (5, "Native auto-scaling"),
        "Managed (Rwy)":   (3, "Manual but simple scaling"),
        "Self-hosted":     (4, "Configurable but complex"),
    },
    "Control": {
        "Local (VPS)":     (4, "Direct SSH, OS control"),
        "Serverless":      (2, "Code only, no infra access"),
        "Managed (Rwy)":   (3, "Variables, logs, basic metrics"),
        "Self-hosted":     (5, "Full control"),
    },
    "Memory (ChromaDB)": {
        "Local (VPS)":     (5, "4GB+ of RAM available"),
        "Serverless":      (1, "Lambda is stateless, ChromaDB doesn't persist"),
        "Managed (Rwy)":   (4, "Railway Pro allows up to 8GB"),
        "Self-hosted":     (5, "Unlimited"),
    },
}

Score calculation

def calculate_final_scores(criteria: dict, evaluation: dict) -> dict:
    """Calculates weighted scores for the decision matrix v_final."""
    strategies = ["Local (VPS)", "Serverless", "Managed (Rwy)", "Self-hosted"]
    scores = {s: 0 for s in strategies}
    breakdown = {s: [] for s in strategies}

    for criterion, config in criteria.items():
        weight = config["weight"]
        for strategy in strategies:
            raw_score, note = evaluation[criterion][strategy]
            weighted = weight * raw_score
            scores[strategy] += weighted
            breakdown[strategy].append({
                "criterion": criterion,
                "weight": weight,
                "raw": raw_score,
                "weighted": weighted,
                "note": note,
            })

    max_possible = sum(c["weight"] for c in criteria.values()) * 5  # max score = 5

    return {
        "scores": dict(sorted(scores.items(), key=lambda x: x[1], reverse=True)),
        "max_possible": max_possible,
        "breakdown": breakdown,
    }

results = calculate_final_scores(criteria_final, evaluation_final)

print("=== Decision Matrix v_final ===")
print(f"Max possible: {results['max_possible']}")
print()
for strategy, score in results["scores"].items():
    pct = (score / results["max_possible"]) * 100
    bar = "█" * int(pct / 5) + "░" * (20 - int(pct / 5))
    print(f"  {strategy:20s} {score:4d}/{results['max_possible']} ({pct:.0f}%) {bar}")

Expected output

=== Decision Matrix v_final ===
Max possible: 500

  Managed (Rwy)         425/500 (85%) █████████████████░░░
  Local (VPS)           325/500 (65%) █████████████░░░░░░░
  Serverless            280/500 (56%) ███████████░░░░░░░░░
  Self-hosted           230/500 (46%) █████████░░░░░░░░░░░

Post-Deployment Validation Section

After deploying, update your matrix

post_deploy_validation = {
    "what_worked": [
        "Railway deploy from git push: 2 min consistently",
        "Free tier covered the first 2 months with no infra cost",
        "Excellent developer experience: logs, metrics, easy rollback",
        "ChromaDB works well with 2GB of RAM on Railway",
    ],
    "what_was_different": [
        {
            "expected": "Inference latency <3s",
            "actual": "First request after inactivity: 5-15s (cold start on free tier)",
            "impact": "Degraded UX for the first user of each session",
        },
        {
            "expected": "Railway Pro needed from day 1",
            "actual": "Free tier was enough for the first 150 users",
            "impact": "$5/month savings for 2 months",
        },
    ],
    "what_would_change": [
        "I would have started with Railway Pro to eliminate cold starts from day 1",
        "I would have implemented caching earlier — 30% of queries are repetitive",
    ],
}

Re-evaluation conditions with real data

re_evaluation_triggers = {
    "scale_trigger": {
        "metric": "Daily active users",
        "threshold": "> 500",
        "action": "Evaluate Railway Pro vs a VPS with more RAM",
        "current_value": 150,
    },
    "cost_trigger": {
        "metric": "Total monthly bill (infra + APIs)",
        "threshold": "> $100/month",
        "action": "Evaluate VPS (fixed cost) vs managed (variable)",
        "current_value": "$20/month",
    },
    "latency_trigger": {
        "metric": "Inference p95 latency",
        "threshold": "> 5s excluding cold starts",
        "action": "Evaluate infra with more resources or caching",
        "current_value": "2.8s p95",
    },
    "compliance_trigger": {
        "metric": "SOC2 requirement or sensitive data",
        "threshold": "Any new requirement",
        "action": "Evaluate self-hosted or AWS with VPC",
        "current_value": "Not required",
    },
}

review_schedule = {
    "next_review": "3 months from deploy",
    "reviewer": "[Your name]",
    "metrics_to_check": [
        "Real vs estimated traffic",
        "Real vs estimated costs",
        "Downtime incidents (count, duration)",
        "User feedback about latency",
        "Did any re-evaluation trigger fire?",
    ],
}

How to Present Your Decision

The 2-minute pitch

Structure:
1. Context (15s): "We have a RAG app for 150 internal users..."
2. Decision (15s): "We chose Railway (managed) because..."
3. Justification (45s): "The 3 criteria that weighed most were..."
4. Trade-offs (30s): "We sacrificed X in exchange for Y..."
5. Re-evaluation (15s): "We re-evaluate if..."

For a job interview

Question: "Tell me about an infrastructure decision you made"

"For DocuSearch AI, a RAG system that serves internal documentation,
I evaluated 4 deployment strategies across 6 weighted dimensions.

I chose Railway (managed platform) over AWS (self-hosted) because:
- With a team of 2, we couldn't dedicate 5+ hrs/week to ops
- The budget of <$50/month ruled out self-hosted
- ChromaDB requires persistent memory, which ruled out serverless
- The 2-minute time-to-deploy lets us iterate on prompts quickly

The trade-off was less control: we don't have SSH to the server or
granular networking configuration. But for our case —
internal unregulated data, 150 users — it was an acceptable trade-off.

I documented the decision with a decision matrix that includes weighted
criteria, cost estimation, and re-evaluation triggers.
We re-evaluate if we reach 500 users or if p95 latency exceeds 5 seconds."

Troubleshooting

Problem 1: "My M1 decision matrix no longer reflects my current case"

Cause: The data changed between M1 (when you planned) and M8 (when you executed).

Solution: That's exactly what v_final documents. The "Post-Deployment Validation" section captures the differences. Don't modify v1 or v2 — keep them as evidence of the evolution. v_final documents reality.

Problem 2: "Two strategies are still very close in score"

Solution:

# Sensitivity analysis: change a weight by ±10 and observe
sensitivity_test = {
    "Original weights": {"Managed": 425, "Local": 325},
    "Complexity +10, Scalability -10": {"Managed": 445, "Local": 305},
    "Cost +10, Time-to-deploy -10": {"Managed": 415, "Local": 335},
}
# If Managed wins in every scenario → robust decision
# If it depends on the scenario → document both as viable

Problem 3: "My manager wants to see the matrix but doesn't understand weights and scores"

Solution: Extract a 3-line executive summary at the start of the document:

## Executive Summary
We chose Railway (managed) for DocuSearch AI.
Cost: $20/month. Deploy: 2 minutes from git push. Uptime: 99.5%.
We re-evaluate in 3 months or if we reach 500 daily users.

Problem 4: "I chose a different platform from what my matrix recommended"

Solution: Document it honestly. Sometimes we choose based on non-quantifiable factors (familiarity with the tool, a colleague's recommendation, a tutorial we watched). Document: "The matrix recommended X, I chose Y because [reason]. The trade-off is [what I sacrificed]."

Hands-On Exercises

Exercise 1: Decision matrix v_final with your real data

Take your M1/M7 decision matrix and update it with real deployment data.

# Use the criteria and evaluation template shown above.
# Steps:
# 1. Copy your criteria and weights from M1
# 2. Update the evaluations (1-5) with real experience
# 3. Run calculate_final_scores()
# 4. Compare with what your v1 recommended

# Example update:
# M1 evaluation: "Managed time-to-deploy: 5 (estimated 1-3 min)"
# M8 evaluation: "Managed time-to-deploy: 5 (measured: 2 min consistently)"
# → The score doesn't change but now has real data

# M1 evaluation: "Serverless cost: 5 (estimated $5/month)"
# M8 evaluation: "Serverless cost: 1 (ChromaDB doesn't work on Lambda)"
# → The score changes drastically with real experience

Exercise 2: Post-deployment validation section

Document what worked, what was different, and what you'd change.

## Post-Deployment Validation

### What worked as expected
- ✅ Deploy from git push in <3 minutes consistently
- ✅ Costs within budget ($20/month vs $50 budget)
- ✅ Health checks work and detect problems quickly
- ✅ Rollback available with one click from the dashboard

### What was different from expected
- ⚠️ Cold starts on free tier: we expected 0ms, we measured 5-15s
  Impact: First request of each session is slow
- ⚠️ Logs more limited than we thought
  Impact: Debugging requires adding more logging to the code
- ⚠️ Latency to OpenAI varies between 1-4s (we thought 1-2s consistent)
  Impact: Total latency more variable than estimated

### What we would change
- We would have started with Railway Pro ($5/month) to avoid cold starts
- We would have implemented response caching from day 1
- We would have configured structured logging before the deploy

Exercise 3: Prepare the 2-minute pitch

Write your pitch following the structure: context → decision → justification → trade-offs → re-evaluation.

My pitch:

CONTEXT: "I built [name], a [type] app that [what it does],
for [how many users], with a budget of [budget]."

DECISION: "I chose [platform] as a [type] deployment strategy."

JUSTIFICATION: "The criteria that weighed most were:
1. [Criterion 1 with weight X%] — because [reason]
2. [Criterion 2 with weight Y%] — because [reason]
3. [Criterion 3 with weight Z%] — because [reason]"

TRADE-OFFS: "I sacrificed [what] in exchange for [what].
Specifically, [specific example of the trade-off]."

RE-EVALUATION: "I re-evaluate the decision if [trigger 1] or [trigger 2].
The next review is on [date]."

Total: ~2 minutes spoken, covers all the key points.

Exercise 4: Sensitivity analysis

Change your matrix weights by ±10 on the top 2 criteria and verify whether the recommendation changes.

import copy

def sensitivity_analysis(criteria, evaluation, top_criteria, delta=10):
    """Sensitivity test: varies weights and observes whether the winner changes."""
    base_results = calculate_final_scores(criteria, evaluation)
    base_winner = list(base_results["scores"].keys())[0]
    print(f"Base winner: {base_winner} ({list(base_results['scores'].values())[0]})")

    for criterion in top_criteria:
        for direction in [+delta, -delta]:
            test_criteria = copy.deepcopy(criteria)
            test_criteria[criterion]["weight"] += direction

            others = [c for c in test_criteria if c != criterion]
            adjust = -direction / len(others)
            for other in others:
                test_criteria[other]["weight"] += adjust

            results = calculate_final_scores(test_criteria, evaluation)
            winner = list(results["scores"].keys())[0]
            change = "CHANGED" if winner != base_winner else "same"
            sign = "+" if direction > 0 else ""
            print(f"  {criterion} {sign}{direction}: winner = {winner} [{change}]")

# Run:
sensitivity_analysis(
    criteria_final, evaluation_final,
    top_criteria=["Operational complexity", "Total monthly cost"]
)

Summary

• The decision matrix v_final integrates M1 (strategies) + M7 (platforms) + real deployment experience
• It's the most valuable version because it has real data, not estimates
• It includes: updated context, weighted matrix, justification, trade-offs, and post-deployment validation
• The "what was different" section is the most honest and useful part of the document
• Re-evaluation conditions with specific metrics prevent stagnant decisions
• The 2-minute pitch is how you communicate the decision in interviews and meetings
• Sensitivity analysis verifies that your decision is robust against changes in priorities

Additional Resources

1. Architecture Decision Records (ADR) — Standard format for documenting decisions
2. Lightweight Architecture Decision Records — Simplified version of ADRs
3. Decision Matrix — Wikipedia — Decision matrix theory
4. AWS Well-Architected Framework — Architecture evaluation framework
5. Sensitivity Analysis — Wikipedia — Sensitivity analysis theory
6. Technology Radar — Thoughtworks — Reference for evaluating tools and platforms