Module 1: Decision Framework for LLM Access

Module 1: Decision Framework for LLM Access

Next →

Module overview

Welcome to the most important module of this guide: the Decision Framework.

Most tutorials and courses about LLMs tell you "use the OpenAI API" and stop right there. The problem is that OpenAI is not always the best option for your project. It depends on your budget, privacy requirements, the speed you need, and how complex the implementation is.

This module teaches you how to decide which access strategy to use based on your specific requirements. It won't tell you "use X"; instead it gives you a structured framework to evaluate your needs and choose knowingly between the OpenAI API, local models (LM Studio, Ollama), multi-provider aggregators (OpenRouter), or serverless deployment (Modal).

By the end of this module, you'll be able to:

  • Evaluate projects against 5 critical dimensions (cost, quality, privacy, speed, simplicity)
  • Apply a decision matrix to pick the optimal provider
  • Identify trade-offs and specific use cases
  • Make informed decisions before writing a single line of code

This module is a mandatory prerequisite for modules 2-8. If you're not clear about WHICH provider you need, there's no point learning HOW to use OpenAI, Ollama, or Modal.

🎯 Module goal

Professional goal:

Be able to evaluate a project's AI requirements and choose the optimal LLM access strategy in under 30 minutes, based on a structured framework and quantitative data.

Why does it matter?

Choosing the wrong LLM provider can cost you:

  • Money: an unnecessary $500-5000/month if you use OpenAI when local Ollama would have been enough
  • Time: 2-4 weeks migrating from one provider to another
  • Privacy: sensitive data sent to the cloud when it should have stayed on-premise
  • Speed: 3-5s latency when you needed responses under 1s

This module saves you from those mistakes by teaching you to decide BEFORE you implement.

📚 Module content

Capsule 01: Module introduction (you are here)

  • Goal and philosophy of the decision framework
  • Why most tutorials fail (bias toward OpenAI)
  • Module progression

Capsule 02: The 5 evaluation dimensions

  • Cost ($/month, $/1M tokens, upfront costs)
  • Quality (accuracy, model capabilities)
  • Privacy (data retention, on-premise vs cloud)
  • Speed (latency, throughput)
  • Simplicity (setup time, technical complexity)

Capsule 03: The options landscape 2024-2026

  • OpenAI API (standard cloud)
  • LM Studio (local GUI, development)
  • Ollama (local CLI, production)
  • OpenRouter (multi-provider aggregator)
  • Modal (serverless deployment)
  • Quick comparison of each option

Capsule 04: The decision matrix

  • Structured framework for choosing
  • Use cases → recommended provider
  • Examples: e-commerce chatbot, legal analysis, MVP prototype, production app

Capsule 05: Quantitative trade-offs

  • Cost comparison ($/1M tokens)
  • Speed comparison (average latency)
  • Privacy comparison (retention policies)
  • Quality comparison (capabilities per model)

Capsule 06: Real-world use cases

  • Startup with no budget → local Ollama
  • Company with sensitive data → on-premise LM Studio
  • Quick prototype → OpenAI API
  • Production with flexibility → OpenRouter
  • Deploy without DevOps → Modal serverless

Capsule 07: Common mistakes when choosing

  • Mistake #1: Choosing by popularity (not by fit)
  • Mistake #2: Not considering costs at scale
  • Mistake #3: Ignoring privacy requirements
  • Mistake #4: Underestimating the complexity of local setup
  • Mistake #5: Having no fallback plan

Capsule 08: Mini-project - Requirements assessment

  • Hypothetical project: E-commerce chatbot
  • Apply the complete framework
  • Document the decision with justification
  • Compare your choice with the instructor's recommendation

🔗 Connection with other modules

Prerequisites:

  • None - This is the foundational module
  • You only need basic Python (not used in this module, but yes in 2-8)

This module prepares you for:

  • Module 2: OpenAI API (you'll learn it IF you chose standard cloud)
  • Module 3: LM Studio (you'll learn it IF you chose local GUI)
  • Module 4: Ollama (you'll learn it IF you chose local CLI)
  • Module 5: OpenRouter (you'll learn it IF you chose multi-provider)
  • Module 6: Modal (you'll learn it IF you chose serverless)
  • Module 7: Trade-offs Comparison (you'll validate your decision with real benchmarks)
  • Module 8: Unified Client (you'll integrate every option)

Recommended flow:

Module 1: Decision Framework
  ↓ (You decide: "My project needs X")
Module 2-6: Learn X specifically
  ↓ (You implement your choice)
Module 7: Validate with a benchmark
  ↓ (You confirm or adjust)
Module 8: Build an abstraction for future flexibility

⏱️ Estimated time

Reading and comprehension: 45-60 minutes

Breakdown per capsule:

  • Capsule 01: 5 min (introduction)
  • Capsule 02: 8-10 min (5 dimensions)
  • Capsule 03: 8-10 min (options landscape)
  • Capsule 04: 10-12 min (decision matrix)
  • Capsule 05: 8-10 min (quantitative trade-offs)
  • Capsule 06: 6-8 min (use cases)
  • Capsule 07: 5-7 min (common mistakes)
  • Capsule 08: 10-15 min (mini-project)

Total: 60-72 minutes

Note: This module is 100% conceptual (there's no code). Modules 2-8 do include runnable code.

🎓 What will you learn in this module?

By the end of this module, you'll be able to:

1. Evaluate projects against 5 dimensions

  • ✅ Estimate costs ($/month, $/1M tokens)
  • ✅ Assess quality requirements (which capabilities you need)
  • ✅ Identify privacy constraints (cloud vs on-premise)
  • ✅ Define speed requirements (latency, throughput)
  • ✅ Estimate implementation complexity (setup time)

2. Apply the decision matrix

  • ✅ Given a project, choose a provider in <30 min
  • ✅ Justify your decision with data
  • ✅ Identify the trade-offs of your choice
  • ✅ Have a fallback plan if the provider fails

3. Compare options quantitatively

  • ✅ Cost benchmarks (OpenAI: $X, Ollama: $0, Modal: $Y)
  • ✅ Speed benchmarks (OpenAI: 2.3s, Ollama: 5.8s local)
  • ✅ Privacy policies (OpenAI: 30 days, Ollama: never leaves your machine)
  • ✅ Compared capabilities (GPT-4 vs Mistral 7B vs Llama 2)

4. Avoid common mistakes

  • ✅ Don't choose by popularity (OpenAI is popular, not always optimal)
  • ✅ Don't ignore costs at scale (a prototype works, production doesn't)
  • ✅ Don't underestimate privacy (GDPR, HIPAA, sensitive data)
  • ✅ Don't skip the fallback plan (if OpenAI goes down, what do you do?)

💡 Module philosophy

Why the "Decision-First" approach

Typical problem:

Traditional tutorial:
1. "Here's how to use the OpenAI API"
2. [2-hour tutorial]
3. "Done!"

Result: You know how to use OpenAI, but:
- Is it the best option for your project? You don't know
- Are there cheaper alternatives? You don't know them
- What happens if OpenAI doesn't meet your requirements? You have no plan

Our approach:

Decision-First:
1. "Evaluate your requirements" (Module 1)
2. "Compare options with a structured framework"
3. "Choose the optimal provider"
4. THEN learn how to use it (Modules 2-6)

Result: You know why you chose X, which trade-offs you accepted,
and you have a fallback plan.

Key differentiator vs the competition

95% of courses/tutorials:

  • Teach only the OpenAI API
  • Assume it's the only option
  • Don't compare trade-offs
  • Don't give you a decision framework

This guide:

  • A full Module 1 about DECIDING
  • 5 options compared (cloud, local, serverless, aggregator)
  • A structured framework (not just "here are the options")
  • Quantitative trade-offs (not just "OpenAI is good")

Analogy:

Imagine learning to cook and the instructor only teaches you to use an electric oven. What happens if:

  • You have no electricity? (OpenAI is down)
  • You need to cook fast? (latency requirements)
  • You have a limited budget? (costs)
  • You need to cook on-site? (privacy)

A good instructor teaches you: electric oven, gas, grill, microwave, and how to decide which to use depending on the context.

That's what this module does: it gives you the framework to decide, not just one option.

🚫 What this module does NOT cover

This module does NOT cover:

❌ How to use each provider (that's Modules 2-6)

  • You won't see OpenAI API code here
  • You won't install LM Studio here
  • You won't deploy with Modal here
  • That comes later, once you decide

❌ Prompt engineering (that's another guide)

  • You won't see advanced prompting techniques
  • You won't see few-shot, chain-of-thought, etc.
  • This module is about ACCESS, not about optimal use

❌ Fine-tuning or training (that's advanced content)

  • We assume you use pre-trained models
  • Fine-tuning is a completely different topic

❌ RAG or embeddings (that's another guide)

  • This module is about accessing LLMs for chat/completions
  • RAG (Retrieval-Augmented Generation) is a different architectural pattern

❌ Comparison of specific models (that's Module 7)

  • Here you'll see the general landscape
  • Module 7 has detailed benchmarks of GPT-4 vs Claude vs Mistral

Clear scope: This module is about choosing an ACCESS strategy (cloud vs local vs serverless vs aggregator), NOT about what to do with the LLM once you have access.

✅ Success criteria

You've successfully completed this module when:

You can answer these questions:

  1. ✅ What are the 5 evaluation dimensions?

    • Answer: Cost, Quality, Privacy, Speed, Simplicity

  2. ✅ What access options exist in 2024-2026?

    • Answer: OpenAI API, LM Studio, Ollama, OpenRouter, Modal (at least 5)

  3. ✅ When would you use the OpenAI API vs local Ollama?

    • Answer: OpenAI if you need maximum quality and cost/privacy don't matter. Ollama if you need zero cost and absolute privacy (on-premise).

  4. ✅ What trade-offs do you accept when choosing local models?

    • Answer: Lower quality (vs GPT-4), higher latency (if hardware is limited), setup complexity. In exchange: zero cost, total privacy, no rate limits.

  5. ✅ How do you decide between OpenRouter vs OpenAI directly?

    • Answer: OpenRouter if you need flexibility (100+ models), cost optimization (pick the cheapest model), fallback strategies. OpenAI directly if you only need GPT-4 and don't want a middleman.

You can apply the framework:

  1. ✅ Given a hypothetical project, you choose a provider in <30 min
  2. ✅ You justify your decision with the 5 dimensions
  3. ✅ You identify the trade-offs of your choice
  4. ✅ You have a fallback plan if your first option fails

Validation test:

Hypothetical project: A startup building a technical support chatbot

Requirements:

  • 10,000 users/month
  • NON-sensitive data (public support)
  • Needs fast responses (<2s)
  • Budget: $500/month maximum
  • A team of 2 junior developers

Which provider would you choose and why?

See the recommended answer

Choice: OpenAI API (GPT-3.5-turbo)

Justification per dimension:

  1. Cost: $500/month covers ~500k tokens/day with GPT-3.5 ($0.0015/1k input). 10k users × 3 queries/user × 200 tokens = 6M tokens/month = $9/month. ✅ Within budget.

  2. Quality: GPT-3.5 is enough for technical support (you don't need GPT-4). ✅ Meets requirements.

  3. Privacy: NON-sensitive data (public support). OpenAI's policy (30-day retention) is acceptable. ✅ No constraint.

  4. Speed: OpenAI ~2s latency. ✅ Meets the <2s requirement.

  5. Simplicity: A junior team. The OpenAI API is the simplest (official SDK, excellent docs). ✅ Setup in 1 day vs 1 week with local Ollama.

Accepted trade-offs:

  • You depend on OpenAI (vendor lock-in). Mitigation: OpenRouter as a fallback (API compatible).
  • You pay per use (vs free with Ollama). Justification: $9/month is negligible vs the time a junior team spends fighting with a local setup.

Fallback plan:

  • If OpenAI goes down: OpenRouter with gpt-3.5-turbo (same model, different provider)
  • If GPT-3.5 isn't enough: Upgrade to GPT-4 (within budget)

If your answer is similar (even if you choose a different provider but justify it well), ✅ you PASSED the module.

🎯 Skills you'll develop

This module develops technical decision-making skills, not programming skills (that comes later).

Evaluation skills:

  1. Requirements analysis - Extract the critical dimensions of a project
  2. Quantitative comparison - Use data ($/month, latency), not just intuition
  3. Trade-off identification - Understand what you gain and what you lose
  4. Systems thinking - See the long-term implications

Decision skills:

  1. Structured framework - Apply a repeatable process (not ad-hoc)
  2. Data-backed justification - Defend your choice with numbers
  3. Contingency plan - Have a fallback before you need it
  4. Prioritization - Know which dimension is most critical in your context

These skills are transferable:

  • They apply to choosing a database (Postgres vs MongoDB)
  • They apply to choosing a cloud provider (AWS vs GCP vs Azure)
  • They apply to choosing a framework (FastAPI vs Flask vs Django)

Learning to DECIDE is more valuable than learning to USE a specific tool.

📖 How to use this module

Recommended strategy:

  1. Read sequentially (Capsules 01 → 02 → 03 → ... → 08)

    • Don't skip capsules
    • Each one builds on the previous

  2. Take notes on the 5 dimensions (capsule 02)

    • You'll have to apply them in capsule 08

  3. Compare options actively (capsules 03-05)

    • Don't just read, think "which one would I use in project X?"

  4. Apply the framework (capsule 08)

    • The mini-project is critical
    • Don't skip it thinking "I already get it"

Suggested time:

Option A: One session (60-75 min)

  • Read it all in one go
  • Advantage: Fresh context
  • Disadvantage: It can be dense

Option B: Two sessions (30-40 min each)

  • Session 1: Capsules 01-04 (introduction, dimensions, options, matrix)
  • Session 2: Capsules 05-08 (trade-offs, use cases, mistakes, project)
  • Advantage: You absorb it better
  • Disadvantage: You need to remember the context

Recommendation: Option B (two sessions with 1 day in between)

🔗 Resources for this module

Official provider documentation:

  1. OpenAI Pricing - Up-to-date costs
  2. Ollama Model Library - Models available locally
  3. Modal Documentation - Serverless deployment
  4. OpenRouter Models - Complete list of models

Context articles:

  1. State of LLMs 2024 - Current trends
  2. Local vs Cloud LLMs - Detailed comparison
  3. Cost Optimization for LLMs - Saving strategies

Tools and comparators:

  1. LLM Comparison Tool - Compare models side-by-side
  2. Artificial Analysis - Up-to-date LLM benchmarks

Decision guides:

  1. Choosing the Right LLM - Chip Huyen's guide
  2. LLM Selection Framework - Patterns and anti-patterns

Note: These resources are for context; you don't need to read them to complete the module. Capsules 02-08 are self-contained.

💬 Frequently asked questions

Do I need previous experience with LLMs?

No. This module assumes zero experience. You only need to understand that LLMs are language models (like GPT-4, Claude, Llama) that generate text.

Am I going to write code in this module?

No. Module 1 is 100% conceptual. Code comes in Modules 2-8.

Which provider should I choose IF I don't have a specific project yet?

Read the whole module first. If afterward you still have no project, the recommendation is:

  1. OpenAI API (to learn quickly, the simplest)
  2. Ollama (if you have no budget)

But don't choose until you complete the module. The goal is for you to learn to decide knowingly.

Does this module tell me which is "the best" provider?

No. There is no "the best." There is "the best FOR your project." This module teaches you to identify which is yours.

What if I choose wrong?

Migrating to another provider later is NOT the end of the world. But:

  • It costs time (2-4 weeks)
  • It can cost money (refactoring)
  • It can break production (downtime)

That's why this module exists: so you decide well the first time.

🚀 Ready to start?

Next step:

Go to Capsule 02: The 5 evaluation dimensions

There you'll learn the complete framework to evaluate any project according to:

  1. Cost (how much can I spend?)
  2. Quality (how good does it need to be?)
  3. Privacy (where can my data go?)
  4. Speed (how fast do I need responses?)
  5. Simplicity (how much time do I have to implement?)

These 5 dimensions are the foundation of the entire decision framework.

Reading time: 5 minutes
Next: 02-the-five-dimensions-of-evaluation.md

Next →The 5 Evaluation Dimensions for Choosing an LLM Provider