Module 3: LM Studio - Introduction

What will you learn in this module?

In Module 2 you learned the OpenAI API (cloud, pay-per-use). Now you'll learn to run LLMs locally on your machine with LM Studio, a desktop application with a graphical interface.

Advantages of local:

• ✅ $0 cost (only the hardware you already have)
• ✅ 100% privacy (data never leaves your machine)
• ✅ No internet needed (works offline)
• ✅ No rate limits (use as much as you want)

LM Studio specifically:

• ✅ Graphical interface (no CLI, no code at the start)
• ✅ Download models with 1 click
• ✅ OpenAI-compatible API (reuse your Module 2 code!)

By the end, you'll have an LLM running locally and you'll understand the trade-offs vs cloud.

---

🎯 Module goal

Master LM Studio: From setup to a working project

Deliverables:

1. LM Studio installed and running
2. A model downloaded (Mistral 7B or similar)
3. A local API running (OpenAI-compatible)
4. The Module 2 chatbot working with the local API (without changing the code!)

Final project: The same Module 2 chatbot, but 100% local (zero API calls to OpenAI)

---

📋 Module structure

Capsule progression:

Phase 1: Setup (capsules 1-3)

• 01: Module introduction
• 02: Installing LM Studio
• 03: Downloading models (model zoo)

Phase 2: Basic Use (capsules 4-5)

• 04: Graphical interface (chat, playground)
• 05: Local API (OpenAI-compatible server)

Phase 3: Integration (capsules 6-7)

• 06: Migrating OpenAI code to LM Studio
• 07: Performance comparison (local vs cloud)

Phase 4: Project (capsule 8)

• 08: Mini-project - 100% local chatbot

---

🔗 Connections with other modules

Prerequisite:

• Module 2 (OpenAI API): You'll use the same code, only changing the endpoint

Direct comparison:

• Module 2: Cloud ($$$), maximum quality, simplicity
• Module 3: Local ($0), privacy, medium quality

Next step:

• Module 4 (Ollama): Local CLI (production, Docker, scaling)

---

⏱️ Estimated time

Total module: 2-3 hours

Breakdown:

• Capsules 1-3 (Setup): 45 minutes
• Capsules 4-5 (Basic use): 30 minutes
• Capsules 6-7 (Integration): 45 minutes
• Capsule 8 (Project): 30-45 minutes

Note: Downloading models can take 10-30 min (depending on your internet)

---

🛠️ Skills you'll develop

Technical:

1. LM Studio:

   • Multi-platform installation (Mac/Windows/Linux)
   • Model management (download, switching)
   • Local API setup
   • Quantization (4-bit, 8-bit)

2. Local LLM fundamentals:

   • Hardware requirements (RAM, GPU)
   • Model formats (GGUF)
   • Inference speed optimization
   • Context window management

3. API compatibility:

   • Drop-in replacement for the OpenAI SDK
   • Change only base_url
   • Same code, different backend

Conceptual:

1. Local vs Cloud trade-offs:

   • Cost: $0 vs $$$
   • Quality: Medium vs High
   • Privacy: Maximum vs Medium
   • Speed: Variable vs Consistent

2. Hardware considerations:

   • RAM requirements (8GB min, 16GB recommended)
   • GPU acceleration (optional but improves 10x)
   • Storage (models 4-8GB each)

3. Model selection:

   • Mistral 7B vs Llama 2 13B vs Llama 2 70B
   • Quantization (Q4 vs Q8)
   • Context window (4k vs 32k)

---

📦 Prerequisites

Required skills:

• ✅ Module 2 completed (OpenAI API)
• ✅ Basic Python (for the integration)
• ✅ Basic terminal (running commands)

Minimum hardware:

Component	Minimum	Recommended	Optimal
RAM	8GB	16GB	32GB+
Storage	10GB free	50GB free	100GB+
CPU	Intel i5/AMD Ryzen 5	Intel i7/AMD Ryzen 7	Apple M1/M2/M3
GPU	None (CPU only)	NVIDIA 4GB VRAM	NVIDIA 8GB+ VRAM
OS	macOS 11+, Windows 10+, Linux	macOS 12+, Windows 11	Latest

Note: It works on modest hardware, but more RAM = better models and more speed.

---

Check your hardware:

macOS:

# RAM
sysctl hw.memsize

# CPU
sysctl -n machdep.cpu.brand_string

# GPU (Apple Silicon)
system_profiler SPDisplaysDataType

Windows:

# RAM
systeminfo | findstr "Total Physical Memory"

# CPU
wmic cpu get name

# GPU
wmic path win32_VideoController get name

Linux:

# RAM
free -h

# CPU
lscpu

# GPU
lspci | grep -i vga

---

💰 Module cost

LM Studio:

• Download: Free (open source)
• License: MIT (commercial use allowed)

Models:

• Download: Free (Hugging Face, open source)
• License: Varies (Mistral: Apache 2.0, Llama 2: Meta license)

Hardware:

• If you already have a laptop/desktop: $0
• If you need a RAM upgrade: $50-200
• If you need a GPU: $300-1500 (optional)

Total for the module: $0 (assuming existing hardware)

---

🎯 Why LM Studio vs Ollama?

Both are local, but they're for different use cases:

Feature	LM Studio	Ollama
Interface	GUI (point & click)	CLI (terminal)
Difficulty	Low (beginners)	Medium (devs)
Ideal use	Development, prototyping	Production, servers
Docker	No	Yes
Scaling	No (single machine)	Yes (multi-node)

Summary:

• LM Studio: Personal laptop, development
• Ollama: Servers, production

This module: LM Studio (friendlier to learn with)
Module 4: Ollama (more powerful for production)

---

📊 What you'll build

Final mini-project (capsule 08):

100% local chatbot (no OpenAI API)

You'll take the Module 2 chatbot and migrate it to LM Studio:

• The same Python code
• You only change base_url and api_key
• Cost: $0 (vs ~$0.50 in Module 2)
• Privacy: 100% local

Example change (2 lines):

# Module 2 (OpenAI cloud)
client = OpenAI(api_key="sk-proj-...")

# Module 3 (LM Studio local)
client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="not-needed"
)

All the rest of the code is identical.

---

🔄 Module methodology

Visual-practical learning:

Pattern: GUI first → API later

1. Explore the GUI (15 minutes):

   • Download a model
   • Chat in the interface
   • See how it works visually

2. Enable the API (10 minutes):

   • Start the local server
   • Test with curl
   • Verify it works

3. Integrate with code (30 minutes):

   • Migrate the Module 2 chatbot
   • Run it locally
   • Compare the results

Approach: Less theory, more hands-on (because the GUI is self-explanatory)

---

✅ Module success criteria

You completed it successfully when:

Setup:

• LM Studio installed and opens correctly
• You downloaded 1+ model (Mistral 7B recommended)
• You chatted in the GUI (interface working)

API:

• Local server running (localhost:1234)
• Test with curl successful
• The OpenAI SDK connects to the local API

Integration:

• The Module 2 chatbot works with LM Studio (without the OpenAI API)
• You compared performance (local vs cloud)

Understanding:

• You understand the trade-offs (cost, quality, speed)
• You know when to use local vs cloud
• You know the hardware limitations

If you meet all of them: ✅ Module 3 passed

---

🔗 Module resources

LM Studio:

1. LM Studio Website - Official download
2. LM Studio Docs - Documentation
3. GitHub - Source code
4. Discord - Community support

Models:

1. Hugging Face - Model hub
2. Mistral AI - Mistral models
3. Meta Llama - Llama 2/3

Community:

1. r/LocalLLaMA - Subreddit
2. LM Studio Discord - Real-time chat

---

🚀 Get started

Next step: 02-installing-lm-studio.md

In the next capsule:

• You'll download LM Studio (Mac/Windows/Linux)
• You'll install the application
• You'll verify it works
• First run (UI tour)

Time: 15 minutes
Difficulty: Very low

---

📝 Important notes

⚠️ Warnings:

1. Downloading models is slow:

   • Mistral 7B: ~4GB (10-30 min depending on your internet)
   • Llama 2 70B: ~40GB (1-3 hours)
   • Download on stable WiFi

2. Hardware matters:

   • 8GB RAM: Only small models (7B)
   • 16GB RAM: Medium models (13B)
   • 32GB+ RAM: Large models (70B)

3. A GPU accelerates but isn't required:

   • Without a GPU: 5-15s per response (CPU)
   • With a GPU: 1-3s per response
   • Apple M1/M2: Very fast (Metal acceleration)

✅ Tips for success:

1. Start small:

   • First model: Mistral 7B (4GB, fast)
   • Later: Experiment with larger ones

2. USE quantization:

   • Q4 (4-bit): Faster, less RAM
   • Q8 (8-bit): Slower, better quality
   • Start with Q4

3. Close heavy apps:

   • Chrome with 50 tabs = 8GB RAM
   • Free up RAM for LM Studio

---

Total estimated time: 2-3 hours
Next: 02-installing-lm-studio.md