OpenAI-Compatible Local API

Overview

LM Studio can act as an OpenAI-compatible local API server. This lets you use your Module 2 code without changes.

Time: 25 minutes
Difficulty: Medium

---

🎯 Objectives

• ✅ Start the local API server
• ✅ Verify the endpoint with curl
• ✅ Connect with the OpenAI SDK
• ✅ Reuse your Module 2 code

---

🚀 Step 1: Start the Server

In LM Studio:

1. ⚙️ Settings tab
2. "Local Server" section
3. Click [Start Server]

Default endpoint:

http://localhost:1234/v1

Port: 1234 (configurable in Settings)

---

✅ Step 2: Verify with Curl

# List models
curl http://localhost:1234/v1/models

# Expected output:
{
  "data": [
    {
      "id": "mistral-7b-instruct",
      "object": "model"
    }
  ]
}

---

💻 Step 3: Connect with Python

Code (drop-in replacement):

from openai import OpenAI

# Change only base_url and api_key
client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="not-needed"  # API key not needed (local)
)

# The rest of the code is IDENTICAL to Module 2
response = client.chat.completions.create(
    model="mistral-7b-instruct",
    messages=[
        {"role": "user", "content": "Hi"}
    ]
)

print(response.choices[0].message.content)

All the rest of the Module 2 code works without changes.

---

🔄 Step 4: Migrate the Module 2 Chatbot

Required change (2 lines):

# Before (Module 2):
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Now (Module 3):
client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="not-needed"
)

The rest of the code (conversations, context, error handling) is identical.

---

📊 Comparison: Cloud vs Local

Dimension	OpenAI API	LM Studio Local
Cost	$0.50/1M tokens	$0
Latency	1.5s	5-15s (CPU)
Privacy	Cloud	100% local
Quality (7B)	GPT-3.5: 70% MMLU	Mistral: 62% MMLU
Setup	5 min	30 min

---

🐛 Troubleshooting

Error: Connection refused

Cause: The server isn't started

Solution:

• LM Studio → Settings → Start Server
• Check for 🟢 "Server running"

---

Error: Model not found

Cause: Incorrect model name

Solution:

# List available models
curl http://localhost:1234/v1/models

# Use the exact "id" in your code

---

✅ Summary

• Local API server in LM Studio
• 100% compatible with the OpenAI SDK
• Minimal change: base_url + api_key
• All of the Module 2 code is reusable

---

Next: 06-code-migration.md