Migrating OpenAI Code to LM Studio

Overview

You'll take the Module 2 chatbot and migrate it to LM Studio locally. Minimal code change, maximum impact.

Time: 20 minutes
Difficulty: Low

---

🎯 Objectives

• ✅ Migrate the Module 2 chatbot to LM Studio
• ✅ Verify it works locally
• ✅ Measure the performance differences
• ✅ Compare costs

---

🔄 Step 1: Original Code (Module 2)

# chatbot_openai.py (Module 2)
from openai import OpenAI
import os
from dotenv import load_dotenv

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

messages = [
    {"role": "system", "content": "You are a helpful assistant."}
]

def chat(user_message):
    messages.append({"role": "user", "content": user_message})
    
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=messages
    )
    
    assistant_message = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_message})
    
    return assistant_message

# CLI
while True:
    user_input = input("You: ")
    if user_input.lower() == "salir":
        break
    response = chat(user_input)
    print(f"Bot: {response}\n")

---

✅ Step 2: Migrated Code (LM Studio)

# chatbot_lmstudio.py (Module 3)
from openai import OpenAI

# ONLY CHANGE: base_url and api_key
client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="not-needed"
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."}
]

def chat(user_message):
    messages.append({"role": "user", "content": user_message})
    
    response = client.chat.completions.create(
        model="mistral-7b-instruct",  # Change to a local model
        messages=messages
    )
    
    assistant_message = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_message})
    
    return assistant_message

# CLI (IDENTICAL)
while True:
    user_input = input("You: ")
    if user_input.lower() == "salir":
        break
    response = chat(user_input)
    print(f"Bot: {response}\n")

Total changes: 3 lines

1. base_url points to localhost
2. api_key not needed
3. model changes to a local model

---

📊 Step 3: Performance Comparison

Test script:

import time
from openai import OpenAI

# OpenAI Cloud
client_cloud = OpenAI(api_key="sk-...")

# LM Studio Local
client_local = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="not-needed"
)

def test_latency(client, model, name):
    start = time.time()
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": "Hi"}]
    )
    latency = time.time() - start
    print(f"{name}: {latency:.2f}s")
    return latency

# Test
lat_cloud = test_latency(client_cloud, "gpt-3.5-turbo", "OpenAI Cloud")
lat_local = test_latency(client_local, "mistral-7b-instruct", "LM Studio")

print(f"\nDifference: {lat_local/lat_cloud:.1f}x slower (local)")

Expected output:

OpenAI Cloud: 1.5s
LM Studio: 8.2s

Difference: 5.5x slower (local)

---

💰 Step 4: Cost Comparison

1000 queries (500 average tokens):

OpenAI API:

1000 queries × 500 tokens × $0.002/1k = $1.00

LM Studio:

$0 (operating cost)

Savings: $1.00 for every 1000 queries

Break-even: Immediate (cost $0 from query 1)

---

🎯 When to Use Each Option

Use the OpenAI API when:

• You need maximum quality (GPT-4)
• Latency is critical (<2s)
• You don't have powerful hardware
• Quick prototype

Use LM Studio when:

• Privacy is critical (on-premise)
• Budget is $0 (side projects)
• High volume (>100k queries/month)
• Local development (no internet)

---

🔄 Step 5: Hybrid Strategy

import os

# Detect the environment
IS_PRODUCTION = os.getenv("ENV") == "production"

if IS_PRODUCTION:
    # Production: OpenAI (maximum quality)
    client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
    model = "gpt-3.5-turbo"
else:
    # Development: LM Studio (free, local)
    client = OpenAI(
        base_url="http://localhost:1234/v1",
        api_key="not-needed"
    )
    model = "mistral-7b-instruct"

# The rest of the code is identical
response = client.chat.completions.create(
    model=model,
    messages=[...]
)

Advantage: Develop for free locally, deploy to the cloud for production.

---

✅ Summary

• Trivial migration (3 lines changed)
• 95% reusable code
• Trade-off: Speed vs Cost
• Hybrid strategy possible

---

Next: 07-performance-comparison.md