Mini-Project: 100% Local Chatbot

Project overview

Module 3's final project: a fully local chatbot (cost $0, 100% privacy) using LM Studio.

You'll take the Module 2 chatbot and adapt it to run 100% offline with local Mistral 7B.

Time: 30-45 minutes
Difficulty: Medium

---

🎯 Goal

Build a local CLI chatbot with:

• ✅ Conversations with context
• ✅ Cost tracking (confirm $0)
• ✅ Performance metrics (latency)
• ✅ Persistence (JSON)
• ✅ 100% offline (no cloud API calls)

---

💻 Full Code

#!/usr/bin/env python3
"""
100% Local Chatbot with LM Studio
Module 3 - Final Project
"""

import json
import time
from datetime import datetime
from pathlib import Path
from openai import OpenAI

# ============================================================================
# CONFIGURATION
# ============================================================================

# LM Studio client (local)
client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="not-needed"
)

MODEL = "mistral-7b-instruct"  # Adjust to your model

Path("conversations_local").mkdir(exist_ok=True)

# ============================================================================
# LOCAL CHATBOT CLASS
# ============================================================================

class LocalChatBot:
    """100% local chatbot with LM Studio."""
    
    def __init__(self):
        self.messages = [
            {"role": "system", "content": "You are a helpful and friendly assistant."}
        ]
        self.total_time = 0.0
        self.query_count = 0
        self.conversation_id = datetime.now().strftime("%Y%m%d_%H%M%S")
    
    def chat(self, user_message: str) -> str:
        """Send a message and return the response."""
        
        self.messages.append({"role": "user", "content": user_message})
        
        # Measure latency
        start = time.time()
        
        try:
            response = client.chat.completions.create(
                model=MODEL,
                messages=self.messages,
                temperature=0.7,
                max_tokens=150
            )
            
            latency = time.time() - start
            self.total_time += latency
            self.query_count += 1
            
            assistant_message = response.choices[0].message.content
            self.messages.append({"role": "assistant", "content": assistant_message})
            
            print(f"[Latency: {latency:.2f}s]")
            
            return assistant_message
            
        except Exception as e:
            print(f"❌ Error: {e}")
            return None
    
    def show_stats(self):
        """Show statistics."""
        if self.query_count == 0:
            print("\n📊 No queries yet")
            return
        
        avg_latency = self.total_time / self.query_count
        
        print("\n" + "="*50)
        print("📊 STATISTICS")
        print("="*50)
        print(f"Queries:        {self.query_count}")
        print(f"Total time:     {self.total_time:.2f}s")
        print(f"Avg latency:    {avg_latency:.2f}s")
        print(f"💰 Cost:        $0.00 (100% local!)")
        print("="*50 + "\n")
    
    def show_history(self):
        """Show the history."""
        print("\n" + "="*50)
        print("HISTORY")
        print("="*50)
        
        for i, msg in enumerate(self.messages[1:], 1):
            role = "YOU" if msg["role"] == "user" else "BOT"
            print(f"\n[{i}] {role}: {msg['content']}")
        
        print("\n" + "="*50 + "\n")
    
    def save_conversation(self):
        """Save the conversation."""
        filename = f"conversations_local/conversation_{self.conversation_id}.json"
        
        data = {
            "conversation_id": self.conversation_id,
            "timestamp": datetime.now().isoformat(),
            "model": MODEL,
            "messages": self.messages[1:],
            "query_count": self.query_count,
            "total_time": self.total_time,
            "avg_latency": self.total_time / self.query_count if self.query_count > 0 else 0,
            "cost": 0.0  # $0 local!
        }
        
        with open(filename, "w", encoding="utf-8") as f:
            json.dump(data, f, indent=2, ensure_ascii=False)
        
        print(f"💾 Conversation saved: {filename}")

# ============================================================================
# MAIN
# ============================================================================

def main():
    """Main CLI."""
    
    print("\n" + "="*50)
    print("🤖 100% LOCAL CHATBOT (LM Studio)")
    print("="*50)
    print(f"\nModel: {MODEL}")
    print("Cost: $0.00 (offline)")
    print("\nCommands:")
    print("  - 'salir'       → Quit")
    print("  - 'historial'   → See history")
    print("  - 'stats'       → See stats")
    print("\n" + "="*50 + "\n")
    
    # Check the LM Studio server
    print("Checking the LM Studio server...")
    try:
        models = client.models.list()
        print(f"✅ Server active ({len(models.data)} models available)\n")
    except Exception as e:
        print(f"❌ Error: {e}")
        print("\n⚠️ Make sure LM Studio is running with the server active")
        print("   Settings → Local Server → Start Server\n")
        return
    
    bot = LocalChatBot()
    
    # Main loop
    while True:
        try:
            user_input = input("You: ").strip()
            
            if user_input.lower() in ["salir", "exit", "quit"]:
                print("\n👋 See you soon!\n")
                bot.show_stats()
                bot.save_conversation()
                break
            
            if user_input.lower() == "historial":
                bot.show_history()
                continue
            
            if user_input.lower() == "stats":
                bot.show_stats()
                continue
            
            if not user_input:
                print("⚠️ Type a message.\n")
                continue
            
            response = bot.chat(user_input)
            
            if response:
                print(f"\nBot: {response}\n")
        
        except KeyboardInterrupt:
            print("\n\n👋 Interrupted by the user\n")
            bot.show_stats()
            bot.save_conversation()
            break
        
        except Exception as e:
            print(f"\n❌ Unexpected error: {e}\n")
            break

if __name__ == "__main__":
    main()

---

🚀 Usage

1. Start the LM Studio server:

• LM Studio → Settings → Start Server
• Check for 🟢 "Server running on port 1234"

2. Load a model:

• Home → My Models → Load (Mistral 7B)

3. Run the chatbot:

python chatbot_local.py

---

📊 Example Conversation

==================================================
🤖 100% LOCAL CHATBOT (LM Studio)
==================================================

Model: mistral-7b-instruct
Cost: $0.00 (offline)

Commands:
  - 'salir'       → Quit
  - 'historial'   → See history
  - 'stats'       → See stats

==================================================

Checking the LM Studio server...
✅ Server active (1 models available)

You: Hi, what model are you?
[Latency: 4.2s]

Bot: I'm Mistral 7B, a local language model running on your machine.

You: What is the capital of France?
[Latency: 3.8s]

Bot: The capital of France is Paris.

You: stats

==================================================
📊 STATISTICS
==================================================
Queries:        2
Total time:     8.00s
Avg latency:    4.00s
💰 Cost:        $0.00 (100% local!)
==================================================

You: salir

👋 See you soon!

==================================================
📊 STATISTICS
==================================================
Queries:        2
Total time:     8.00s
Avg latency:    4.00s
💰 Cost:        $0.00 (100% local!)
==================================================

💾 Conversation saved: conversations_local/conversation_20240215_143022.json

---

✅ Self-Assessment Rubric

Functionality (40 pts):

• (10) The chatbot answers coherently
• (10) Maintains context
• (10) Commands work (historial, stats, salir)
• (10) Conversations saved

Performance (30 pts):

• (10) Latency tracking works
• (10) Accurate stats
• (10) Cost confirmed at $0

Local (30 pts):

• (15) Works without the OpenAI API
• (15) Works offline (disconnect WiFi and test)

Total: ___/100

---

🎯 Comparison with Module 2

Metric	Module 2 (OpenAI)	Module 3 (LM Studio)
Latency	1.5s	4.0s (2.6x slower)
Cost (100 queries)	$0.10	$0.00
Privacy	Cloud	100% local ✅
Offline	❌	✅
Setup	5 min	30 min

---

🔗 Optional Extensions

1. Side-by-side comparator:

# Send the query to both (OpenAI + LM Studio)
# Compare responses and latency

2. Auto-switch by latency:

# If LM Studio >10s → fall back to OpenAI

3. Hybrid mode:

# Simple queries → LM Studio ($0)
# Complex queries → GPT-4 ($$$)

---

✅ Module 3 Summary

What you mastered:

• ✅ Complete LM Studio setup
• ✅ Model management (download, quantization)
• ✅ OpenAI-compatible local API
• ✅ Trivial code migration
• ✅ 100% local chatbot working

Trade-offs understood:

• Cost $0 vs 3-5x slower latency
• Maximum privacy vs more complex setup
• Offline vs 10% lower quality

---

➡️ Next step

Next module: Module 4 - Ollama (Local CLI)

You'll learn Ollama, an alternative to LM Studio but:

• CLI instead of GUI (more pro)
• Docker support (production)
• Multi-node clustering (scalability)

Time: 2-3 hours

---

Congratulations! You completed Module 3. 🎉

Total module time: 2-3 hours
Next: Module 4 - Ollama