Python Integration with Ollama

Overview

You'll build a chatbot with Ollama using Python. You'll reuse code from previous modules with minimal changes.

Time: 30 minutes
Difficulty: Medium

---

🎯 Objectives

• ✅ Migrate the chatbot to Ollama
• ✅ Compare with OpenAI/LM Studio
• ✅ Measure performance

---

💻 Code: Ollama Chatbot

#!/usr/bin/env python3
"""Chatbot with the Ollama API"""

from openai import OpenAI

# Ollama client (OpenAI SDK compatible)
client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."}
]

def chat(user_message: str) -> str:
    messages.append({"role": "user", "content": user_message})
    
    response = client.chat.completions.create(
        model="mistral",
        messages=messages
    )
    
    assistant_message = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_message})
    
    return assistant_message

# CLI
while True:
    user_input = input("You: ")
    if user_input.lower() == "salir":
        break
    response = chat(user_input)
    print(f"Bot: {response}\n")

Change vs Module 2: Only 2 lines (base_url + api_key)

---

🔄 Multi-Provider Comparison

import os
from openai import OpenAI

PROVIDER = os.getenv("LLM_PROVIDER", "ollama")  # ollama, openai, lmstudio

if PROVIDER == "openai":
    client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
    model = "gpt-3.5-turbo"
elif PROVIDER == "lmstudio":
    client = OpenAI(base_url="http://localhost:1234/v1", api_key="not-needed")
    model = "mistral-7b-instruct"
elif PROVIDER == "ollama":
    client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
    model = "mistral"

# The rest of the code is identical
response = client.chat.completions.create(
    model=model,
    messages=[...]
)

Advantage: Switch provider with an environment variable

---

📊 Performance Benchmark

import time
from openai import OpenAI

def benchmark_provider(base_url, model, name):
    client = OpenAI(base_url=base_url, api_key="test")
    
    start = time.time()
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": "Hi"}]
    )
    latency = time.time() - start
    
    print(f"{name}: {latency:.2f}s")
    return latency

# Test
print("Benchmark (first queries, cold start):")
benchmark_provider("http://localhost:11434/v1", "mistral", "Ollama")
benchmark_provider("http://localhost:1234/v1", "mistral-7b-instruct", "LM Studio")

Typical output:

Ollama: 4.5s
LM Studio: 4.8s

Similar because both are local with the same hardware

---

🚀 Ollama-Specific Features

Streaming responses:

response = client.chat.completions.create(
    model="mistral",
    messages=[...],
    stream=True  # Ollama default
)

for chunk in response:
    print(chunk.choices[0].delta.content, end="", flush=True)

---

Custom parameters:

response = client.chat.completions.create(
    model="mistral",
    messages=[...],
    temperature=0.7,
    top_p=0.9,
    max_tokens=500,
    # Ollama-specific
    num_ctx=4096,  # Context window
    repeat_penalty=1.1  # Penalizes repetition
)

---

✅ Ollama vs LM Studio Advantages

Feature	Ollama	LM Studio
CLI	✅	❌ (GUI only)
Automation	✅ Scripts	❌ Manual
Docker	✅	❌
CI/CD	✅	❌
GUI	❌	✅
Beginner-friendly	⚠️	✅

Recommendation:

• Simple local dev: LM Studio
• Production/servers: Ollama

---

🐛 Troubleshooting

Error: Model not loaded

# Check the models
ollama list

# Pull if missing
ollama pull mistral

---

Slow performance

# Check available RAM
free -h  # Linux
vm_stat  # macOS

# Close heavy apps
# Or use a smaller model (Q2 quantization)

---

✅ Summary

• Code identical to Module 2 (only the endpoint changes)
• Performance similar to LM Studio (same hardware)
• Ollama is better for automation/production
• LM Studio is better for beginners

---

Next: 06-docker-deployment.md

You'll learn to deploy Ollama in a Docker container.

Time: 40 min