Ollama's REST API

Overview

Ollama includes an OpenAI-compatible REST API server. You'll learn to use it to integrate with Python/JavaScript.

Time: 25 minutes
Difficulty: Medium

---

🎯 Objectives

• ✅ Start the Ollama server
• ✅ Test with curl
• ✅ Integrate with the OpenAI SDK
• ✅ Understand the API differences

---

🚀 Start the Server

Background daemon (automatic):

Ollama starts the server automatically when you use ollama run.

Verify:

curl http://localhost:11434/api/tags

---

Manual start:

ollama serve

Output:

Ollama server running on http://localhost:11434

Default port: 11434 (vs LM Studio's 1234)

---

✅ Test with Curl

List models:

curl http://localhost:11434/api/tags

Output:

{
  "models": [
    {
      "name": "mistral:latest",
      "size": 4108916224
    }
  ]
}

---

Generate (completion):

curl http://localhost:11434/api/generate -d '{
  "model": "mistral",
  "prompt": "What is Python?"
}'

Output (streaming):

{"model":"mistral","response":"Python"}
{"model":"mistral","response":" is"}
{"model":"mistral","response":" a"}
...
{"done":true}

---

Chat (conversational):

curl http://localhost:11434/api/chat -d '{
  "model": "mistral",
  "messages": [
    {"role": "user", "content": "Hi"}
  ]
}'

---

💻 Integration with Python (OpenAI SDK)

Code (OpenAI-compatible):

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"  # Required but not validated
)

response = client.chat.completions.create(
    model="mistral",
    messages=[
        {"role": "user", "content": "Hi"}
    ]
)

print(response.choices[0].message.content)

Same code as Module 2, only base_url changes!

---

🔄 Migrate from LM Studio to Ollama

LM Studio:

client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="not-needed"
)

Ollama:

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"
)

Only change: Port (1234 → 11434)

---

📊 Ollama API vs OpenAI API

Feature	Ollama	OpenAI
Endpoint	localhost:11434	api.openai.com
Auth	Not required	API key required
Streaming	✅ Default	✅ Optional
Models	Local (mistral, llama2)	Cloud (gpt-3.5, gpt-4)
Cost	$0	$$$

---

⚙️ API Endpoints

1. Generate (completion):

POST /api/generate
Body: {"model": "mistral", "prompt": "..."}

2. Chat (conversational):

POST /api/chat
Body: {"model": "mistral", "messages": [...]}

3. List models:

GET /api/tags

4. Pull model:

POST /api/pull
Body: {"name": "mistral"}

5. Delete model:

DELETE /api/delete
Body: {"name": "mistral"}

---

🧪 Complete Test Script

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"
)

# Test 1: Simple chat
print("Test 1: Simple chat")
response = client.chat.completions.create(
    model="mistral",
    messages=[{"role": "user", "content": "Hi"}]
)
print(response.choices[0].message.content)

# Test 2: Conversation with context
print("\nTest 2: With context")
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "My name is Juan"},
    {"role": "assistant", "content": "Hi Juan"},
    {"role": "user", "content": "What is my name?"}
]
response = client.chat.completions.create(
    model="mistral",
    messages=messages
)
print(response.choices[0].message.content)  # "Your name is Juan"

---

🐛 Troubleshooting

Error: Connection refused

Cause: The server isn't running

Solution:

ollama serve
# Or
ollama run mistral  # Auto-starts the server

---

Error: Model not found

Cause: The model isn't downloaded

Solution:

ollama pull mistral
ollama list  # Verify

---

✅ Summary

• API server at localhost:11434
• Compatible with the OpenAI SDK (minimal change)
• Streaming by default
• No API key needed
• Reuses code from Modules 2-3

---

Next: 05-python-integration.md

You'll build a complete chatbot with Ollama + Python.

Time: 30 min