Ollama Docker Deployment

Overview

You'll learn to run Ollama in a Docker container for deployment on servers.

Time: 40 minutes
Difficulty: Medium-High

---

🎯 Objectives

• ✅ Run Ollama in Docker
• ✅ Persistent storage (models)
• ✅ Docker Compose setup
• ✅ Network configuration

---

📦 Prerequisite: Docker

Install Docker:

macOS/Windows:

• Docker Desktop: https://www.docker.com/products/docker-desktop

Linux:

curl -fsSL https://get.docker.com | sh

Verify:

docker --version

---

🚀 Run Ollama in Docker

Method 1: Docker run (simple):

docker run -d \
  -v ollama:/root/.ollama \
  -p 11434:11434 \
  --name ollama \
  ollama/ollama

Explanation:

• -d: Background
• -v ollama:/root/.ollama: Persistent storage (models)
• -p 11434:11434: Exposes the port
• --name ollama: Container name

---

Pull a model inside the container:

docker exec -it ollama ollama pull mistral

---

Test the API:

curl http://localhost:11434/api/tags

---

🐳 Method 2: Docker Compose (recommended)

docker-compose.yml:

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    restart: unless-stopped
    environment:
      - OLLAMA_HOST=0.0.0.0

volumes:
  ollama_data:

Start:

docker-compose up -d

Stop:

docker-compose down

Logs:

docker-compose logs -f ollama

---

🖥️ With GPU (NVIDIA)

Dockerfile with CUDA:

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

volumes:
  ollama_data:

Prerequisite: NVIDIA Container Toolkit

---

📦 Complete Stack: Ollama + App

docker-compose.yml (Ollama + Python app):

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    restart: unless-stopped

  chatbot:
    build: .
    container_name: chatbot
    depends_on:
      - ollama
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    ports:
      - "8000:8000"

volumes:
  ollama_data:

Dockerfile (Python app):

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY chatbot.py .

CMD ["python", "chatbot.py"]

requirements.txt:

openai==1.12.0

chatbot.py:

from openai import OpenAI
import os

client = OpenAI(
    base_url=os.getenv("OLLAMA_BASE_URL", "http://localhost:11434") + "/v1",
    api_key="ollama"
)

# Your chatbot code here

Start the stack:

docker-compose up -d

---

🔧 Management Commands

Pull a model:

docker exec ollama ollama pull mistral

List models:

docker exec ollama ollama list

Logs:

docker logs ollama -f

Restart:

docker restart ollama

Interactive exec:

docker exec -it ollama bash

---

📊 Production Best Practices

1. Health checks:

services:
  ollama:
    # ... config ...
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
      interval: 30s
      timeout: 10s
      retries: 3

---

2. Resource limits:

services:
  ollama:
    # ... config ...
    deploy:
      resources:
        limits:
          memory: 16G
          cpus: '4'

---

3. Logging:

services:
  ollama:
    # ... config ...
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

---

🐛 Troubleshooting

Error: Cannot connect

# Check that the container is running
docker ps

# Check the logs
docker logs ollama

---

Error: Out of memory

# Increase the memory limit
docker update --memory 16g ollama

---

✅ Summary

• Ollama in Docker: Production-ready
• Persistent storage with volumes
• Docker Compose: Multi-container
• GPU support (optional)
• Health checks and resource limits

---

Next: 07-performance-tuning.md

You'll optimize Ollama's performance (GPU, RAM, quantization).

Time: 20 min