6. Debugging Local Deployment

Overview

In this capsule you'll master the debugging flow when something fails in your Docker Compose. Knowing how to bring up services is half of it; knowing how to diagnose when they don't start or fail is the other half. By the end, you'll have a systematic process to solve the most common problems in local deployment of AI apps.

Context: The student who doesn't know how to debug Docker Compose gets stuck on the first error and searches StackOverflow for 30 minutes. The one who has a systematic flow solves it in 5 minutes. This capsule gives you that flow.

The Systematic Debugging Flow

The 5 steps

1. Which service failed?       → docker compose ps
2. What do the logs say?       → docker compose logs <service>
3. Can I get into the container? → docker compose exec <service> sh
4. Does the network work?        → ping/curl from inside
5. Is the config correct?      → docker compose config

Step 1: Service status

docker compose ps
# NAME           SERVICE   STATUS          PORTS
# mod02-api-1    api       Up (healthy)    0.0.0.0:8000->8000/tcp
# mod02-cache-1  cache     Up (healthy)    6379/tcp
# mod02-worker-1 worker    Exited (1)

# The worker went down. Next: check logs.

Pay attention to the STATUS column. The key states:

Step 2: Logs

# Logs of a specific service
docker compose logs worker
# [ERROR] redis.ConnectionError: Connection refused

# Logs with follow (in real time)
docker compose logs -f api

# Last 50 lines
docker compose logs --tail 50 api

# Logs of all services
docker compose logs

# Logs with timestamps (useful to correlate between services)
docker compose logs -t api cache

# Logs from a specific moment
docker compose logs --since 5m api
# Last 5 minutes

# Logs between two moments
docker compose logs --since 2024-01-15T10:00:00 --until 2024-01-15T10:05:00 api

Step 3: Exec (get into the container)

# Interactive shell
docker compose exec api sh
# or bash if available:
docker compose exec api bash

# Run a one-off command
docker compose exec api python -c "import redis; r = redis.Redis.from_url('redis://cache:6379'); print(r.ping())"

# Verify environment variables
docker compose exec api env | grep REDIS

# Verify that the files are where you expect
docker compose exec api ls -la /app/

# Verify the Python version and installed packages
docker compose exec api python --version
docker compose exec api pip list

# Run a quick debug script
docker compose exec api python -c "
from config import settings
print(f'Environment: {settings.environment}')
print(f'Redis URL: {settings.redis_url}')
print(f'Model: {settings.model_name}')
"

Step 4: Networking

# From inside the api container, verify connectivity
docker compose exec api ping -c 3 cache
docker compose exec api curl -s http://localhost:8000/health

# See open ports
docker compose exec api netstat -tlnp 2>/dev/null || ss -tlnp

# Verify DNS resolution inside the Compose network
docker compose exec api nslookup cache 2>/dev/null || \
  docker compose exec api getent hosts cache

# Verify that Redis responds from the API container
docker compose exec api redis-cli -h cache -p 6379 ping

# List the Compose networks
docker network ls | grep module-02

# Inspect which containers are on the network
docker network inspect module-02_default

Step 5: Verify config

# See the merged config (with substituted variables)
docker compose config

# See only a service
docker compose config --services

# Verify that the environment variables resolved correctly
docker compose config | grep -A 5 "environment:"

# Validate that the compose file has no syntax errors
docker compose config --quiet && echo "✅ Valid config" || echo "❌ Invalid config"

# See the config with a specific override
docker compose -f docker-compose.yml -f docker-compose.prod.yml config

Comparison: logs vs inspect vs exec

You have three main tools to get information from a container. Each has a distinct use case:

When to use each one — practical examples

# SCENARIO: "The API doesn't start"
# Step 1: Is there an error in the logs?
docker compose logs --tail 30 api
# → If you see "ModuleNotFoundError" → missing dependency
# → If you see "Connection refused" → dependency not ready

# SCENARIO: "The API starts but doesn't connect to Redis"
# Step 1: Is Redis running?
docker compose ps cache
# Step 2: Is the URL correct?
docker inspect module-02-api-1 --format='{{range .Config.Env}}{{println .}}{{end}}' | grep REDIS
# → REDIS_URL=redis://cache:6379
# Step 3: Can I connect from inside?
docker compose exec api redis-cli -h cache ping

# SCENARIO: "The container restarts nonstop"
# Step 1: How many restarts so far?
docker inspect module-02-api-1 --format='{{.RestartCount}}'
# → 15 (many restarts)
# Step 2: What does the log say before it dies?
docker compose logs --tail 5 api
# Step 3: Stop the restart to investigate
docker compose stop api

docker inspect — the most useful fields

# Container state (running, exited, restarting)
docker inspect --format='{{.State.Status}}' module-02-api-1

# Exit code of the last crash
docker inspect --format='{{.State.ExitCode}}' module-02-api-1

# How many restarts it has had
docker inspect --format='{{.RestartCount}}' module-02-api-1

# Container IP on the Compose network
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' module-02-api-1

# Mounts (mounted volumes)
docker inspect --format='{{json .Mounts}}' module-02-api-1 | python -m json.tool

# Timestamp of the last start
docker inspect --format='{{.State.StartedAt}}' module-02-api-1

Resource Monitoring with docker stats

In AI apps, resource monitoring is critical. A model that loads embeddings consumes gigabytes. A long request to OpenAI can keep connections open. docker stats is your real-time resource dashboard.

Basic usage

# Real-time monitoring (updates every second)
docker stats
# CONTAINER       CPU %   MEM USAGE / LIMIT     MEM %   NET I/O         BLOCK I/O
# mod02-api-1     2.5%    145MiB / 512MiB        28%     1.2kB / 800B    0B / 0B
# mod02-cache-1   0.1%    12MiB / 128MiB         9%      500B / 300B     0B / 4kB

# Snapshot (no live update)
docker stats --no-stream

# Custom format
docker stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.MemPerc}}"

# Only one container
docker stats module-02-api-1

What to watch in AI apps

METRIC           NORMAL (AI app)     ALERT               CRITICAL
CPU %            1-20% (idle)        >50% sustained      >90%
                 30-80% (request)
MEM USAGE        100-300MiB          >80% of the limit   Near the limit
NET I/O          Variable            Growth without      Saturation
                                     requests

Continuous monitoring script

#!/bin/bash
# monitor.sh — Records stats every 5 seconds

LOG_FILE="docker-stats-$(date +%Y%m%d-%H%M%S).csv"
echo "timestamp,container,cpu,mem_usage,mem_limit,mem_pct" > "$LOG_FILE"

while true; do
  docker stats --no-stream --format "$(date +%s),{{.Name}},{{.CPUPerc}},{{.MemUsage}},{{.MemPerc}}" \
    >> "$LOG_FILE"
  sleep 5
done

Set memory limits in Compose

services:
  api:
    deploy:
      resources:
        limits:
          memory: 512M
          cpus: "1.0"
        reservations:
          memory: 256M
          cpus: "0.5"

When a container exceeds the memory limit, Docker kills it with signal 9 (exit code 137). In AI apps this is common when:

1. You load a large model in memory — If your app loads embeddings or a local model, it needs more RAM
2. You accumulate responses without releasing them — If you store all responses in an in-memory list
3. FastAPI with many workers — Each worker is a separate process with its own memory

Common Problems in AI Apps

Problem 1: ImportError in the API

api-1  | ImportError: No module named 'openai'

# Cause: requirements.txt doesn't include the dependency
# or the Dockerfile doesn't run pip install

# Solution: Verify requirements.txt and the Dockerfile
docker compose exec api pip list | grep openai

# If missing, add it to requirements.txt and rebuild:
docker compose build api
docker compose up -d

Problem 2: Redis ConnectionError

api-1  | redis.exceptions.ConnectionError: Error while connecting to redis://cache:6379

# Verify that Redis is running
docker compose ps cache
# If it's "Exited", see logs:
docker compose logs cache

# Verify that the URL is correct
docker compose exec api env | grep REDIS
# Should be: REDIS_URL=redis://cache:6379

# Verify connectivity
docker compose exec api redis-cli -h cache ping
# PONG = OK

Problem 3: Out of Memory (OOM)

api-1  | Killed
# or
api-1 exited with code 137  # 137 = killed by OOM

# Check memory usage
docker stats
# CONTAINER   CPU %   MEM USAGE / LIMIT
# api-1       2.5%    450MiB / 512MiB    ← Near the limit

# Solutions:
# 1. Increase the memory limit in compose
# 2. Reduce the app's memory usage
# 3. Don't load large models in memory

Problem 4: OpenAI API timeout

api-1  | openai.APITimeoutError: Request timed out

# Increase the timeout in the client
client = OpenAI(
    api_key=os.environ["OPENAI_API_KEY"],
    timeout=60.0,  # Default is 10s, increase for long prompts
    max_retries=3,
)

Problem 5: Container restarts in a loop

docker compose ps
# api-1  Restarting (1) 5 seconds ago

# See the last logs before the crash
docker compose logs --tail 20 api
# Look for the error causing the crash

# Stop the restarts to investigate
docker compose stop api
docker compose logs api

AI-Specific Debugging: Problems That Only Happen in AI Apps

AI apps have problems that don't exist in traditional web apps. This section covers the most common ones.

OOM Kills from Model Loading

When your app loads a local model (embeddings, sentence-transformers, etc.), memory consumption spikes on startup:

# Typical pattern in logs
api-1  | Loading model sentence-transformers/all-MiniLM-L6-v2...
api-1  | Killed

# The container dies before finishing loading the model
docker inspect --format='{{.State.ExitCode}}' module-02-api-1
# 137 = OOM kill

Diagnosis:

# See how much memory your app needs on startup
docker stats --no-stream
# Before loading the model: 80MiB
# After: 450MiB
# Limit: 512MiB → leaves no room for requests

# Solution 1: Increase the limit
# In docker-compose.yml:
# deploy.resources.limits.memory: 1G

# Solution 2: Use smaller models
# all-MiniLM-L6-v2 (~80MB) vs all-mpnet-base-v2 (~420MB)

# Solution 3: Lazy loading (load on the first request, not on startup)

# Lazy loading pattern
class ModelManager:
    def __init__(self):
        self._model = None

    @property
    def model(self):
        if self._model is None:
            from sentence_transformers import SentenceTransformer
            self._model = SentenceTransformer("all-MiniLM-L6-v2")
        return self._model

model_manager = ModelManager()

LLM API Timeouts and Retries

LLMs are slow compared to traditional APIs. A request to GPT-4 can take 10-30 seconds:

import time
import logging
from openai import OpenAI, APITimeoutError, RateLimitError

logger = logging.getLogger(__name__)

client = OpenAI(
    api_key=os.environ["OPENAI_API_KEY"],
    timeout=60.0,
    max_retries=3,
)

def ask_llm_with_retry(prompt: str, max_retries: int = 3) -> str:
    for attempt in range(max_retries):
        try:
            start = time.time()
            response = client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[{"role": "user", "content": prompt}],
                max_tokens=500,
            )
            elapsed = time.time() - start
            logger.info(f"LLM response in {elapsed:.1f}s (attempt {attempt + 1})")
            return response.choices[0].message.content

        except APITimeoutError:
            logger.warning(f"Timeout on attempt {attempt + 1}/{max_retries}")
            if attempt == max_retries - 1:
                raise
            time.sleep(2 ** attempt)

        except RateLimitError:
            wait = 2 ** (attempt + 2)
            logger.warning(f"Rate limited, waiting {wait}s")
            time.sleep(wait)

Slow Cold Starts

The first request after docker compose up takes much longer than the following ones:

# Measure the cold start
time curl -s -X POST http://localhost:8000/ask \
  -H "Content-Type: application/json" \
  -d '{"prompt":"test"}'
# real    0m8.234s ← first request (cold)

time curl -s -X POST http://localhost:8000/ask \
  -H "Content-Type: application/json" \
  -d '{"prompt":"test"}'
# real    0m0.045s ← second request (cached)

Causes and solutions:

# Warm-up pattern: run at startup
from contextlib import asynccontextmanager
from fastapi import FastAPI

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Warm up connections
    try:
        cache.ping()
        logger.info("Redis connection warm")
    except Exception:
        logger.warning("Redis not available at startup")
    yield

app = FastAPI(lifespan=lifespan)

Debugging Tools

docker compose exec vs docker compose run

# exec: gets into a container that is ALREADY running
docker compose exec api sh

# run: creates a new temporary container
docker compose run --rm api python -c "print('test')"

# Key difference:
# exec → same container, same state, same connections
# run → new, clean container, no mapped ports

Inspect a container

# See all the container's config
docker inspect module-02-api-1

# See only the environment variables
docker inspect --format='{{range .Config.Env}}{{println .}}{{end}}' module-02-api-1

# See the mounts
docker inspect --format='{{json .Mounts}}' module-02-api-1 | python -m json.tool

Clean rebuild

# If you suspect the build is corrupted:
docker compose down
docker compose build --no-cache
docker compose up -d

docker compose events — see events in real time

# Monitor events of all containers
docker compose events

# Typical events you'll see:
# container start, container die, container health_status
# network connect, volume mount

# Useful for debugging restart loops:
docker compose events --filter event=die
# Shows you every time a container dies

Structured logs with JSON

If you configure your app to log in JSON, you can filter more easily:

import logging
import json

class JSONFormatter(logging.Formatter):
    def format(self, record):
        log_entry = {
            "timestamp": self.formatTime(record),
            "level": record.levelname,
            "message": record.getMessage(),
            "module": record.module,
        }
        if record.exc_info:
            log_entry["exception"] = self.formatException(record.exc_info)
        return json.dumps(log_entry)

handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logger = logging.getLogger("ai-api")
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)

# Filter JSON logs by error level
docker compose logs api 2>&1 | python -c "
import sys, json
for line in sys.stdin:
    try:
        entry = json.loads(line.split('| ', 1)[-1])
        if entry.get('level') == 'ERROR':
            print(json.dumps(entry, indent=2))
    except (json.JSONDecodeError, IndexError):
        pass
"

Hands-On Exercises

Exercise 1: Diagnose a simulated failure

Stop Redis while the API is running. What happens? How do you diagnose it?

# Stop Redis
docker compose stop cache

# Verify status
docker compose ps
# cache: Exited, api: Up (but healthy?)

# Make a request
curl http://localhost:8000/ask -X POST \
  -H "Content-Type: application/json" \
  -d '{"prompt":"test"}'
# If you implemented graceful degradation: it works without cache
# If not: it returns a connection error

# Verify health
curl http://localhost:8000/health
# {"status":"degraded","services":{"api":"up","redis":"down"}}

# Restore
docker compose start cache

Exercise 2: Debug a memory leak

Use docker stats to monitor your API's memory usage over 10 requests. Does the memory grow?

# Terminal 1: monitor stats
docker stats module-02-api-1

# Terminal 2: send requests
for i in $(seq 1 10); do
  curl -s -X POST http://localhost:8000/ask \
    -H "Content-Type: application/json" \
    -d '{"prompt":"Test request '$i'"}' > /dev/null
  echo "Request $i sent"
done

# Watch in Terminal 1 whether MEM USAGE grows significantly
# If it grows and doesn't come down: possible memory leak
# If it stabilizes: normal (caching, object pools)

Exercise 3: Diagnostic script

Create a debug.sh script that runs the 5 debugging steps automatically.

#!/bin/bash
echo "=== 1. Service Status ==="
docker compose ps

echo -e "\n=== 2. Recent Logs (last 10 lines per service) ==="
for service in $(docker compose config --services); do
  echo "--- $service ---"
  docker compose logs --tail 10 "$service" 2>&1
done

echo -e "\n=== 3. Health Checks ==="
curl -s http://localhost:8000/health | python -m json.tool 2>/dev/null || echo "API not responding"

echo -e "\n=== 4. Resource Usage ==="
docker stats --no-stream

echo -e "\n=== 5. Config Validation ==="
docker compose config --quiet && echo "Config OK" || echo "Config ERROR"

Exercise 4: Diagnose an exit code 137

Configure a very low memory limit (64MB) for the api service and make a request that loads a lot in memory. Diagnose the crash and fix it.

# docker-compose.debug.yml — override with very low memory
services:
  api:
    deploy:
      resources:
        limits:
          memory: 64M

# Bring it up with the restrictive override
docker compose -f docker-compose.yml -f docker-compose.debug.yml up -d

# Verify it started
docker compose ps
# api   Up (healthy) — it may start with 64MB

# Make a heavy request
curl -X POST http://localhost:8000/ask \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Write an extensive essay about the history of artificial intelligence from the 1950s to today", "max_tokens": 2000}'

# Verify whether it died
docker compose ps
# If STATUS is "Exited (137)" → OOM kill

# Confirm with inspect
docker inspect --format='{{.State.ExitCode}}' module-02-api-1
# 137

# Check how much memory it was using before the kill
docker compose logs --tail 5 api
# There may be no log (killed abruptly)

# Solution: increase the limit
# Change 64M → 512M in the override
# Or remove the override: docker compose up -d (uses the base config)

# Verify the fix
docker compose up -d
docker stats --no-stream
# MEM USAGE should have enough headroom

Exercise 5: Correlating logs between services

Your API reports "Redis connection error" but Redis says it's UP. Use logs with timestamps to diagnose the real problem.

# See logs with timestamps from both services
docker compose logs -t api cache 2>&1 | sort -t 'Z' -k1

# Example output:
# 2024-01-15T10:00:01Z cache-1 | Ready to accept connections
# 2024-01-15T10:00:01Z api-1   | Starting uvicorn...
# 2024-01-15T10:00:02Z api-1   | redis.ConnectionError: Connection refused
# 2024-01-15T10:00:05Z cache-1 | Ready to accept connections on port 6379

# The problem: api tried to connect BEFORE Redis was ready
# Even though Redis "started", it wasn't accepting connections yet

# Solution: depends_on with condition: service_healthy
# (Already implemented in our compose, but if you didn't have it)

# Verify that Redis's health check works
docker compose exec cache redis-cli ping
# PONG

# Verify that depends_on is well configured
docker compose config | grep -A 5 "depends_on"
#   depends_on:
#     cache:
#       condition: service_healthy

# If the timing is still a problem, add a retry in the app:

import time
import redis

def get_redis_connection(url: str, max_retries: int = 5) -> redis.Redis:
    for attempt in range(max_retries):
        try:
            r = redis.Redis.from_url(url, decode_responses=True)
            r.ping()
            return r
        except redis.ConnectionError:
            if attempt < max_retries - 1:
                wait = 2 ** attempt
                print(f"Redis not ready, retrying in {wait}s...")
                time.sleep(wait)
            else:
                raise

Troubleshooting

"docker compose logs shows nothing"

If a container dies immediately, there may be no logs. Use docker compose run to see the error:

docker compose run --rm api python -c "from config import settings; print('OK')"

If the error is in the settings import (like a missing variable), you'll see it here.

"Container Exited (2) — syntax error"

Exit code 2 is usually a bash/shell error. Check the CMD/ENTRYPOINT in your Dockerfile:

docker compose logs api
# /bin/sh: uvicorn: not found

# Solution: pip install didn't run, rebuild:
docker compose build --no-cache api

"Port already in use"

# Error: bind: address already in use
# Someone else is using port 8000

# Find which process uses the port
lsof -i :8000
# or
netstat -tlnp | grep 8000

# Solution: change the port in .env or kill the process
# In .env: API_PORT=8001

"Container can't resolve another service's hostname"

# Error: Could not resolve host: cache
# The containers are not on the same network

docker network ls | grep module-02
# If there's no network → the containers are not connected

# Solution: verify that both services are in the same compose file
docker compose config --services
# Should list: api, cache

"The health check always fails but the app works"

# The health check uses curl, but curl isn't installed in the image
docker compose exec api curl --version
# sh: curl: not found

# Solution: install curl in the Dockerfile
# RUN apt-get update && apt-get install -y curl

# Or use a health check that doesn't require curl:
# test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]

Summary

• Systematic debugging in 5 steps: ps → logs → exec → networking → config.
• The most common problems in AI apps: ImportError, ConnectionError, OOM, LLM API timeout, restart loops.
• docker compose logs -f <service> is your main tool.
• docker compose exec <service> sh gives you direct access to the container.
• docker stats monitors CPU and memory in real time — critical for AI apps with heavy models.
• docker inspect reveals internal config: env vars, IPs, restart count, exit codes.
• OOM kills (exit code 137) are the #1 problem in AI apps — always monitor memory.
• LLM timeouts need retry with exponential backoff — LLMs are slow.
• Clean rebuild (build --no-cache) when you suspect build corruption.

Additional Resources

1. Docker Compose CLI Reference — All the Compose commands
2. Docker Logs — Logging configuration
3. Docker Stats — Resource monitoring
4. Debugging Docker Containers — Metrics and debugging
5. FastAPI Debugging — Debugging tips in FastAPI
6. Python Logging Best Practices — Configure logging in Python
7. Docker Events — Real-time event monitoring