7. Multimodal Troubleshooting

Description

Working with multimodal APIs brings errors that don't exist in text-only APIs: images with the wrong format, audio files that are too large, corrupt Base64 encoding, invisible token limits and costs that spike without warning. This capsule gives you a framework to diagnose, prevent and fix the most common errors.

Why it matters: In production, an unhandled error takes down the whole chain. If your system receives a TIFF image (unsupported), it fails silently. If an audio file exceeds 25MB, Whisper rejects it. If you don't control retries, a rate limit generates hundreds of failed requests. Knowing how to troubleshoot separates a fragile prototype from a robust system.

Connection with the module: Capsules 02-06 taught you to use vision, audio, combinations, the landscape and the formats. This capsule closes the loop: what to do when something fails. You'll use the patterns you learn here directly in the multimodal Classifier (capsule 08) and in every later module.

Category 1: Format and Encoding Errors

"Invalid image format"

The provider rejects the image because the format isn't supported (TIFF, BMP, HEIC) or the file is corrupt.

from PIL import Image
from pathlib import Path
import io

SUPPORTED_FORMATS = {".png", ".jpg", ".jpeg", ".gif", ".webp"}

def convert_image_format(image_path: str, target_format: str = "PNG") -> bytes:
    """Convert any image to a format supported by the APIs."""
    with Image.open(image_path) as img:
        if img.mode in ("RGBA", "LA", "P") and target_format == "JPEG":
            img = img.convert("RGB")
        buffer = io.BytesIO()
        img.save(buffer, format=target_format)
        return buffer.getvalue()

# Usage: convert an unsupported .tiff to PNG
png_bytes = convert_image_format("document.tiff", "PNG")

Base64 errors

Badly encoded Base64 produces cryptic errors like Invalid base64 or empty responses.

import base64

def safe_base64_encode(file_path: str) -> str:
    """Encode a file to Base64 with an integrity check."""
    path = Path(file_path)
    if not path.exists():
        raise FileNotFoundError(f"File not found: {file_path}")
    with open(path, "rb") as f:
        raw_bytes = f.read()
    if len(raw_bytes) == 0:
        raise ValueError("The file is empty")
    encoded = base64.b64encode(raw_bytes).decode("utf-8")
    decoded = base64.b64decode(encoded)
    assert len(decoded) == len(raw_bytes), "Length mismatch after decoding"
    return encoded

Incorrect MIME type

Sending image/png when the file is a JPEG causes silent rejections in Anthropic.

import mimetypes

MIME_MAP = {
    ".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg",
    ".gif": "image/gif", ".webp": "image/webp",
    ".mp3": "audio/mpeg", ".wav": "audio/wav", ".m4a": "audio/mp4",
}

def detect_mime_type(file_path: str) -> str:
    """Detect the file's real MIME type."""
    ext = Path(file_path).suffix.lower()
    if ext in MIME_MAP:
        return MIME_MAP[ext]
    mime, _ = mimetypes.guess_type(file_path)
    if mime:
        return mime
    raise ValueError(f"Could not determine the MIME type for: {file_path}")

Audio with an unsupported format

Whisper only accepts: mp3, mp4, mpeg, mpga, m4a, wav, webm.

from pydub import AudioSegment

WHISPER_FORMATS = {".mp3", ".mp4", ".mpeg", ".mpga", ".m4a", ".wav", ".webm"}

def convert_audio_for_whisper(audio_path: str, output_format: str = "mp3") -> str:
    """Convert audio to a Whisper-compatible format."""
    path = Path(audio_path)
    if path.suffix.lower() in WHISPER_FORMATS:
        return audio_path
    audio = AudioSegment.from_file(str(path))
    output_path = str(path.with_suffix(f".{output_format}"))
    audio.export(output_path, format=output_format)
    return output_path

Category 2: Size and Limit Errors

Image too large (>20MB)

OpenAI/Gemini reject images >20MB. Anthropic: >5MB.

from PIL import Image
import io

def resize_image_to_limit(image_path: str, max_size_mb: float = 4.0) -> bytes:
    """Progressively reduce quality and dimensions until it meets the limit."""
    with Image.open(image_path) as img:
        if img.mode != "RGB":
            img = img.convert("RGB")
        quality = 95
        current = img.copy()
        while True:
            buffer = io.BytesIO()
            current.save(buffer, format="JPEG", quality=quality, optimize=True)
            if len(buffer.getvalue()) / (1024 * 1024) <= max_size_mb:
                return buffer.getvalue()
            if quality > 40:
                quality -= 10
            else:
                w, h = current.size
                current = current.resize((w // 2, h // 2), Image.LANCZOS)
                quality = 85
            if current.size[0] < 100:
                raise ValueError("Could not reduce it without losing useful quality")

Audio too long (>25MB)

For long files, split into segments before transcribing.

from pydub import AudioSegment
import math
from openai import OpenAI

def split_audio(audio_path: str, max_size_mb: float = 24.0) -> list[str]:
    """Split audio into chunks that don't exceed Whisper's limit."""
    audio = AudioSegment.from_file(audio_path)
    file_size_mb = Path(audio_path).stat().st_size / (1024 * 1024)
    if file_size_mb <= max_size_mb:
        return [audio_path]
    num_chunks = math.ceil(file_size_mb / max_size_mb)
    chunk_ms = len(audio) // num_chunks
    paths = []
    for i in range(num_chunks):
        chunk = audio[i * chunk_ms : min((i + 1) * chunk_ms, len(audio))]
        path = f"./chunks/chunk_{i:03d}.mp3"
        Path("./chunks").mkdir(exist_ok=True)
        chunk.export(path, format="mp3")
        paths.append(path)
    return paths

def transcribe_long_audio(audio_path: str) -> str:
    """Transcribe long audio by splitting it into chunks."""
    client = OpenAI()
    chunks = split_audio(audio_path)
    texts = []
    for chunk_path in chunks:
        with open(chunk_path, "rb") as f:
            result = client.audio.transcriptions.create(
                model="whisper-1", file=f, language="en"
            )
            texts.append(result.text)
    return " ".join(texts)

Context window exceeded

When you send multiple images, you can exceed the context window without noticing.

import math

def estimate_image_tokens(width: int, height: int, detail: str = "auto") -> int:
    """Estimate an image's tokens in OpenAI (low=85 fixed, high=85 base + 170/tile)."""
    if detail == "low":
        return 85
    scale = min(2048 / max(width, height), 1.0)
    sw, sh = int(width * scale), int(height * scale)
    short = min(sw, sh)
    if short > 768:
        ratio = 768 / short
        sw, sh = int(sw * ratio), int(sh * ratio)
    return 85 + 170 * math.ceil(sw / 512) * math.ceil(sh / 512)

def check_context_budget(images: list[dict], text_tokens: int, max_ctx: int = 128_000) -> dict:
    """Check whether a request fits in the context window."""
    img_tokens = sum(estimate_image_tokens(i["w"], i["h"], i.get("detail", "auto")) for i in images)
    total = text_tokens + img_tokens
    return {"fits": total <= max_ctx, "total": total, "remaining": max_ctx - total}

Category 3: API and Network Errors

Rate limit → exponential backoff

import time
import random
from openai import OpenAI, RateLimitError, APITimeoutError, APIConnectionError

def safe_api_call(func, *args, max_retries: int = 5, base_delay: float = 1.0, **kwargs):
    """Wrapper with retry and exponential backoff for any API call."""
    last_error = None
    for attempt in range(max_retries):
        try:
            return func(*args, **kwargs)
        except RateLimitError as e:
            last_error = e
            delay = min(base_delay * (2 ** attempt), 60) + random.uniform(0, 1)
            print(f"  [Rate limit] Attempt {attempt+1}/{max_retries}, waiting {delay:.1f}s")
            time.sleep(delay)
        except APITimeoutError as e:
            last_error = e
            time.sleep(base_delay * (2 ** attempt))
        except APIConnectionError as e:
            last_error = e
            time.sleep(base_delay * 2)
        except Exception:
            raise
    raise last_error

# Usage
client = OpenAI()
response = safe_api_call(
    client.chat.completions.create,
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=100,
)

Authentication (401)

from openai import OpenAI, AuthenticationError

def validate_api_key(api_key: str | None = None) -> dict:
    """Validate that the OpenAI API key works."""
    try:
        client = OpenAI(api_key=api_key) if api_key else OpenAI()
        client.models.list()
        return {"valid": True, "error": None}
    except AuthenticationError as e:
        return {"valid": False, "error": f"Invalid API key: {e}"}
    except Exception as e:
        return {"valid": False, "error": f"Unexpected error: {e}"}

500 errors → fallback

from openai import InternalServerError

def call_with_fallback(primary_func, fallback_func, *args, **kwargs):
    """Try the primary; if it fails with a 500, use the fallback."""
    try:
        return safe_api_call(primary_func, *args, max_retries=3, **kwargs)
    except InternalServerError:
        print("  [Fallback] Primary server erroring. Using the fallback...")
        return safe_api_call(fallback_func, *args, max_retries=3, **kwargs)

Category 4: Quality Errors

Quality errors don't raise exceptions — the request works, but the result is bad.

Inaccurate OCR → preprocessing

from PIL import Image, ImageEnhance, ImageFilter
import io

def preprocess_for_ocr(image_path: str) -> bytes:
    """Preprocess an image to maximize OCR accuracy."""
    with Image.open(image_path) as img:
        img = img.convert("L")
        if img.width < 1000 or img.height < 1000:
            scale = max(1000 / img.width, 1000 / img.height)
            img = img.resize((int(img.width * scale), int(img.height * scale)), Image.LANCZOS)
        img = ImageEnhance.Contrast(img).enhance(2.0)
        img = img.filter(ImageFilter.SHARPEN)
        img = img.point(lambda x: 255 if x > 128 else 0)
        buffer = io.BytesIO()
        img.save(buffer, format="PNG")
        return buffer.getvalue()

Inaccurate transcription → language hints + prompt

from openai import OpenAI

def transcribe_with_hints(audio_path: str, language: str = "en", context: str = "") -> str:
    """Transcribe with language and context hints for greater accuracy.
    The prompt gives it the expected vocabulary: names, acronyms, technical terms.
    """
    client = OpenAI()
    with open(audio_path, "rb") as f:
        result = client.audio.transcriptions.create(
            model="whisper-1", file=f, language=language,
            prompt=context, temperature=0.0,
        )
    return result.text

# Usage
transcript = transcribe_with_hints(
    "meeting.mp3", language="en",
    context="Meeting about deploying FastAPI with Docker. Participants: María, Carlos.",
)

Incorrect classification → prompt + temperature=0

from openai import OpenAI
import base64

def classify_with_precision(image_path: str, categories: list[str]) -> dict:
    """Classify an image with a prompt optimized for consistency."""
    client = OpenAI()
    with open(image_path, "rb") as f:
        b64 = base64.b64encode(f.read()).decode()
    cats = ", ".join(categories)
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": [
            {"type": "text", "text": f"Classify this image into EXACTLY one category: {cats}. Reply with the name ONLY."},
            {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}},
        ]}],
        max_tokens=50, temperature=0,
    )
    result = response.choices[0].message.content.strip()
    return {"category": result, "valid": result in categories}

Hallucinations → ask for a confidence level

from openai import OpenAI
import base64, json

def extract_with_verification(image_path: str, fields: list[str]) -> dict:
    """Extract data asking for per-field confidence to detect hallucinations."""
    client = OpenAI()
    with open(image_path, "rb") as f:
        b64 = base64.b64encode(f.read()).decode()
    fields_str = ", ".join(fields)
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": [
            {"type": "text", "text": (
                f"Extract these fields from the image: {fields_str}.\n"
                f"Reply with JSON: {{\"field\": {{\"value\": \"...\", \"confidence\": \"high|medium|low\", "
                f"\"source\": \"visible|inferred|not_found\"}}}}.\n"
                f"If a field is NOT visible, value=null and confidence=\"low\". Do NOT make things up."
            )},
            {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}},
        ]}],
        max_tokens=1000, temperature=0,
    )
    raw = response.choices[0].message.content
    try:
        clean = raw.split("```json")[-1].split("```")[0] if "```" in raw else raw
        return json.loads(clean.strip())
    except json.JSONDecodeError:
        return {"raw_response": raw, "parse_error": True}

Category 5: Cost Errors

Cost tracker with alerts

import time
import functools

class CostTracker:
    """Track the accumulated cost of multimodal API calls."""
    PRICING = {
        "gpt-4o": {"input": 2.50, "output": 10.00},
        "gpt-4o-mini": {"input": 0.15, "output": 0.60},
        "whisper-1": {"per_minute": 0.006},
        "tts-1": {"per_1k_chars": 0.015},
        "dall-e-3": {"per_image": 0.040},
    }

    def __init__(self, budget_limit: float = 1.0):
        self.total_cost = 0.0
        self.budget_limit = budget_limit
        self.calls = []

    def log_call(self, model: str, **kwargs) -> float:
        cost = self._estimate(model, **kwargs)
        self.total_cost += cost
        self.calls.append({"model": model, "cost": cost, "time": time.time(), **kwargs})
        if self.total_cost > self.budget_limit:
            print(f"  ALERT: ${self.total_cost:.4f} exceeds the ${self.budget_limit:.2f} budget")
        return cost

    def _estimate(self, model: str, **kw) -> float:
        p = self.PRICING.get(model, {})
        if "input_tokens" in kw:
            return (kw["input_tokens"]/1e6)*p.get("input",0) + (kw.get("output_tokens",0)/1e6)*p.get("output",0)
        if "audio_minutes" in kw:
            return kw["audio_minutes"] * p.get("per_minute", 0)
        if "characters" in kw:
            return (kw["characters"]/1000) * p.get("per_1k_chars", 0)
        return kw.get("images_generated", 0) * p.get("per_image", 0)

    def summary(self) -> dict:
        return {"total": round(self.total_cost, 6), "calls": len(self.calls),
                "remaining": round(self.budget_limit - self.total_cost, 6)}

Monitoring decorator

def track_cost(model: str, tracker: CostTracker):
    """Decorator that logs the cost of each call automatically."""
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            result = func(*args, **kwargs)
            cost_kw = {}
            if hasattr(result, "usage"):
                cost_kw["input_tokens"] = result.usage.prompt_tokens
                cost_kw["output_tokens"] = result.usage.completion_tokens
            tracker.log_call(model, **cost_kw)
            return result
        return wrapper
    return decorator

# Usage
tracker = CostTracker(budget_limit=0.50)

@track_cost("gpt-4o", tracker)
def analyze_image(image_path: str):
    client = OpenAI()
    with open(image_path, "rb") as f:
        b64 = base64.b64encode(f.read()).decode()
    return client.chat.completions.create(
        model="gpt-4o", max_tokens=300,
        messages=[{"role": "user", "content": [
            {"type": "text", "text": "Describe this image."},
            {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}},
        ]}],
    )

Comprehensive Diagnostic Function

from pathlib import Path
from PIL import Image
import os

def diagnose_multimodal_input(file_path: str) -> dict:
    """Diagnose common problems before sending to an API."""
    path = Path(file_path)
    issues, suggestions = [], []

    if not path.exists():
        return {"valid": False, "issues": ["File not found"],
                "suggestions": [f"Check the path: {file_path}"]}

    size_mb = path.stat().st_size / (1024 * 1024)
    if size_mb == 0:
        issues.append("Empty file (0 bytes)")
    elif size_mb > 20:
        issues.append(f"File too large: {size_mb:.1f}MB")
        suggestions.append("Use resize_image_to_limit() or split_audio()")

    ext = path.suffix.lower()
    img_exts = {".png", ".jpg", ".jpeg", ".gif", ".webp"}
    audio_exts = {".mp3", ".mp4", ".mpeg", ".mpga", ".m4a", ".wav", ".webm"}

    if ext not in (img_exts | audio_exts):
        issues.append(f"Unsupported format: {ext}")
        suggestions.append("Convert to PNG/JPEG (image) or MP3/WAV (audio)")

    if ext in img_exts:
        try:
            with Image.open(path) as img:
                img.verify()
            with Image.open(path) as img:
                w, h = img.size
                if w < 50 or h < 50:
                    issues.append(f"Image too small: {w}x{h}")
        except Exception as e:
            issues.append(f"Corrupt image: {e}")
        if size_mb > 5:
            suggestions.append("For Anthropic, reduce to <5MB")

    if ext in audio_exts and size_mb > 25:
        issues.append(f"Audio exceeds Whisper's 25MB: {size_mb:.1f}MB")
        suggestions.append("Use split_audio() to split it into chunks")

    if not os.environ.get("OPENAI_API_KEY"):
        issues.append("OPENAI_API_KEY is not configured")

    return {"valid": len(issues) == 0, "size_mb": round(size_mb, 2),
            "format": ext, "issues": issues, "suggestions": suggestions}

Pipeline: Image Preprocessing by Provider

from PIL import Image
import base64, io
from pathlib import Path

def prepare_image_pipeline(image_path: str, provider: str = "openai") -> dict:
    """Full pipeline: validate → convert → resize → encode → payload."""
    path = Path(image_path)
    if not path.exists():
        raise FileNotFoundError(image_path)

    limits = {
        "openai": {"max_mb": 20, "fmts": {".png",".jpg",".jpeg",".gif",".webp"}},
        "anthropic": {"max_mb": 5, "fmts": {".png",".jpg",".jpeg",".gif",".webp"}},
        "google": {"max_mb": 20, "fmts": {".png",".jpg",".jpeg",".gif",".webp"}},
    }
    cfg = limits.get(provider, limits["openai"])

    ext = path.suffix.lower()
    if ext not in cfg["fmts"]:
        img_bytes = convert_image_format(image_path, "PNG")
        mime = "image/png"
    else:
        with open(path, "rb") as f:
            img_bytes = f.read()
        mime = detect_mime_type(image_path)

    if len(img_bytes) / (1024*1024) > cfg["max_mb"]:
        img_bytes = resize_image_to_limit(image_path, cfg["max_mb"] * 0.9)
        mime = "image/jpeg"

    b64 = base64.b64encode(img_bytes).decode()

    payloads = {
        "openai": {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{b64}"}},
        "anthropic": {"type": "image", "source": {"type": "base64", "media_type": mime, "data": b64}},
        "google": {"mime_type": mime, "data": b64},
    }
    return {"payload": payloads[provider], "provider": provider,
            "final_size_mb": round(len(img_bytes)/(1024*1024), 2), "mime": mime}

Quick Reference

Exercises

Exercise 1: Image validator (Easy)

Create validate_image(path) -> dict that checks: the file exists, the extension is supported (PNG, JPEG, GIF, WebP), size < 20MB, PIL can open it, dimensions >= 50x50. It returns valid, issues and metadata.

from PIL import Image
from pathlib import Path

def validate_image(path: str) -> dict:
    p = Path(path)
    issues, metadata = [], {}
    if not p.exists():
        return {"valid": False, "issues": ["File not found"], "metadata": {}}

    ext = p.suffix.lower()
    if ext not in {".png", ".jpg", ".jpeg", ".gif", ".webp"}:
        issues.append(f"Unsupported format: {ext}")

    size_mb = p.stat().st_size / (1024 * 1024)
    metadata["size_mb"] = round(size_mb, 2)
    if size_mb > 20:
        issues.append(f"Exceeds 20MB: {size_mb:.1f}MB")

    try:
        with Image.open(p) as img:
            img.verify()
        with Image.open(p) as img:
            w, h = img.size
            metadata.update({"width": w, "height": h, "format": img.format})
            if w < 50 or h < 50:
                issues.append(f"Too small: {w}x{h}")
    except Exception as e:
        issues.append(f"Could not open it: {e}")

    return {"valid": len(issues) == 0, "issues": issues, "metadata": metadata}

Exercise 2: Audio validator (Easy)

Create validate_audio(path) -> dict that checks: the file exists, the extension is Whisper-compatible, size < 25MB. If it exceeds it, include how many chunks would be needed.

from pathlib import Path
import math

def validate_audio(path: str) -> dict:
    p = Path(path)
    issues, metadata = [], {}
    if not p.exists():
        return {"valid": False, "issues": ["File not found"], "metadata": {}}

    ext = p.suffix.lower()
    if ext not in {".mp3", ".mp4", ".mpeg", ".mpga", ".m4a", ".wav", ".webm"}:
        issues.append(f"Format not compatible with Whisper: {ext}")

    size_mb = p.stat().st_size / (1024 * 1024)
    metadata["size_mb"] = round(size_mb, 2)
    metadata["format"] = ext

    if size_mb > 25:
        chunks = math.ceil(size_mb / 24)
        issues.append(f"Exceeds 25MB: {size_mb:.1f}MB")
        metadata["chunks_needed"] = chunks

    return {"valid": len(issues) == 0, "issues": issues, "metadata": metadata}

Exercise 3: API connectivity diagnosis (Medium)

Create test_api_connectivity() -> dict that tests connectivity with OpenAI for chat, vision and whisper. For each service it reports: available, latency_ms, error.

from openai import OpenAI
import time

def test_api_connectivity() -> dict:
    client = OpenAI()
    results = {}

    # Test Chat
    try:
        start = time.time()
        client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": "test"}],
            max_tokens=1,
        )
        results["chat"] = {"available": True, "latency_ms": round((time.time()-start)*1000)}
    except Exception as e:
        results["chat"] = {"available": False, "error": str(e)}

    # Test Models (checks the available models without consuming tokens)
    try:
        start = time.time()
        models = client.models.list()
        model_ids = [m.id for m in models.data]
        results["models"] = {
            "available": True,
            "latency_ms": round((time.time()-start)*1000),
            "vision": any("gpt-4o" in m for m in model_ids),
            "whisper": any("whisper" in m for m in model_ids),
        }
    except Exception as e:
        results["models"] = {"available": False, "error": str(e)}

    return results

Exercise 4: Universal preprocessing pipeline (Medium)

Create preprocess_image(path, provider) -> bytes that opens any image (including TIFF, BMP), converts it to JPEG, resizes it according to the provider's limit (openai: 20MB, anthropic: 5MB) and returns bytes ready for Base64.

from PIL import Image
import io

def preprocess_image(path: str, provider: str = "openai") -> bytes:
    limits = {"openai": 20, "anthropic": 5, "google": 20}
    target_mb = limits.get(provider, 20) * 0.85

    with Image.open(path) as img:
        if img.mode != "RGB":
            img = img.convert("RGB")
        quality = 95
        current = img.copy()
        while True:
            buffer = io.BytesIO()
            current.save(buffer, format="JPEG", quality=quality, optimize=True)
            if len(buffer.getvalue()) / (1024 * 1024) <= target_mb:
                return buffer.getvalue()
            if quality > 50:
                quality -= 10
            else:
                w, h = current.size
                current = current.resize((w*3//4, h*3//4), Image.LANCZOS)
                quality = 85
            if current.size[0] < 100:
                raise ValueError("Could not reduce it enough")

Exercise 5: Complete wrapper with logging, retry and fallback (Hard)

Build a RobustAPIClient class that: wraps OpenAI with automatic retry (exponential backoff), logs every call (timestamp, model, tokens, cost, error), has a configurable maximum budget, supports fallback (gpt-4o → gpt-4o-mini), and exposes analyze_image(path, prompt).

Hints:

• Use @dataclass for APILog with fields: timestamp, model, input_tokens, output_tokens, cost, error
• In _retry_call: catch RateLimitError, APITimeoutError, InternalServerError and retry with 2**attempt + random.uniform(0, 1)
• In analyze_image: if the primary fails after all the retries, try the fallback
• Before each call, check that self.spent < self.budget
• summary() returns a dict with calls, spent, remaining, errors

Summary

• Format errors are the #1 cause of failures. Validate format, MIME type and encoding before sending.
• Size errors have mechanical solutions: resize_image_to_limit() for images, split_audio() for audio.
• API errors require retry with exponential backoff. Never retry immediately.
• Quality errors don't raise exceptions. Fight inaccurate OCR with preprocessing, bad transcriptions with language hints, and hallucinations by asking for a confidence level.
• Cost errors are prevented with active monitoring. Use CostTracker and set budgets.
• diagnose_multimodal_input() is your first line of defense: run it before any call.
• safe_api_call() is your second line: wrap every call with retry, timeout and logging.

Additional Resources

1. OpenAI Error Codes Reference — Complete catalog of API errors with solutions.
2. OpenAI Rate Limits — Documentation on rate limits, tiers and strategies.
3. Pillow Documentation — Reference for image manipulation in Python.
4. pydub — Audio manipulation: conversion, split, compression.
5. OpenAI Vision Guide — Formats and limits for vision models.
6. Anthropic Vision Docs — Claude's specific limits for images.
7. Exponential Backoff (Google Cloud) — Exponential backoff pattern with jitter.
8. Counting Tokens (OpenAI Cookbook) — Estimate tokens and costs.