4. Document Images

Description

When a document is a scanned PDF or an image (photo of an invoice, screenshot), you need to convert the pages to images and send them to Vision APIs. In this capsule you'll learn to prepare document images for analysis: conversion with different libraries, optimization to reduce tokens and cost, sending to multiple providers, and intelligent processing of documents with many pages.

Why it matters: Vision APIs require images in a specific format (Base64, URL). A 50-page PDF sent without optimization can cost 10x more than necessary. Mastering the preparation avoids errors, optimizes costs and lets you scale to real production volumes.

Connection with the module: In capsule 02 you learned to extract text and images from PDFs with PyMuPDF. In capsule 03 you saw traditional OCR vs Vision APIs. Here you combine both: you convert complete pages to images and send them to Vision to get full understanding of the document — layout, tables, images and text together.

Key Concepts

Formats accepted by Vision APIs

Limits per provider

How Vision APIs process images

Vision APIs don't receive the image as-is. Internally they resize it and split it into tiles. OpenAI with detail=high:

1. Scales the image to fit within 2048×2048
2. Scales it so the shorter side is 768px
3. Splits it into 512×512 tiles
4. Each tile = 170 tokens, plus 85 base tokens

A 1024×1024 image consumes ~765 tokens. A 4096×4096 one consumes the same as a 2048×2048 (because of step 1).

Conversion Methods: PDF to Images

Method 1: PyMuPDF (fitz) — direct pixmap

PyMuPDF generates native pixmaps with no external system dependencies. It's the most portable option.

import fitz
import io
import base64
from PIL import Image

def pdf_to_images_pymupdf(pdf_path: str, dpi: int = 150) -> list[Image.Image]:
    """Converts a PDF to a list of PIL images using PyMuPDF."""
    doc = fitz.open(pdf_path)
    images = []
    zoom = dpi / 72
    mat = fitz.Matrix(zoom, zoom)

    for page in doc:
        pix = page.get_pixmap(matrix=mat, alpha=False)
        img = Image.open(io.BytesIO(pix.tobytes("png")))
        images.append(img)

    doc.close()
    return images

Method 2: pdf2image (Poppler)

pdf2image uses Poppler as the backend. It produces high-quality images and offers advanced rendering options.

from pdf2image import convert_from_path

def pdf_to_images_poppler(pdf_path: str, dpi: int = 150) -> list[Image.Image]:
    """Converts a PDF to images using pdf2image (requires Poppler)."""
    return convert_from_path(
        pdf_path,
        dpi=dpi,
        fmt="png",
        thread_count=4,
        grayscale=False,
        size=None  # None = uses DPI, or (width, height) to force a size
    )

def pdf_specific_pages(pdf_path: str, first: int, last: int, dpi: int = 150) -> list[Image.Image]:
    """Converts only a range of pages (1-indexed)."""
    return convert_from_path(pdf_path, dpi=dpi, first_page=first, last_page=last)

Handling different page sizes

PDFs can have pages of mixed sizes (letter, legal, A4). It's important to normalize.

def normalize_page_sizes(images: list[Image.Image], target_width: int = 1024) -> list[Image.Image]:
    """Normalizes images to a consistent width while keeping proportions."""
    normalized = []
    for img in images:
        w, h = img.size
        if w != target_width:
            ratio = target_width / w
            img = img.resize((target_width, int(h * ratio)), Image.Resampling.LANCZOS)
        normalized.append(img)
    return normalized

Method comparison

DPI and its impact

Recommendation: 150 DPI is the sweet spot for Vision APIs. Above 200 DPI it doesn't improve the model's understanding and multiplies the token cost.

Image Optimization for Vision

Resize for optimal token use

OpenAI resizes internally to a max of 2048×2048. Sending larger images wastes bandwidth with no benefit.

def optimize_for_vision(
    img: Image.Image,
    max_side: int = 2048,
    jpeg_quality: int = 85
) -> bytes:
    """Resizes if it exceeds max_side and compresses to JPEG."""
    w, h = img.size
    if max(w, h) > max_side:
        ratio = max_side / max(w, h)
        img = img.resize((int(w * ratio), int(h * ratio)), Image.Resampling.LANCZOS)

    if img.mode == "RGBA":
        img = img.convert("RGB")

    buf = io.BytesIO()
    img.save(buf, format="JPEG", quality=jpeg_quality, optimize=True)
    return buf.getvalue()

JPEG quality vs size: the tradeoff

def compare_jpeg_quality(img: Image.Image) -> dict:
    """Shows the impact of quality on size."""
    if img.mode == "RGBA":
        img = img.convert("RGB")
    results = {}
    for quality in [50, 65, 75, 85, 95]:
        buf = io.BytesIO()
        img.save(buf, format="JPEG", quality=quality, optimize=True)
        results[quality] = f"{len(buf.getvalue()) / 1024:.0f} KB"
    return results

# Typical result for a document page at 150 DPI:
# {50: "80 KB", 65: "120 KB", 75: "160 KB", 85: "250 KB", 95: "600 KB"}

Key finding: Quality 75-85 is the sweet spot. Below 65 the text gets blurry. Above 90 the size grows exponentially with no perceptible improvement.

Calculate the optimal DPI for the document

def calculate_optimal_dpi(
    page_width_inches: float,
    page_height_inches: float,
    target_pixels: int = 1500
) -> int:
    """Calculates the DPI that produces an image close to the target on its longer side."""
    longer_side = max(page_width_inches, page_height_inches)
    ideal_dpi = int(target_pixels / longer_side)
    return max(100, min(ideal_dpi, 250))

# Letter (8.5×11"): 136 → 150 | A4 (8.27×11.69"): 128 → 150 | Legal (8.5×14"): 107

Complete preparation function

def prepare_page_for_vision(
    img: Image.Image,
    max_side: int = 2048,
    jpeg_quality: int = 80,
    max_bytes: int = 5 * 1024 * 1024
) -> dict:
    """Prepares a page image with progressive optimization."""
    img_bytes = optimize_for_vision(img, max_side, jpeg_quality)

    while len(img_bytes) > max_bytes and jpeg_quality > 40:
        jpeg_quality -= 10
        img_bytes = optimize_for_vision(img, max_side, jpeg_quality)

    if len(img_bytes) > max_bytes:
        max_side = int(max_side * 0.7)
        img_bytes = optimize_for_vision(img, max_side, jpeg_quality)

    b64 = base64.b64encode(img_bytes).decode()
    return {
        "base64": b64,
        "media_type": "image/jpeg",
        "size_bytes": len(img_bytes),
        "quality_used": jpeg_quality
    }

Sending to Multiple Providers

Each provider expects the image in a slightly different format. Reference: Module 1, capsule 06.

OpenAI: data URI

from openai import OpenAI

def send_to_openai(pages: list[dict], prompt: str, detail: str = "high") -> str:
    """Sends page images to OpenAI Vision."""
    client = OpenAI()
    content = [{"type": "text", "text": prompt}]
    for page in pages[:10]:
        content.append({
            "type": "image_url",
            "image_url": {
                "url": f"data:{page['media_type']};base64,{page['base64']}",
                "detail": detail
            }
        })

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": content}],
        max_tokens=4000
    )
    return response.choices[0].message.content

Anthropic: Base64 + explicit media_type

import anthropic

def send_to_anthropic(pages: list[dict], prompt: str) -> str:
    """Sends page images to Claude Vision."""
    client = anthropic.Anthropic()
    content = []
    for page in pages[:5]:
        content.append({
            "type": "image",
            "source": {
                "type": "base64",
                "media_type": page["media_type"],
                "data": page["base64"]
            }
        })
    content.append({"type": "text", "text": prompt})

    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=4000,
        messages=[{"role": "user", "content": content}]
    )
    return response.content[0].text

Google Gemini: direct PIL objects

import google.generativeai as genai

def send_to_gemini(page_images: list[Image.Image], prompt: str) -> str:
    """Sends images to Gemini — accepts PIL directly."""
    model = genai.GenerativeModel("gemini-2.0-flash")
    parts = [prompt] + page_images[:16]
    response = model.generate_content(parts)
    return response.text

Unified function

def send_to_provider(pages: list[dict], images: list[Image.Image], prompt: str, provider: str = "openai") -> str:
    providers = {
        "openai": lambda: send_to_openai(pages, prompt),
        "anthropic": lambda: send_to_anthropic(pages, prompt),
        "gemini": lambda: send_to_gemini(images, prompt),
    }
    if provider not in providers:
        raise ValueError(f"Unsupported provider: {provider}")
    return providers[provider]()

Multi-Page Processing

Strategy 1: Sequential

Processes each page separately. Useful when each page is independent.

def process_sequential(images: list[Image.Image], prompt: str) -> list[str]:
    """Processes each page individually."""
    results = []
    for i, img in enumerate(images):
        page_data = prepare_page_for_vision(img)
        result = send_to_openai([page_data], f"Page {i + 1}. {prompt}")
        results.append(result)
    return results

Strategy 2: Batch

Groups pages into a single request. Better context between pages and fewer requests.

def process_batch(images: list[Image.Image], prompt: str, batch_size: int = 5) -> list[str]:
    """Processes pages in groups to reduce requests."""
    results = []
    for i in range(0, len(images), batch_size):
        batch = images[i:i + batch_size]
        pages_data = [prepare_page_for_vision(img) for img in batch]
        batch_prompt = f"{prompt}\n\nPages {i + 1} to {i + len(batch)} of the document."
        result = send_to_openai(pages_data, batch_prompt)
        results.append(result)
    return results

Strategy 3: Selective (intelligent)

Only sends the pages that need it to Vision. Text-only pages are processed with local OCR.

import numpy as np

def classify_page(img: Image.Image) -> str:
    """Classifies a page as 'text_only' or 'complex' using grayscale variation."""
    small = img.resize((100, 100)).convert("L")
    arr = np.array(small)
    return "text_only" if len(np.unique(arr)) < 20 else "complex"


def process_selective(images: list[Image.Image], prompt: str) -> list[dict]:
    """Sends only complex pages to Vision. Simple pages → local OCR."""
    import pytesseract
    results = []
    for i, img in enumerate(images):
        page_type = classify_page(img)
        if page_type == "complex":
            page_data = prepare_page_for_vision(img)
            text = send_to_openai([page_data], prompt)
            method = "vision"
        else:
            text = pytesseract.image_to_string(img, lang="eng")
            method = "ocr"
        results.append({"page": i + 1, "type": page_type, "method": method, "text": text})
    return results

When to use each strategy

Complete Pipeline: PDF → Images → Vision → Text

import json
from dataclasses import dataclass, field

@dataclass
class PageResult:
    page_number: int
    text: str
    method: str
    tokens_estimated: int = 0

@dataclass
class DocumentResult:
    source: str
    total_pages: int
    pages: list[PageResult] = field(default_factory=list)
    combined_text: str = ""
    total_cost: float = 0.0

def estimate_image_tokens(size_bytes: int, detail: str = "high") -> int:
    if detail == "low":
        return 85
    if size_bytes < 200_000:
        return 765
    elif size_bytes < 500_000:
        return 1105
    return 1105 + (size_bytes - 500_000) // 100_000 * 170


def full_pipeline(
    pdf_path: str,
    prompt: str = "Extract all the visible text. Keep the structure.",
    dpi: int = 150,
    batch_size: int = 5,
    detail: str = "high",
    selective: bool = True
) -> DocumentResult:
    """Complete pipeline: PDF → Images → Vision API → Combined text."""
    images = pdf_to_images_pymupdf(pdf_path, dpi=dpi)
    images = normalize_page_sizes(images)
    doc_result = DocumentResult(source=pdf_path, total_pages=len(images))
    import pytesseract

    for batch_start in range(0, len(images), batch_size):
        batch_imgs = images[batch_start:batch_start + batch_size]
        pages_data = []

        for i, img in enumerate(batch_imgs):
            page_num = batch_start + i + 1
            if selective and classify_page(img) == "text_only":
                text = pytesseract.image_to_string(img, lang="eng").strip()
                doc_result.pages.append(PageResult(page_num, text, "ocr_local"))
                continue
            page_data = prepare_page_for_vision(img)
            page_data["page_number"] = page_num
            pages_data.append(page_data)

        if pages_data:
            page_nums = [p["page_number"] for p in pages_data]
            batch_prompt = (
                f"{prompt}\n\nPages: {page_nums}. "
                f"Separate with '--- Page N ---'."
            )
            result_text = send_to_openai(pages_data, batch_prompt, detail)
            sections = result_text.split("--- Page")
            for j, pd in enumerate(pages_data):
                section = sections[j + 1] if j + 1 < len(sections) else result_text
                tokens = estimate_image_tokens(pd["size_bytes"], detail)
                doc_result.pages.append(PageResult(pd["page_number"], section.strip(), "vision_api", tokens))

    doc_result.pages.sort(key=lambda p: p.page_number)
    doc_result.combined_text = "\n\n".join(p.text for p in doc_result.pages)
    doc_result.total_cost = sum(p.tokens_estimated * 0.0000025 for p in doc_result.pages)
    return doc_result

Using the pipeline

result = full_pipeline("service_contract.pdf", selective=True)
print(f"Pages: {result.total_pages} | Cost: ${result.total_cost:.4f}")
for p in result.pages:
    print(f"  Page {p.page_number}: {p.method} ({p.tokens_estimated} tokens)")

Cost and Optimization

Calculating cost for multi-page documents

def estimate_document_cost(num_pages: int, detail: str = "high", model: str = "gpt-4o") -> dict:
    """Estimates the cost of processing a complete document."""
    prices = {
        "gpt-4o": {"input": 2.50, "output": 10.00},
        "gpt-4o-mini": {"input": 0.15, "output": 0.60},
    }
    price = prices.get(model, prices["gpt-4o"])
    tokens_per_image = 85 if detail == "low" else 765
    input_tokens = num_pages * tokens_per_image + 100
    output_tokens = num_pages * 500

    input_cost = (input_tokens / 1_000_000) * price["input"]
    output_cost = (output_tokens / 1_000_000) * price["output"]
    return {
        "model": model, "detail": detail,
        "input_tokens": input_tokens, "output_tokens": output_tokens,
        "total_cost": f"${input_cost + output_cost:.4f}"
    }

Comparative costs

6 strategies to reduce cost

Budget function

def process_within_budget(pdf_path: str, prompt: str, max_budget: float = 0.10) -> DocumentResult:
    """Processes a document, adjusting the configuration to respect a budget."""
    images = pdf_to_images_pymupdf(pdf_path, dpi=150)
    n = len(images)

    configs = [
        {"detail": "high", "model": "gpt-4o"},
        {"detail": "low", "model": "gpt-4o"},
        {"detail": "high", "model": "gpt-4o-mini"},
        {"detail": "low", "model": "gpt-4o-mini"},
    ]

    for cfg in configs:
        cost = estimate_document_cost(n, cfg["detail"], cfg["model"])
        total = float(cost["total_cost"].replace("$", ""))
        if total <= max_budget:
            print(f"Config: {cfg['model']}, detail={cfg['detail']}, cost≈${total:.4f}")
            return full_pipeline(pdf_path, prompt=prompt, selective=True, detail=cfg["detail"])

    print(f"Budget too low for {n} pages. Using minimal config.")
    return full_pipeline(pdf_path, prompt=prompt, selective=True, detail="low")

Use Case: Multi-Page Invoice

A 4-page invoice: cover page with issuer data, line-item detail (table), taxes/totals, and terms.

def process_invoice(pdf_path: str) -> dict:
    """Processes a multi-page invoice and extracts structured data."""
    extraction_prompt = """Analyze this invoice and extract as JSON:
{
    "issuer": {"name": "", "tax_id": "", "address": ""},
    "recipient": {"name": "", "tax_id": "", "address": ""},
    "number": "", "date": "",
    "line_items": [{"description": "", "quantity": 0, "unit_price": 0, "total": 0}],
    "subtotal": 0, "tax": 0, "total": 0, "currency": "", "payment_method": ""
}
Respond ONLY with valid JSON. Use null for fields not found.
Extract ALL the product lines from the table."""

    images = pdf_to_images_pymupdf(pdf_path, dpi=200)
    pages_data = [prepare_page_for_vision(img) for img in images]

    client = OpenAI()
    content = [{"type": "text", "text": extraction_prompt}]
    for page in pages_data:
        content.append({
            "type": "image_url",
            "image_url": {"url": f"data:{page['media_type']};base64,{page['base64']}", "detail": "high"}
        })

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": content}],
        response_format={"type": "json_object"},
        temperature=0,
        max_tokens=4000
    )
    invoice = json.loads(response.choices[0].message.content)

    if invoice.get("line_items"):
        calc = sum(l.get("total", 0) for l in invoice["line_items"] if l.get("total"))
        reported = invoice.get("subtotal", 0)
        if reported and abs(calc - reported) > 1:
            invoice["_warning"] = f"Calculated subtotal ({calc}) ≠ reported ({reported})"

    return invoice

Usage

invoice = process_invoice("services_invoice_2024.pdf")
print(f"Issuer: {invoice['issuer']['name']} | Number: {invoice['number']}")
for line in invoice.get("line_items", []):
    print(f"  {line['description']}: {line['quantity']} × ${line['unit_price']}")
print(f"Total: ${invoice['total']} {invoice['currency']}")

Troubleshooting

Problem 1: "Poppler not found" when using pdf2image

Cause: pdf2image needs Poppler installed at the system level.

Solution:

# macOS
brew install poppler
# Ubuntu/Debian
sudo apt-get install poppler-utils

If you can't install Poppler, use PyMuPDF as an alternative (it requires no system dependencies).

Problem 2: MemoryError with large PDFs

Cause: Converting all pages to high-resolution images at once exhausts memory.

Solution: Process page by page with a generator.

def process_large_pdf(pdf_path: str, dpi: int = 150):
    """Processes large PDFs without exhausting memory."""
    doc = fitz.open(pdf_path)
    mat = fitz.Matrix(dpi / 72, dpi / 72)
    for i in range(len(doc)):
        pix = doc[i].get_pixmap(matrix=mat, alpha=False)
        yield i + 1, Image.open(io.BytesIO(pix.tobytes("png")))
    doc.close()

Problem 3: Blurry images in the conversion

Cause: DPI too low for the content (tables with small text).

Solution: Increase DPI to 200. If only certain areas need detail, crop the region:

def extract_region(img: Image.Image, bbox: tuple) -> Image.Image:
    """Crops a specific region to send it in high resolution."""
    return img.crop(bbox)  # bbox = (x1, y1, x2, y2)

Problem 4: Exceeding the images-per-request limit

Cause: More than 10 images to OpenAI or more than 5 to Anthropic.

Solution: Split into chunks respecting each provider's limit.

PROVIDER_LIMITS = {"openai": 10, "anthropic": 5, "gemini": 16}

def chunk_pages(pages: list, provider: str = "openai") -> list[list]:
    limit = PROVIDER_LIMITS.get(provider, 5)
    return [pages[i:i + limit] for i in range(0, len(pages), limit)]

Problem 5: Timeout on documents with many pages

Cause: A request with many images exceeds the HTTP client timeout.

Solution: client = OpenAI(timeout=120.0, max_retries=3)

Exercises

Exercise 1: Compare quality by DPI

Write a function that converts the first page of a PDF at three different DPIs (100, 150, 200) and returns the size in KB of each one in PNG and JPEG.

def compare_dpi_quality(pdf_path: str) -> dict:
    doc = fitz.open(pdf_path)
    page = doc[0]
    results = {}
    for dpi in [100, 150, 200]:
        mat = fitz.Matrix(dpi / 72, dpi / 72)
        pix = page.get_pixmap(matrix=mat, alpha=False)
        img = Image.open(io.BytesIO(pix.tobytes("png")))

        buf_png = io.BytesIO()
        img.save(buf_png, format="PNG")
        buf_jpg = io.BytesIO()
        img.convert("RGB").save(buf_jpg, format="JPEG", quality=80)

        results[dpi] = {
            "resolution": f"{img.size[0]}×{img.size[1]}",
            "png_kb": round(len(buf_png.getvalue()) / 1024),
            "jpeg_kb": round(len(buf_jpg.getvalue()) / 1024)
        }
    doc.close()
    return results

Exercise 2: Send the same page to two providers

Create a function that takes a PIL image, prepares it, and sends it to OpenAI and Anthropic with the same prompt. Returns both responses to compare.

def compare_providers(img: Image.Image, prompt: str) -> dict:
    page_data = prepare_page_for_vision(img)
    return {
        "openai": send_to_openai([page_data], prompt),
        "anthropic": send_to_anthropic([page_data], prompt),
    }

Exercise 3: Selective pipeline with report

Implement a function that processes a PDF with the selective strategy and generates a report: how many pages with Vision, how many with OCR, and the estimated cost.

def selective_with_report(pdf_path: str, prompt: str) -> dict:
    images = pdf_to_images_pymupdf(pdf_path, dpi=150)
    results = process_selective(images, prompt)

    vision_pages = [r for r in results if r["method"] == "vision"]
    ocr_pages = [r for r in results if r["method"] == "ocr"]
    cost = len(vision_pages) * 0.003

    return {
        "total_pages": len(results),
        "vision_pages": len(vision_pages),
        "ocr_pages": len(ocr_pages),
        "vision_nums": [r["page"] for r in vision_pages],
        "estimated_cost": f"${cost:.4f}",
        "savings": f"{(1 - len(vision_pages)/max(len(results),1))*100:.0f}%",
        "pages": results
    }

Exercise 4: Validate and compress before sending

Write a function that takes images, verifies that the total size doesn't exceed the provider's limit, and if it does, compresses progressively until it fits.

def validate_and_compress(images: list[Image.Image], max_total_mb: float = 20.0) -> list[dict]:
    max_bytes = max_total_mb * 1024 * 1024
    quality = 85

    while quality >= 40:
        pages_data = [prepare_page_for_vision(img, jpeg_quality=quality) for img in images]
        total = sum(p["size_bytes"] for p in pages_data)
        if total <= max_bytes:
            return pages_data
        quality -= 10

    return [prepare_page_for_vision(img, max_side=1024, jpeg_quality=40) for img in images]

Additional Resources

1. OpenAI Vision — Image limits and pricing
2. Anthropic Vision — Image format requirements
3. PyMuPDF (fitz) — Pixmap documentation
4. pdf2image — GitHub repository
5. PIL Image.resize