7. Multi-Image and Context

Description

Until now you sent one image per request. But many real use cases require the model to see multiple images simultaneously: comparing a design before and after, analyzing the 12 pages of a contract, cataloging 20 products from an inventory, or detecting changes between photos taken on different dates.

Sending multiple images is not simply repeating a single-image request N times. It changes the structure of the request, the way you reference each image in the prompt, token consumption, and the strategies for when the set exceeds the context window. Each provider has its own format.

In this capsule you'll master the three formats, build four complete use cases, and learn to manage the context window as images pile up. The patterns from capsule 06 (description, OCR, comparison) apply here over sets of images, and the Base64 functions from capsule 06 of Module 1 are a prerequisite.

Multi-Image with OpenAI

Structure of the content array

To send multiple images with OpenAI, you add several image_url objects to the message's content array. The model processes all the images in order and can reference them by position.

import base64
from pathlib import Path
from openai import OpenAI

client = OpenAI()

MIME_TYPES = {
    ".jpg": "image/jpeg", ".jpeg": "image/jpeg",
    ".png": "image/png", ".gif": "image/gif", ".webp": "image/webp",
}


def encode_image(path: str) -> str:
    p = Path(path)
    mime = MIME_TYPES[p.suffix.lower()]
    with open(p, "rb") as f:
        b64 = base64.b64encode(f.read()).decode("utf-8")
    return f"data:{mime};base64,{b64}"


def analyze_multiple_openai(
    image_paths: list[str],
    prompt: str,
    detail: str = "auto",
    model: str = "gpt-4o",
) -> str:
    content = [{"type": "text", "text": prompt}]

    for path in image_paths:
        content.append({
            "type": "image_url",
            "image_url": {
                "url": encode_image(path),
                "detail": detail,
            },
        })

    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": content}],
        max_tokens=2048,
    )
    return response.choices[0].message.content

Referencing images in the prompt

The model receives the images in the order they appear in the array. Reference them explicitly by position:

prompt = """You have 3 images:
- Image 1: Architecture diagram
- Image 2: Code screenshot
- Image 3: Screenshot of the result

Describe the relationship between the three. Does the code implement the diagram?
Is the result consistent with what's expected?"""

result = analyze_multiple_openai(
    ["diagram.png", "code.png", "result.png"],
    prompt,
    detail="high",
)

The detail parameter per image

You can assign a different detail to each image within the same request. Build the content array manually instead of using the helper function, and assign "high" to images that need fine-grained analysis (documents, text) and "low" to context images (logos, reference photos). This optimizes token consumption without sacrificing quality where it matters.

OpenAI limits

• Practical maximum: ~10 images per request (depends on resolution and detail)
• Each image at high consumes 85 + 170×tiles tokens
• The total tokens (text + images) cannot exceed the 128K-token context window
• At low detail, each image consumes only 85 tokens — you can send more images

Multi-Image with Anthropic

Anthropic structures multi-image with image-type blocks in the content array. Each image goes as Base64 with its media_type separate — it does not accept URLs.

import anthropic

client_anthropic = anthropic.Anthropic()


def analyze_multiple_anthropic(
    image_paths: list[str],
    prompt: str,
    model: str = "claude-sonnet-4-20250514",
) -> str:
    content = []

    for i, path in enumerate(image_paths, 1):
        p = Path(path)
        mime = MIME_TYPES[p.suffix.lower()]
        with open(p, "rb") as f:
            b64 = base64.b64encode(f.read()).decode("utf-8")

        content.append({
            "type": "image",
            "source": {
                "type": "base64",
                "media_type": mime,
                "data": b64,
            },
        })

    content.append({"type": "text", "text": prompt})

    response = client_anthropic.messages.create(
        model=model,
        max_tokens=2048,
        messages=[{"role": "user", "content": content}],
    )
    return response.content[0].text


result = analyze_multiple_anthropic(
    ["photo_before.jpg", "photo_after.jpg"],
    "Compare Image 1 and Image 2. List similarities and differences.",
)

Order of the blocks

In Anthropic, the image blocks appear before the text block in the content array. The model processes them in sequential order: Image 1 is the first image block, Image 2 the second, etc. You can reference them by position in the prompt just like with OpenAI.

Anthropic limits

• Each image: maximum 5 MB (in Base64, which inflates ~33%)
• The original file should not exceed ~3.75 MB so it doesn't surpass the limit after encoding
• Context window: 200K tokens (Claude Sonnet)
• No explicit limit on the number of images, but the context window is the real ceiling

Multi-Image with Gemini

Gemini has the simplest structure for multi-image: a flat list where you mix text and images as sequential elements.

import google.generativeai as genai
from PIL import Image

genai.configure()
gemini_model = genai.GenerativeModel("gemini-2.0-flash")


def analyze_multiple_gemini(
    image_paths: list[str],
    prompt: str,
) -> str:
    contents = [prompt]

    for path in image_paths:
        contents.append(Image.open(path))

    response = gemini_model.generate_content(contents)
    return response.text


result = analyze_multiple_gemini(
    ["dashboard_january.png", "dashboard_february.png", "dashboard_march.png"],
    "Analyze the evolution of the 3 monthly dashboards. "
    "Image 1 = January, Image 2 = February, Image 3 = March. "
    "Which metrics improved and which got worse?",
)

Advantage: extended context window

Gemini's main advantage for multi-image is its massive context window: 1M tokens in gemini-1.5-pro, up to 2M in gemini-2.0-flash. Where OpenAI caps at ~10 images at high, Gemini can take in dozens of images in a single request. For large catalogs, simply iterate over a directory and append each image to the contents list.

Gemini limits

• Context window: up to 2M tokens (model-dependent)
• Individual image: maximum 20 MB
• Formats: PNG, JPEG, GIF, WebP, BMP
• Accepts HTTP URLs, gs:// URIs, and PIL objects directly

Format Comparison

Few images with high precision → OpenAI detail=high or Anthropic. Dozens of images at once → Gemini.

Use Case 1: Before/After Comparison

A frequent pattern is sending two images to generate a change report. Applies to: renovations, design versions, dashboard evolution, quality control.

COMPARISON_PROMPT = """You have two images:
- Image 1: BEFORE state
- Image 2: AFTER state

Generate a report with: Similarities, Differences, the top 3 Main Changes,
and an Assessment (direction: improvement/deterioration/neutral, confidence: high/medium/low,
summary in one sentence)."""


def compare_before_after(
    before_path: str,
    after_path: str,
    context: str = "",
    provider: str = "openai",
) -> str:
    prompt = COMPARISON_PROMPT
    if context:
        prompt += f"\n\nAdditional context: {context}"

    if provider == "openai":
        return analyze_multiple_openai([before_path, after_path], prompt, detail="high")
    elif provider == "anthropic":
        return analyze_multiple_anthropic([before_path, after_path], prompt)
    else:
        return analyze_multiple_gemini([before_path, after_path], prompt)


report = compare_before_after(
    "office_2024.jpg",
    "office_2025.jpg",
    context="Corporate office remodel",
    provider="openai",
)
print(report)

The key is for the prompt to explicitly assign roles ("Image 1 = before", "Image 2 = after") so the model doesn't confuse the order.

Use Case 2: Multi-Page Document

Converting a PDF into images and sending them as multi-image is one of the most practical applications. The key decision is all pages at once vs. batch chunking.

Converting PDF to images

from pdf2image import convert_from_path
import tempfile


def pdf_to_images(pdf_path: str, dpi: int = 200) -> list[str]:
    images = convert_from_path(pdf_path, dpi=dpi)
    paths = []

    with tempfile.TemporaryDirectory() as tmpdir:
        for i, img in enumerate(images):
            path = f"{tmpdir}/page_{i+1:03d}.jpg"
            img.save(path, "JPEG", quality=85)
            paths.append(path)

    return paths

Strategy 1: All pages at once

For short documents (≤10 pages), send everything in one request:

def analyze_full_document(pdf_path: str, prompt: str) -> str:
    page_paths = pdf_to_images(pdf_path)

    numbered_prompt = f"You have a {len(page_paths)}-page document.\n"
    for i in range(len(page_paths)):
        numbered_prompt += f"- Image {i+1}: Page {i+1}\n"
    numbered_prompt += f"\n{prompt}"

    return analyze_multiple_openai(page_paths, numbered_prompt, detail="high")


result = analyze_full_document(
    "contract_5_pages.pdf",
    "Extract all the clauses, amounts, and deadlines from the contract.",
)

Strategy 2: Batch chunking

For long documents, split into chunks, analyze each one, and consolidate:

def analyze_document_chunked(
    pdf_path: str,
    prompt: str,
    chunk_size: int = 5,
) -> str:
    page_paths = pdf_to_images(pdf_path)
    partial_results = []

    for start in range(0, len(page_paths), chunk_size):
        chunk = page_paths[start:start + chunk_size]
        first_page, last_page = start + 1, start + len(chunk)
        chunk_prompt = (
            f"Pages {first_page}-{last_page} of {len(page_paths)}.\n\n{prompt}"
        )
        result = analyze_multiple_openai(chunk, chunk_prompt, detail="high")
        partial_results.append(f"### Pages {first_page}-{last_page}\n{result}")

    consolidation = client.chat.completions.create(
        model="gpt-4o",
        messages=[{
            "role": "user",
            "content": "Consolidate these partial analyses into a unified summary. "
                "Remove redundancies.\n\n" + "\n\n".join(partial_results),
        }],
        max_tokens=4096,
    )
    return consolidation.choices[0].message.content

Cost estimation for multi-page

A typical document page (1700×2200 at 200 DPI) consumes ~1105 tokens at high detail. To estimate the cost of a full document:

Exercise 3 at the end of this capsule includes a complete function to calculate these costs with real dimensions.

Use Case 3: Product Catalog

Analyze a batch of product images extracting structured data from each one. The key is for the prompt to reference each image by number so the model returns organized data.

import json

CATALOG_PROMPT = """You have {n} product images.
{image_list}

For EACH product, extract in JSON: image_number, product_name, category,
visible_price (null if not visible), color, condition (new/used/undeterminable),
description (one sentence).

Respond ONLY with a valid JSON array."""


def analyze_product_catalog(
    image_paths: list[str],
    provider: str = "openai",
) -> list[dict]:
    image_list = "\n".join(
        f"- Image {i+1}: Product {i+1}" for i in range(len(image_paths))
    )
    prompt = CATALOG_PROMPT.format(n=len(image_paths), image_list=image_list)

    if provider == "openai":
        raw = analyze_multiple_openai(image_paths, prompt, detail="high")
    elif provider == "anthropic":
        raw = analyze_multiple_anthropic(image_paths, prompt)
    else:
        raw = analyze_multiple_gemini(image_paths, prompt)

    cleaned = raw.strip()
    if cleaned.startswith("```"):
        cleaned = cleaned.split("\n", 1)[1].rsplit("```", 1)[0]

    return json.loads(cleaned)


products = analyze_product_catalog(
    ["shoe_1.jpg", "shoe_2.jpg", "shoe_3.jpg", "shoe_4.jpg"],
    provider="openai",
)

for p in products:
    print(f"Image {p['image_number']}: {p['product_name']} - {p['category']}")

For large catalogs (>10 products), process in batches of 5-8 images and concatenate the JSON arrays.

Use Case 4: Temporal Evolution

Sending chronological images lets you detect trends, construction progress, metric evolution, or medical follow-up.

TEMPORAL_PROMPT = """You have {n} images ordered chronologically:
{timeline}

Analyze the temporal evolution:

1. **Overall trend**: Improvement, deterioration, stability, or cycle?
2. **Changes between each consecutive pair**: List what changed between image N and image N+1
3. **Inflection point**: Is there a moment where the pattern changes?
4. **Prediction**: Based on the trend, what would you expect in the next observation?"""


def analyze_temporal_evolution(
    image_paths: list[str],
    labels: list[str],
    provider: str = "openai",
) -> str:
    timeline = "\n".join(
        f"- Image {i+1}: {label}" for i, label in enumerate(labels)
    )
    prompt = TEMPORAL_PROMPT.format(n=len(image_paths), timeline=timeline)

    if provider == "openai":
        return analyze_multiple_openai(image_paths, prompt, detail="high")
    elif provider == "anthropic":
        return analyze_multiple_anthropic(image_paths, prompt)
    else:
        return analyze_multiple_gemini(image_paths, prompt)


report = analyze_temporal_evolution(
    ["dashboard_q1.png", "dashboard_q2.png", "dashboard_q3.png", "dashboard_q4.png"],
    ["Q1 2025", "Q2 2025", "Q3 2025", "Q4 2025"],
    provider="gemini",
)
print(report)

Gemini is ideal for temporal evolution: 12 months with one image per month fits in a single request, whereas with OpenAI you'd need chunking.

Context Window Management

How multi-image affects tokens

Each image consumes tokens depending on the provider and resolution. The problem appears when you send many images and the total sum exceeds the context window.

import math

def estimate_multi_image_tokens(
    n_images: int, width: int = 1024, height: int = 1024, detail: str = "high",
) -> dict:
    if detail == "low":
        per_image = 85
    else:
        w, h = width, height
        if max(w, h) > 2048:
            scale = 2048 / max(w, h)
            w, h = int(w * scale), int(h * scale)
        if min(w, h) > 768:
            scale = 768 / min(w, h)
            w, h = int(w * scale), int(h * scale)
        per_image = 85 + 170 * (math.ceil(w / 512) * math.ceil(h / 512))

    total = per_image * n_images
    limits = {"openai": 128_000, "anthropic": 200_000, "gemini": 1_000_000}

    return {
        "tokens_per_image": per_image,
        "total_tokens": total,
        "fits_in": {p: total < lim for p, lim in limits.items()},
    }

for n in [5, 10, 20, 50, 100]:
    est = estimate_multi_image_tokens(n, detail="high")
    ok = [p for p, fits in est["fits_in"].items() if fits]
    print(f"{n:>3} imgs: {est['total_tokens']:>7,} tokens → {', '.join(ok)}")

Chunking when the context is exceeded

When the total tokens exceed the context window, split the images into chunks. The analyze_document_chunked function from Use Case 2 implements this pattern: it processes batches of N images, collects partial results, and consolidates with a pure-text call at the end.

Token reduction strategy

Before resorting to chunking, reduce token consumption:

1. Lower detail to low: Reduces from ~765 tokens to 85 tokens per image (OpenAI)
2. Resize images: A 512×512 image consumes fewer tiles than 4000×3000
3. Use gpt-4o-mini: Same context window, lower cost
4. Filter out irrelevant images: Send only those the analysis needs

Troubleshooting

Problem 1: Context window exceeded

Symptom: Error context_length_exceeded or max_tokens when sending many images.

Cause: The sum of tokens from all images + prompt + expected output exceeds the model's limit.

Solution: Use estimate_multi_image_tokens() before sending. If it exceeds, apply chunking or reduce detail to low. For large volumes, migrate to Gemini (1-2M tokens of context).

Problem 2: The model confuses the order of the images

Symptom: The response attributes content from Image 2 to Image 1, or mixes information between images.

Cause: The prompt doesn't assign clear roles to each image, or the prompt appears after many images and the model loses the reference.

Solution: Always include an explicit list at the start of the prompt: "Image 1 = X, Image 2 = Y". With Anthropic, place the images before the text so the model sees them in context before reading the instructions.

Problem 3: A corrupt image fails the entire request

Symptom: Parsing error or invalid_image when sending a batch, even though most of the images are valid.

Cause: A single image with an incorrect format, exceeded size, or invalid Base64 encoding breaks the entire request.

Solution: Validate each image before including it in the batch. Open each file with PIL.Image.open().verify(), check that the size doesn't exceed the provider's limit, and separate valid images from invalid ones. Send only the valid ones and report the ones that failed.

Problem 4: Unexpectedly high cost

Symptom: The bill is much higher than expected after implementing multi-image.

Cause: Multiple images at detail=high × multiple requests × an expensive model scale quickly. 100 requests × 5 images × gpt-4o high ≈ $1.50 in image tokens alone.

Solution: Estimate the cost before running a batch (see Exercise 3). Use gpt-4o-mini for initial filtering and reserve gpt-4o high only for analysis that requires it.

Exercises

Exercise 1: Comparison function with structured output (Easy)

Create a compare_two_images(path1, path2) function that sends two images to OpenAI and returns a dictionary with the keys similarities (list), differences (list), and verdict (string: "same", "similar", or "different").

import json
from openai import OpenAI

client = OpenAI()

def compare_two_images(path1: str, path2: str) -> dict:
    prompt = """Compare Image 1 and Image 2.

Respond ONLY with valid JSON with this structure:
{
  "similarities": ["similarity 1", "similarity 2", ...],
  "differences": ["difference 1", "difference 2", ...],
  "verdict": "same | similar | different"
}"""

    content = [
        {"type": "text", "text": prompt},
        {"type": "image_url", "image_url": {"url": encode_image(path1), "detail": "high"}},
        {"type": "image_url", "image_url": {"url": encode_image(path2), "detail": "high"}},
    ]

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": content}],
        max_tokens=1024,
        temperature=0,
    )

    raw = response.choices[0].message.content.strip()
    if raw.startswith("```"):
        raw = raw.split("\n", 1)[1].rsplit("```", 1)[0]
    return json.loads(raw)


result = compare_two_images("version_a.png", "version_b.png")
print(f"Verdict: {result['verdict']}")
print(f"Similarities: {len(result['similarities'])}")
print(f"Differences: {len(result['differences'])}")

Exercise 2: PDF analyzer with chunking and consolidation (Medium)

Create an analyze_pdf(pdf_path, prompt, chunk_size=5) function that converts a PDF into images, processes them in chunks, and returns a consolidated result.

from pdf2image import convert_from_path
from openai import OpenAI
import tempfile

client = OpenAI()

def analyze_pdf(pdf_path: str, prompt: str, chunk_size: int = 5) -> str:
    with tempfile.TemporaryDirectory() as tmpdir:
        images = convert_from_path(pdf_path, dpi=200)
        page_paths = []
        for i, img in enumerate(images):
            path = f"{tmpdir}/page_{i+1:03d}.jpg"
            img.save(path, "JPEG", quality=85)
            page_paths.append(path)

        if len(page_paths) <= chunk_size:
            return analyze_multiple_openai(page_paths, prompt, detail="high")

        partial_results = []
        for start in range(0, len(page_paths), chunk_size):
            chunk = page_paths[start:start + chunk_size]
            first_page, last_page = start + 1, start + len(chunk)
            chunk_prompt = (
                f"Pages {first_page}-{last_page} of {len(page_paths)}.\n\n{prompt}"
            )
            result = analyze_multiple_openai(chunk, chunk_prompt, detail="high")
            partial_results.append(f"[Pages {first_page}-{last_page}]\n{result}")

        consolidation = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": "Consolidate these partial analyses into a unified summary. "
                    "Remove redundancies.\n\n" + "\n\n---\n\n".join(partial_results),
            }],
            max_tokens=4096,
        )
        return consolidation.choices[0].message.content

Exercise 3: Multi-image cost estimator (Medium)

Create an estimate_batch_cost(image_paths, detail, model) function that reads the real dimensions of each image, calculates the exact tokens, and returns the total estimated cost before making any API call.

import math
from PIL import Image
from pathlib import Path

PRICING = {
    "gpt-4o": {"input": 2.50, "output": 10.00},
    "gpt-4o-mini": {"input": 0.15, "output": 0.60},
}


def calculate_image_tokens(width: int, height: int, detail: str = "high") -> int:
    if detail == "low":
        return 85
    w, h = width, height
    if max(w, h) > 2048:
        scale = 2048 / max(w, h)
        w, h = int(w * scale), int(h * scale)
    if min(w, h) > 768:
        scale = 768 / min(w, h)
        w, h = int(w * scale), int(h * scale)
    tiles = math.ceil(w / 512) * math.ceil(h / 512)
    return 85 + 170 * tiles


def estimate_batch_cost(
    image_paths: list[str],
    detail: str = "high",
    model: str = "gpt-4o",
    output_tokens_per_image: int = 100,
) -> dict:
    total_tokens = 0
    per_image = []
    for path in image_paths:
        with Image.open(path) as img:
            w, h = img.size
        tokens = calculate_image_tokens(w, h, detail)
        total_tokens += tokens
        per_image.append({"path": Path(path).name, "size": f"{w}x{h}", "tokens": tokens})

    total_output = output_tokens_per_image * len(image_paths)
    p = PRICING[model]
    cost = (total_tokens / 1e6) * p["input"] + (total_output / 1e6) * p["output"]

    return {
        "n_images": len(image_paths),
        "total_image_tokens": total_tokens,
        "total_cost_usd": round(cost, 6),
        "per_image": per_image,
    }


est = estimate_batch_cost(["product_1.jpg", "product_2.jpg", "product_3.jpg"], model="gpt-4o-mini")
print(f"{est['n_images']} images: {est['total_image_tokens']:,} tokens, ${est['total_cost_usd']:.6f}")

Summary

• Structure: OpenAI uses an array of image_url, Anthropic uses image blocks with Base64, Gemini a flat content list
• Reference: Always number the images explicitly in the prompt ("Image 1 = X, Image 2 = Y")
• Practical limits: ~10 images (OpenAI high), ~20-30 (Anthropic), ~50-100+ (Gemini)
• Multi-page documents: Batch chunking + consolidation for documents that exceed the context
• Tokens: Each image at high consumes ~765 tokens (1024×1024). Multiply by N images to estimate the total
• Cost: Always estimate before running. detail=low reduces image-token consumption 9x
• Context window: If you exceed the limit, reduce detail, resize, or apply chunking. Gemini (1-2M tokens) is the best option for high volume

Additional Resources

1. OpenAI Vision — Multiple Images — Official documentation with multi-image examples
2. Anthropic Vision Docs — Format for multiple image blocks
3. Google Gemini Vision — Multi-image with extended context
4. pdf2image (PyPI) — PDF-to-images conversion for multi-page analysis
5. OpenAI Token Calculator — Reference for token estimation