2. PDF Processing

Description

PDFs are the standard format for enterprise documents. In this capsule you'll learn to extract text and images using PyMuPDF and pdf2image. You'll master the flow: open PDF → detect type → extract text or convert pages to images → get metadata. By the end, you'll have a pipeline that automatically decides the best extraction strategy.

Why it matters: Without correct PDF extraction, you can't process invoices, contracts or manuals. PyMuPDF is fast and has no external dependencies; pdf2image converts pages to images for Vision APIs. Knowing when to use each tool is the foundation of any document understanding pipeline.

Connection with the module: Capsule 03 (OCR + LLM) needs the images you'll extract here. Capsule 05 (Structured Extraction) starts from the text you get from PyMuPDF. The final project combines this whole flow.

Key Concepts

PDFs with text vs scanned

PyMuPDF vs pdf2image

Text Extraction with PyMuPDF

Open a PDF and iterate over pages

import fitz  # PyMuPDF

doc = fitz.open("invoice.pdf")
print(f"Pages: {len(doc)}")

for page in doc:
    print(f"\n--- Page {page.number + 1} ---")
    text = page.get_text()
    print(f"Characters: {len(text)}")
    print(text[:200])

doc.close()

Extract all the text

import fitz


def extract_text_from_pdf(pdf_path: str) -> str:
    """Extracts all the text from a PDF by concatenating every page."""
    doc = fitz.open(pdf_path)
    text_parts = []
    for page in doc:
        page_text = page.get_text()
        if page_text.strip():
            text_parts.append(page_text)
    doc.close()
    return "\n\n".join(text_parts)


text = extract_text_from_pdf("invoice.pdf")
print(f"Total characters: {len(text):,}")
print(text[:500])

Per-page extraction with metadata

import fitz


def extract_text_by_page(pdf_path: str) -> list[dict]:
    """Extracts text with metadata for each page."""
    doc = fitz.open(pdf_path)
    result = []
    for page in doc:
        text = page.get_text()
        rect = page.rect
        result.append({
            "page_num": page.number + 1,
            "text": text,
            "char_count": len(text),
            "word_count": len(text.split()),
            "width": rect.width,
            "height": rect.height,
            "has_text": len(text.strip()) > 10
        })
    doc.close()
    return result


pages = extract_text_by_page("manual.pdf")
for p in pages:
    status = "with text" if p["has_text"] else "without text"
    print(f"Page {p['page_num']}: {p['word_count']} words ({status})")

Extraction by blocks (layout preserved)

PyMuPDF can extract text respecting the visual position of each block on the page — useful for identifying headers, columns or sections. Use page.get_text("blocks"), which returns tuples (x0, y0, x1, y1, text, block_no, block_type) where block_type == 0 indicates text and 1 indicates an image.

Detect PDF Type

A PDF generated from Word has selectable text. A scanned one is just an image per page. Some are mixed. This function classifies each page:

import fitz


def detect_pdf_type(pdf_path: str) -> dict:
    """
    Classifies each page as 'text' or 'scanned'.
    Returns the overall type and per-page detail.
    """
    doc = fitz.open(pdf_path)
    page_analysis = []
    text_pages = 0
    scanned_pages = 0

    for page in doc:
        char_count = len(page.get_text().strip())
        has_text = char_count > 50

        if has_text:
            text_pages += 1
        else:
            scanned_pages += 1

        page_analysis.append({
            "page_num": page.number + 1,
            "type": "text" if has_text else "scanned",
            "char_count": char_count
        })

    doc.close()

    if scanned_pages == 0:
        overall = "text"
    elif text_pages == 0:
        overall = "scanned"
    else:
        overall = "mixed"

    return {
        "overall_type": overall,
        "total_pages": len(page_analysis),
        "text_pages": text_pages,
        "scanned_pages": scanned_pages,
        "pages": page_analysis
    }


result = detect_pdf_type("document.pdf")
print(f"Type: {result['overall_type']}")
print(f"Pages with text: {result['text_pages']}/{result['total_pages']}")
for p in result["pages"]:
    print(f"  Page {p['page_num']}: {p['type']} ({p['char_count']} chars)")

The 50-character threshold works well in practice: a scanned page may have a few residual characters, but rarely more than 50.

Extraction of Embedded Images

PDFs can contain embedded images (photos, logos, charts). PyMuPDF extracts them directly:

import fitz
from pathlib import Path


def extract_embedded_images(pdf_path: str, output_dir: str = "images") -> list[dict]:
    """Extracts all the embedded images from a PDF."""
    doc = fitz.open(pdf_path)
    Path(output_dir).mkdir(exist_ok=True)
    extracted = []

    for page in doc:
        for img_index, img_info in enumerate(page.get_images(full=True)):
            xref = img_info[0]
            base_image = doc.extract_image(xref)
            image_bytes = base_image["image"]
            image_ext = base_image["ext"]

            filename = f"page{page.number + 1}_img{img_index + 1}.{image_ext}"
            filepath = Path(output_dir) / filename
            with open(filepath, "wb") as f:
                f.write(image_bytes)

            extracted.append({
                "page": page.number + 1,
                "filename": filename,
                "format": image_ext,
                "width": base_image["width"],
                "height": base_image["height"],
                "size_kb": len(image_bytes) / 1024
            })

    doc.close()
    return extracted


images = extract_embedded_images("catalog.pdf")
for img in images:
    print(f"Page {img['page']}: {img['filename']} ({img['width']}x{img['height']})")

PyMuPDF for Page Images

PyMuPDF converts complete pages to images using pixmaps — with no dependency on Poppler.

Convert one page

import fitz


def pdf_page_to_image(pdf_path: str, page_num: int = 0, dpi: int = 150) -> bytes:
    """Converts a page to a PNG image using a PyMuPDF pixmap."""
    doc = fitz.open(pdf_path)
    page = doc[page_num]
    zoom = dpi / 72  # 72 DPI is the base resolution of a PDF
    mat = fitz.Matrix(zoom, zoom)
    pix = page.get_pixmap(matrix=mat, alpha=False)
    img_bytes = pix.tobytes("png")
    doc.close()
    return img_bytes

Convert all pages

import fitz
from pathlib import Path


def pdf_all_pages_to_images(
    pdf_path: str, output_dir: str = "pages", dpi: int = 150, fmt: str = "png"
) -> list[str]:
    """Converts all pages to individual images."""
    doc = fitz.open(pdf_path)
    Path(output_dir).mkdir(exist_ok=True)
    zoom = dpi / 72
    mat = fitz.Matrix(zoom, zoom)
    saved_files = []

    for page in doc:
        pix = page.get_pixmap(matrix=mat, alpha=False)
        filename = f"page_{page.number + 1:03d}.{fmt}"
        filepath = str(Path(output_dir) / filename)
        pix.save(filepath)
        saved_files.append(filepath)

    doc.close()
    return saved_files


files = pdf_all_pages_to_images("report.pdf", dpi=200)
print(f"Saved {len(files)} pages")

DPI comparison

For Vision APIs, 150–200 DPI is enough. Higher doesn't improve results and increases the cost from image tokens.

Conversion with pdf2image

pdf2image uses Poppler as the rendering engine and produces PIL Image objects.

Basic conversion

from pdf2image import convert_from_path

images = convert_from_path("document.pdf", dpi=150)
print(f"Pages converted: {len(images)}")
for i, img in enumerate(images):
    print(f"Page {i + 1}: {img.size[0]}x{img.size[1]} px")

Convert specific pages and save

from pdf2image import convert_from_path
from pathlib import Path


def convert_and_save_pages(
    pdf_path: str, first_page: int = 1, last_page: int = 1,
    dpi: int = 150, output_dir: str = "pages"
) -> list[str]:
    """Converts a range of pages (1-indexed) and saves them as PNG."""
    Path(output_dir).mkdir(exist_ok=True)
    images = convert_from_path(
        pdf_path, dpi=dpi, first_page=first_page, last_page=last_page, fmt="png"
    )
    saved = []
    for i, img in enumerate(images):
        filepath = str(Path(output_dir) / f"page_{first_page + i:03d}.png")
        img.save(filepath, "PNG")
        saved.append(filepath)
    return saved


files = convert_and_save_pages("manual.pdf", first_page=1, last_page=5, dpi=200)
for f in files:
    print(f"Saved: {f}")

Large PDFs: batch conversion

With PDFs of many pages, converting everything at once consumes too much memory. Use first_page and last_page to process in batches, and thread_count=4 to parallelize. Get the total number of pages with pdfinfo_from_path(pdf_path)["Pages"] and iterate in ranges of batch_size.

Table Extraction

PyMuPDF includes table detection (starting from v1.23.0), which lets you extract tabular data without additional libraries.

import fitz


def extract_tables_from_pdf(pdf_path: str) -> list[dict]:
    """Extracts tables from all pages of a PDF."""
    doc = fitz.open(pdf_path)
    all_tables = []

    for page in doc:
        tables = page.find_tables()
        for table_idx, table in enumerate(tables):
            data = table.extract()
            if not data:
                continue
            headers = data[0] if len(data) > 1 else [f"col_{i}" for i in range(len(data[0]))]
            rows = data[1:] if len(data) > 1 else data

            all_tables.append({
                "page": page.number + 1,
                "table_index": table_idx + 1,
                "headers": headers,
                "rows": rows,
                "row_count": len(rows),
                "col_count": len(data[0])
            })

    doc.close()
    return all_tables


tables = extract_tables_from_pdf("financial_report.pdf")
for t in tables:
    print(f"\nPage {t['page']}, Table {t['table_index']}:")
    print(f"  Columns: {t['headers']}")
    print(f"  Rows: {t['row_count']}")
    for row in t["rows"][:3]:
        print(f"  {row}")

To convert the tables into dictionaries, use the first row as headers and map each cell: {headers[i]: cell for i, cell in enumerate(row)} over data[1:].

Metadata and Structure

Document metadata

import fitz
from pathlib import Path


def get_pdf_metadata(pdf_path: str) -> dict:
    """Extracts complete metadata from the PDF."""
    doc = fitz.open(pdf_path)
    meta = doc.metadata
    result = {
        "title": meta.get("title", ""),
        "author": meta.get("author", ""),
        "subject": meta.get("subject", ""),
        "creator": meta.get("creator", ""),
        "producer": meta.get("producer", ""),
        "creation_date": meta.get("creationDate", ""),
        "modification_date": meta.get("modDate", ""),
        "page_count": len(doc),
        "file_size_kb": Path(pdf_path).stat().st_size / 1024
    }
    doc.close()
    return result

Table of contents and page dimensions

import fitz


def get_pdf_structure(pdf_path: str) -> dict:
    """Extracts the table of contents and page dimensions."""
    doc = fitz.open(pdf_path)
    toc = [{"level": l, "title": t, "page": p} for l, t, p in doc.get_toc()]

    pages = []
    for page in doc:
        rect = page.rect
        pages.append({
            "page": page.number + 1,
            "width_pt": rect.width,
            "height_pt": rect.height,
            "width_cm": round(rect.width * 2.54 / 72, 1),
            "height_cm": round(rect.height * 2.54 / 72, 1),
            "orientation": "landscape" if rect.width > rect.height else "portrait"
        })

    doc.close()
    return {"table_of_contents": toc, "pages": pages}

Pipeline: PDF → Text or Images

A central function that combines type detection, text extraction and conversion to images:

import fitz
from pathlib import Path


def process_pdf(pdf_path: str, dpi: int = 150) -> dict:
    """
    Complete pipeline: detects the type per page, extracts text where there is any,
    generates images where there isn't, and includes metadata.
    """
    doc = fitz.open(pdf_path)
    zoom = dpi / 72
    mat = fitz.Matrix(zoom, zoom)

    pages = []
    text_count = 0
    scan_count = 0
    full_text_parts = []

    for page in doc:
        text = page.get_text()
        has_text = len(text.strip()) > 50
        page_data = {"page_num": page.number + 1, "has_text": has_text}

        if has_text:
            text_count += 1
            page_data["text"] = text
            full_text_parts.append(f"--- Page {page.number + 1} ---\n{text}")
        else:
            scan_count += 1
            pix = page.get_pixmap(matrix=mat, alpha=False)
            page_data["image_bytes"] = pix.tobytes("png")

        pages.append(page_data)

    meta = doc.metadata
    metadata = {
        "title": meta.get("title", ""),
        "author": meta.get("author", ""),
        "page_count": len(doc),
        "file_size_kb": Path(pdf_path).stat().st_size / 1024
    }
    doc.close()

    if scan_count == 0:
        overall = "text"
    elif text_count == 0:
        overall = "scanned"
    else:
        overall = "mixed"

    return {
        "type": overall,
        "text_pages": text_count,
        "scanned_pages": scan_count,
        "full_text": "\n\n".join(full_text_parts),
        "pages": pages,
        "metadata": metadata
    }


result = process_pdf("document.pdf")
print(f"Type: {result['type']}")
print(f"Text pages: {result['text_pages']}, scanned: {result['scanned_pages']}")

Troubleshooting

Problem: "No module named 'fitz'"

Cause: PyMuPDF installs as pymupdf but imports as fitz.

Solution: pip install pymupdf and then import fitz.

Problem: pdf2image fails with "Unable to get page count"

Cause: Poppler not installed or not in PATH.

Solution:

# macOS
brew install poppler
# Linux (Debian/Ubuntu)
apt-get install poppler-utils

Problem: Password-protected PDF

Cause: The PDF requires a password to open or copy text.

Solution:

import fitz

doc = fitz.open("protected.pdf")
if doc.is_encrypted:
    if not doc.authenticate("my_password"):
        raise PermissionError("Incorrect password")

Problem: A very large PDF consumes too much memory

Cause: Converting all pages to high-resolution images exhausts the RAM.

Solution: Process in batches and free memory between iterations.

import fitz
import gc


def process_large_pdf_pages(pdf_path: str, dpi: int = 150, batch_size: int = 20):
    """Processes a large PDF in batches, freeing memory."""
    doc = fitz.open(pdf_path)
    zoom = dpi / 72
    mat = fitz.Matrix(zoom, zoom)

    for start in range(0, len(doc), batch_size):
        end = min(start + batch_size, len(doc))
        batch = [doc[i].get_pixmap(matrix=mat, alpha=False).tobytes("png")
                 for i in range(start, end)]
        yield start, batch
        gc.collect()

    doc.close()

Problem: Extracted text with strange characters or mojibake

Cause: The PDF uses embedded fonts with custom encoding that PyMuPDF can't map.

Solution: If page.get_text("text") contains � or garbage, fall back: convert the page to an image with page.get_pixmap() and send it to OCR or a Vision API.

Exercises

Exercise 1: Complete PDF analyzer

Create a function analyze_pdf that takes a PDF path and returns: number of pages, type (text/scanned/mixed), number of embedded images, whether it has a table of contents, and author and title metadata.

import fitz
from pathlib import Path


def analyze_pdf(pdf_path: str) -> dict:
    doc = fitz.open(pdf_path)
    text_pages = scan_pages = total_images = 0

    for page in doc:
        if len(page.get_text().strip()) > 50:
            text_pages += 1
        else:
            scan_pages += 1
        total_images += len(page.get_images(full=True))

    toc = doc.get_toc()
    meta = doc.metadata

    if scan_pages == 0:
        pdf_type = "text"
    elif text_pages == 0:
        pdf_type = "scanned"
    else:
        pdf_type = "mixed"

    result = {
        "file": Path(pdf_path).name,
        "pages": len(doc),
        "type": pdf_type,
        "text_pages": text_pages,
        "scanned_pages": scan_pages,
        "embedded_images": total_images,
        "has_toc": len(toc) > 0,
        "title": meta.get("title", ""),
        "author": meta.get("author", "")
    }
    doc.close()
    return result

Exercise 2: Type detector with report

Create a function that analyzes a PDF and generates a text report: how many pages have text, how many are scanned, and a processing recommendation.

import fitz


def generate_pdf_report(pdf_path: str) -> str:
    doc = fitz.open(pdf_path)
    lines = [f"=== Report: {pdf_path} ===", f"Total pages: {len(doc)}", ""]
    text_count = scan_count = 0

    for page in doc:
        chars = len(page.get_text().strip())
        pn = page.number + 1
        if chars > 50:
            text_count += 1
            lines.append(f"Page {pn}: TEXT ({chars} chars) → direct extraction")
        else:
            scan_count += 1
            lines.append(f"Page {pn}: SCANNED ({chars} chars) → needs OCR/Vision")

    lines.append("")
    lines.append(f"Summary: {text_count} with text, {scan_count} scanned")
    if scan_count == 0:
        lines.append("Recommendation: direct text extraction with PyMuPDF")
    elif text_count == 0:
        lines.append("Recommendation: convert to images and use a Vision API or OCR")
    else:
        lines.append("Recommendation: mixed processing (text + Vision API)")

    doc.close()
    return "\n".join(lines)

Exercise 3: Page converter with resizing

Create a function that converts pages of a PDF to PNG images, with parameters for range, DPI and maximum size in pixels (rescale if it exceeds the limit).

import fitz
from PIL import Image
import io
from pathlib import Path


def convert_pages_to_images(
    pdf_path: str, pages: list[int] | None = None,
    dpi: int = 150, max_pixels: int = 2048, output_dir: str = "output"
) -> list[str]:
    doc = fitz.open(pdf_path)
    Path(output_dir).mkdir(exist_ok=True)
    if pages is None:
        pages = list(range(len(doc)))

    zoom = dpi / 72
    mat = fitz.Matrix(zoom, zoom)
    saved = []

    for pn in pages:
        if pn >= len(doc):
            continue
        pix = doc[pn].get_pixmap(matrix=mat, alpha=False)
        img = Image.open(io.BytesIO(pix.tobytes("png")))
        w, h = img.size

        if max(w, h) > max_pixels:
            ratio = max_pixels / max(w, h)
            img = img.resize((int(w * ratio), int(h * ratio)), Image.Resampling.LANCZOS)

        filepath = str(Path(output_dir) / f"page_{pn + 1:03d}.png")
        img.save(filepath, "PNG")
        saved.append(filepath)

    doc.close()
    return saved

Exercise 4: Metadata and structure extractor

Create a function that returns a dictionary with: document metadata, table of contents, dimensions per page, and the number of tables and images detected per page.

import fitz
from pathlib import Path


def extract_full_structure(pdf_path: str) -> dict:
    doc = fitz.open(pdf_path)
    meta = doc.metadata

    metadata = {
        "title": meta.get("title", ""),
        "author": meta.get("author", ""),
        "creator": meta.get("creator", ""),
        "pages": len(doc),
        "size_kb": Path(pdf_path).stat().st_size / 1024
    }

    toc = [{"level": l, "title": t, "page": p} for l, t, p in doc.get_toc()]

    page_info = []
    for page in doc:
        rect = page.rect
        page_info.append({
            "page": page.number + 1,
            "width_pt": round(rect.width, 1),
            "height_pt": round(rect.height, 1),
            "orientation": "landscape" if rect.width > rect.height else "portrait",
            "tables_found": len(page.find_tables()),
            "images_found": len(page.get_images(full=True))
        })

    doc.close()
    return {"metadata": metadata, "table_of_contents": toc, "pages": page_info}

Additional Resources

1. PyMuPDF Documentation
2. PyMuPDF — Extracting Tables
3. pdf2image GitHub
4. Poppler — Rendering engine for pdf2image
5. PIL/Pillow — Image processing in Python