Module 7: Use Cases

1. Introduction: Use Cases

Description

This is the first capsule of Module 7 of the Multimodal AI Guide. Here you're going to understand how the individual pieces you learned in the previous modules — vision, documents, image generation, audio, and RAG — combine into design patterns that solve real problems. It's not about learning new APIs, but about applying what you already know in architectures that work in production.

Why it matters: Modules 1-6 gave you individual tools: sending an image to GPT-4o, transcribing audio with Whisper, indexing documents with embeddings. But in the real world, nobody uses a single tool. A document analysis system combines text extraction, vision for diagrams, RAG for search, and generation for answers. A meeting assistant combines audio transcription, presentation analysis, and text summarization. This module teaches you to design those combinations systematically, not improvised.

Connection with the final project: This module's project — the Use Case Selector — is the "brain" of the Document Analyzer you'll build in Module 8. It receives an input (PDF, image, audio), detects its type, and applies the correct pipeline. It's the routing piece that connects all the multimodal capabilities into a single system.

Where Are We in the Guide?

Context

This guide has 8 modules organized into 3 phases:

Phase 1: Multimodal Fundamentals (Modules 1-3)
├── Module 1: Introduction to Multimodal AI
├── Module 2: Vision + LLMs
└── Module 3: Document Understanding

Phase 2: Generation and Audio (Modules 4-5)
├── Module 4: Image Generation
└── Module 5: Audio Processing

Phase 3: RAG, Use Cases and Project (Modules 6-8)
├── Module 6: Multimodal RAG
├── Module 7: Use Cases                ← YOU ARE HERE
└── Module 8: Final Project — Multimodal Document Analyzer

Total estimated duration: 6-7 hours (self-paced).

Where are we headed?

The previous modules gave you the pieces:

Module 1 — You understood the landscape: what models exist, what modalities they support, how to send data
Module 2 — You mastered vision: image analysis with GPT-4o, Claude 3, Gemini
Module 3 — You processed documents: PDFs, OCR, structured extraction
Module 4 — You generated images: DALL-E 3, variations, editing
Module 5 — You processed audio: Whisper, TTS, voice pipelines
Module 6 — You built multimodal RAG: embeddings, indexing, retrieval over text + images

Now, in Module 7, you take all of that and apply it in 4 design patterns that cover 90% of real use cases:

Document Q&A — Questions and answers over documents
Automated Image Analysis — Classification and batch analysis
Video Analysis via Frames — Video analysis by extracting frames
Multi-Modality Combinations — Pipelines that mix text, image, and audio

In addition, you'll learn production patterns (caching, batching, costs) and troubleshooting specific to these use cases.

Module 8 takes the Use Case Selector you build here and integrates it into the complete Document Analyzer.

From Pieces to Patterns

The problem

Knowing individual APIs isn't enough. Knowing how to call GPT-4o with an image, or transcribe audio with Whisper, are atomic skills. The real challenge is combining them into coherent flows that solve business problems:

Problem: "I want my users to be able to ask questions about PDFs"

Pieces needed:
├── Text extraction (Module 3)
├── Image extraction (Module 3)
├── Chunking and embeddings (Module 6)
├── Similarity retrieval (Module 6)
├── Answer generation (Module 1-2)
└── Source citation (new in this module)

Pattern: Document Q&A
Pipeline: Document → Extract → Index → Query → Retrieval → LLM → Answer with sources

The solution: design patterns

A design pattern in multimodal AI is a proven sequence of steps that transforms an input into a useful output. It's not reinventing every time, but reusing architectures that work.

Just as in software there are patterns like MVC, Observer, or Strategy, in multimodal AI there are patterns like:

Pattern	Input	Pipeline	Output	Modules used
Document Q&A	PDF + question	Extract → RAG → LLM	Answer with sources	M3, M6, M1
Image Analysis	Image(s)	Vision → classification/extraction	Structured data	M2, M1
Video Analysis	Video	Frames → Vision → LLM	Temporal analysis	M2, M1
Multi-Modal	Text + image + audio	Combine pipelines	Integrated answer	M1-M6
Production	Any input	Caching + batching + retry	Optimized result	Cross-cutting

Each pattern has its dedicated capsule in this module, with complete implementation, troubleshooting, and exercises.

Module Objectives

By the end of this module you'll be able to:

Technical skills

Build a Document Q&A pipeline — Load document, extract content, index, retrieve relevant chunks, generate an answer with cited sources
Automate batch image analysis — Classify, extract data, and process image catalogs using Vision APIs
Analyze video via frames — Extract key frames, send them to Vision, combine analyses into a temporal summary
Design multi-modal pipelines — Combine text + image + audio into coherent flows (meetings, documents, content)
Apply production patterns — Caching, batching, rate limit management, cost optimization
Diagnose pipeline problems — Identify and resolve common errors in each type of use case

Design skills

Choose the right pattern — Given a business problem, select the appropriate design pattern
Build a pipeline router — The Use Case Selector that detects input type and applies the corresponding pipeline

Depth level

This module doesn't repeat what you already saw. It assumes you know how to:

Send images to Vision APIs (M2)
Extract text from PDFs (M3)
Generate images with DALL-E (M4)
Transcribe audio with Whisper (M5)
Build indexes with embeddings (M6)

What's new here is how to combine those pieces into systems that work end-to-end.

Module 7 Roadmap

#	Capsule	Type	What you'll build	Duration
01	Introduction (this one)	Context	Module map, objectives	5 min
02	Document Q&A	Technical	Complete Q&A pipeline with sources	8 min
03	Automated Image Analysis	Technical	Classification and batch analysis	8 min
04	Video: Frames + LLM	Technical	Frame extraction and analysis	8 min
05	Multi-Modality Combinations	Technical	Text + image + audio pipelines	8 min
06	Production Patterns	Technical	Caching, batching, costs	8 min
07	Use Case Troubleshooting	Technical	Diagnosis and resolution	6 min
08	Project: Use Case Selector	Project	Pipeline router	12 min

Total estimated duration: 60-65 min

Recommended flow

Capsules 02-04: Individual patterns (Document Q&A, Image, Video)
    ↓
Capsule 05: Combinations (integrate the previous patterns)
    ↓
Capsule 06: Production (make the pipelines robust)
    ↓
Capsule 07: Troubleshooting (solve problems)
    ↓
Capsule 08: Project (Use Case Selector that unites them all)

Real-World Use Cases

To motivate what's coming, these are concrete examples of each pattern in production:

Document Q&A

Scenario: A legal platform lets lawyers ask questions about 200-page contracts.

Input: PDF of the contract + question "What is the penalty clause?"
Pipeline: Extract → Chunk → Index → Search relevant chunks → LLM generates an answer citing pages
Output: "The penalty clause is in section 12.3 (page 45)..." + source chunks

Automated Image Analysis

Scenario: An e-commerce receives 500 product photos a day and needs to classify them and extract attributes.

Input: Batch of 500 images
Pipeline: For each image → Vision API → Classify category + extract color/size/material
Output: CSV with category, color, size, material for each product

Video Analysis

Scenario: An educational platform analyzes class videos to generate summaries and detect key moments.

Input: 45-minute video
Pipeline: Extract a frame every 30 seconds → Vision describes each frame → LLM identifies topics and transitions → Summary
Output: "0:00-5:00: Introduction to topic X, 5:00-15:00: Demonstration of Y..."

Multi-Modality Combinations

Scenario: A meeting notes system takes the recording of a meeting + the slides presented.

Input: Meeting audio + PDF of the presentation
Pipeline: Transcribe audio → Extract text and images from the PDF → Combine contexts → LLM generates minutes
Output: Meeting minutes that connect what was discussed with the slides shown

Prerequisites

Required knowledge (from previous modules)

Module	Key concept	Why you need it here
M1	Multimodal APIs, formats	Base for all calls
M2	Vision: GPT-4o, Claude 3	Image analysis and video frames
M3	Document extraction	Document Q&A pipeline
M4	DALL-E 3, generation	Multi-modal combinations
M5	Whisper, TTS	Audio in pipelines
M6	Embeddings, ChromaDB, RAG	Retrieval in Document Q&A

Tools

openai >= 1.0         # API calls (GPT-4o, Whisper, DALL-E, TTS, embeddings)
anthropic >= 0.20     # Claude 3 (optional, for fallback)
pymupdf >= 1.24       # PDF extraction
chromadb >= 0.4       # Vector store for RAG
opencv-python >= 4.8  # Video frame extraction
Pillow >= 10.0        # Image processing

Quick setup

pip install openai anthropic pymupdf chromadb opencv-python Pillow

import os
from openai import OpenAI

client = OpenAI()

assert os.environ.get("OPENAI_API_KEY"), "Configure OPENAI_API_KEY"
print("Setup OK — ready for Module 7")

What This Module Does NOT Cover

To keep the focus, this module does not include:

Topic	Where to see it
Fundamentals of each API	Modules 1-5
Building RAG indexes from scratch	Module 6
Deployment to production (Docker, cloud)	Out of scope for this guide
Model fine-tuning	Out of scope for this guide
Response streaming	Module 8 uses it, but doesn't explain it in detail
User interfaces (frontends)	Out of scope for this guide

This module assumes you already know how to use the individual APIs and focuses on how to combine them into functional systems.

Self-Assessment

Before starting, check that you can answer these questions. If any one isn't clear, review the indicated module:

Level 1: Concepts (you should answer without code)

What is RAG and why is it useful for Document Q&A? → M6
Which models support simultaneous image and text input? → M1, M2
What does Whisper do and in what format does it return results? → M5
What is an embedding and what is it for? → M6
What is the difference between gpt-4o and gpt-4o-mini in terms of cost and capability? → M1

Level 2: Code (you should be able to write it)

Send an image + question to GPT-4o and get an answer → M2
Extract text from a PDF with PyMuPDF → M3
Transcribe an audio file with Whisper → M5
Create embeddings and search by similarity with ChromaDB → M6
Generate an image with DALL-E 3 from a prompt → M4

Level 3: Design (what you'll learn here)

Design a pipeline that combines extraction + RAG + generation — capsule 02
Process 100 images in batch with retry and rate limits — capsule 03
Extract frames from a video and analyze them — capsule 04
Combine audio + document in a single flow — capsule 05
Implement caching and cost optimization — capsule 06

If questions 1-10 seem clear to you, you're ready. If not, review the indicated modules before continuing.

Conventions of This Module

Code

All the code is Python 3.10+
It uses openai >= 1.0 (new API with client.chat.completions.create)
The code blocks are complete and runnable — not loose fragments
Each technical capsule includes at least one complete implementation that you can copy and adapt

Structure of each technical capsule

1. Description and context
2. Visual pipeline (ASCII diagram)
3. Step-by-step implementation
4. Complete implementation
5. Variations and extensions
6. Troubleshooting (common errors)
7. Exercises (3-4 per capsule)
8. Additional resources

Structure of the project capsule

1. Description and specifications
2. Step-by-step implementation (6 steps)
3. Optional extensions
4. Troubleshooting
5. Completeness checklist
6. Extension exercises

General Module Architecture

This diagram shows how the capsules connect to each other and to the final project:

                    ┌─────────────────┐
                    │  02: Document   │
                    │     Q&A         │───────────┐
                    └─────────────────┘           │
                    ┌─────────────────┐           │
                    │  03: Image      │           │
                    │  Analysis       │───────────┤
                    └─────────────────┘           │
                    ┌─────────────────┐           ▼
                    │  04: Video      │    ┌──────────────┐
                    │  Frames + LLM   │───▶│ 08: Project  │
                    └─────────────────┘    │  Use Case    │
                    ┌─────────────────┐    │  Selector    │
                    │  05: Multi-     │───▶│              │
                    │  Modality       │    └──────────────┘
                    └─────────────────┘           ▲
                    ┌─────────────────┐           │
                    │  06: Production │───────────┤
                    │  Patterns       │           │
                    └─────────────────┘           │
                    ┌─────────────────┐           │
                    │  07: Trouble-   │───────────┘
                    │  shooting       │
                    └─────────────────┘

Each technical capsule (02-05) defines a design pattern. Capsule 06 adds robustness. 07 solves problems. 08 integrates everything into a router.

Technical Setup

Before starting the technical capsules, check that your environment is ready:

import os
from openai import OpenAI

client = OpenAI()
assert os.environ.get("OPENAI_API_KEY"), "Missing OPENAI_API_KEY"

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Reply OK if you receive this."}],
    max_tokens=10
)
print(f"LLM: {response.choices[0].message.content}")

emb = client.embeddings.create(
    model="text-embedding-3-small",
    input=["test"]
)
print(f"Embeddings: {len(emb.data[0].embedding)} dimensions")

try:
    import fitz
    print(f"PyMuPDF: {fitz.version}")
except ImportError:
    print("MISSING: pip install pymupdf")

try:
    import chromadb
    print(f"ChromaDB: OK")
except ImportError:
    print("MISSING: pip install chromadb")

try:
    import cv2
    print(f"OpenCV: {cv2.__version__}")
except ImportError:
    print("MISSING: pip install opencv-python")

try:
    from PIL import Image
    print(f"Pillow: OK")
except ImportError:
    print("MISSING: pip install Pillow")

print("\nSetup complete — ready for Module 7")

If everything prints without errors, you're ready. If a dependency is missing, install it before continuing.

Evidence of Success

By the end of this module, you'll know you succeeded if:

✅ You can design a Document Q&A pipeline that combines extraction (M3), RAG (M6), and LLM generation
✅ You can process a batch of 100 images with retry, rate limiting, and progress tracking
✅ You can extract frames from a video, analyze them with the Vision API, and generate a summary
✅ You know how to combine multiple modalities (audio + document + image) into a single flow
✅ You implement production patterns: caching, circuit breaker, cost tracking, fallbacks
✅ Your Use Case Selector works: it receives an input, detects its type, and applies the correct pipeline

Summary

Use Cases is the module that applies everything learned in M1-M6 to design patterns for real problems.
The 4 main patterns are: Document Q&A (extraction + RAG + generation), Automated Image Analysis (batch + retry + classification), Video Frames (extraction + temporal analysis), and Multi-Modality Combinations (audio + document + image in a single flow).
The production patterns include: multi-level caching, rate limiting with token bucket, circuit breaker, granular cost tracking, and multi-provider fallback.
The module's project is a Use Case Selector that detects the input type and routes to the correct pipeline — it's the "brain" of Module 8's Document Analyzer.

Quick Glossary

Term	Definition
Pipeline	Sequence of steps that transforms an input into an output
Routing	Deciding which pipeline to apply based on the input type
Batch	Processing multiple items in a single operation
Rate limit	Limit on API calls per unit of time
Fallback	Alternative when the primary method fails
Circuit breaker	Pattern that stops calls to a service that is failing repeatedly
TTL	Time To Live — how long a cache is valid
Embedding	Numerical representation of text for similarity search
Retrieval	Retrieving relevant information from an index
RAG	Retrieval-Augmented Generation — combining search with generation

Additional Resources

OpenAI Best Practices — Official usage guides
Anthropic Production Patterns — Production patterns with Claude
LangChain Use Cases — Reference implementations
ChromaDB Documentation — Vector store for RAG
OpenCV Documentation — Video processing

← Previous8. Project: Basic Multimodal RAG Next →2. Document Q&A