Module 3: Function Calling Patterns

1. Introduction: Function Calling Patterns

Overview

In Module 2 you learned to build tools: precise Pydantic schemas, the tool execution loop step by step, real external APIs with robust error handling, and InjectedToolArg for dependency injection. Your agent has capable hands. But having tools is not the same as knowing how to orchestrate them. A carpenter with a hammer, a saw and a level builds a table. An architect with the same tools builds a house. The difference isn't in the tools — it's in the patterns used to combine them.

This module teaches patterns. Not more individual tools, but advanced ways of orchestrating function calling that separate a demo agent from a production one. Parallel function calling to run multiple tools at the same time. Forced tool calls to steer the model exactly where you need it. Structured extraction to use schemas as a data extraction mechanism. Streaming so the user sees progress in real time. Tool composition to chain tools into pipelines. Retry patterns and circuit breakers so your agent survives when the real world fails. Each pattern solves a concrete problem that shows up when agents leave the notebook and enter the real world.

The analogy that defines this module: creating a tool (M2) is carpentry; orchestrating tools with patterns (M3) is architecture. A developer who only knows how to create tools builds agents that work in demos but fail in production — they don't handle parallel calls, they have no fallbacks when an API goes down, they show no progress to the user while they work, they don't extract structured data precisely. The patterns in this module are what turn that fragile agent into one an engineering team trusts to put in front of real users. When you finish, you close Phase 1 with a complete function calling toolkit that you'll apply in every remaining module of the guide.

Where are we in the guide?

Context

This guide has 10 modules organized into 3 phases:

Phase 1: Agent Foundations (Modules 1-3)
├── Module 01: Anatomy of an AI Agent            ✓ COMPLETED
├── Module 02: Tool Use Fundamentals             ✓ COMPLETED
└── Module 03: Function Calling Patterns         ← YOU ARE HERE (closes Phase 1)

Phase 2: Agent Architecture (Modules 4-7)
├── Module 04: State Machines for Agents
├── Module 05: Multi-Step Reasoning and Planning
├── Module 06: Memory Systems for Agents
└── Module 07: MCP and Advanced Tool Integration

Phase 3: Multi-Agent & Production (Modules 8-10)
├── Module 08: Multi-Agent Orchestration
├── Module 09: Testing and Evaluation of Agents
└── Module 10: Agents in Production and Alternatives

Where are you coming from?

You've built two foundational layers:

Module 1 — What an agent is: Perceive-reason-act cognitive architecture, agent taxonomy, decision framework (agent vs chain vs workflow), manual implementation of the ReAct loop
Module 2 — How you give it tools: @tool with complex Pydantic schemas, bind_tools with tool_choice, manual step-by-step tool execution loop, real external APIs, robust error handling, InjectedToolArg

Module 1 gave the agent a brain. Module 2 gave it hands. This module gives it technique — the patterns that turn individual movements into coordinated choreography.

Where are you headed?

This module closes Phase 1 (Agent Foundations). When you finish it, you'll have the complete function calling toolkit:

Module 1 — What an agent is → Architecture and decisions ✓
Module 2 — How to create tools → Solid, individual tools ✓
Module 3 — How to orchestrate tools → Production patterns ← HERE
Module 4 — How to control the flow → State machines with LangGraph (starts Phase 2)

The transition to Phase 2 is direct: "You have powerful tools and you know how to orchestrate them with patterns → now control how the agent decides when and in what order to use them." State machines with LangGraph (M4) give you that control. But without the M3 patterns, your state machines would have nodes that execute tools naively — no parallelism, no retry, no streaming.

From tools to patterns

In Module 2 you learned to answer questions like:

"How do I create a tool with a complex Pydantic schema?"
"How do I connect tools to the model with bind_tools?"
"What do I do when an API fails?"

Now the questions change:

"I have 5 tools and the user needs data from 3 of them. Do I run them one by one or all at once?"
"The model needs to extract 15 fields from a document. Do I use a prompt with structured output or function calling?"
"My agent takes 8 seconds to respond. How do I show progress to the user?"
"The weather API fails 5% of the time. What do I do — retry? fallback? circuit breaker?"
"I have a research tool that needs to search, filter and summarize. Do I make 3 separate tools or one composite tool?"
"Depending on the request type, only certain tools are relevant. How do I do routing?"

Each of those questions has an answer: a pattern. Not a generic recipe, but a proven pattern with a name, a problem it solves, a clear implementation, and explicit trade-offs.

The jump from M2 to M3 is the jump from knowing how to use individual tools to knowing how to combine them into systems. It's the difference between a musician who masters scales and one who improvises jazz — the vocabulary is the same, but the ability to combine is another league.

What M2 left you ready to do

With what you already master, you can:

Create a tool with a precise Pydantic schema and connect it to a model
Run the complete tool execution loop (manual or with a framework)
Integrate real external APIs with error handling
Inject context without exposing it to the model

What you can't do yet:

Run 5 simultaneous tool calls when the model requests them in parallel
Steer the model to a specific tool based on the request context
Extract 15 fields from a document precisely using function calling
Show progress to the user while the tools work
Compose tools into high-level pipelines
Handle repeated failures with backoff, circuit breaker and fallback

Every "I can't" on that list has a pattern that solves it. When you finish this module, each of those items moves to the "I can" list.

Why patterns deserve their own module

Most courses treat tool use as a single topic: "function calling" in one lesson that covers everything from @tool to parallel calls. This guide splits it into two modules because they are distinct disciplines. Creating a tool requires implementation thinking: correct schema, error handling, validation. Orchestrating tools with patterns requires design thinking: when to parallelize, when to compose, when to add resilience.

A developer who blends both disciplines in their head tends to create tools that try to do too much ("god object" tools that encapsulate all the orchestration) or patterns that are too simple (a generic retry for everything). Separating creation from orchestration forces you to think about each level with the depth it deserves.

The problem patterns solve

Patterns are not theoretical abstractions. Each one exists because there's a real scenario where the naive approach fails. Here are three concrete examples:

Scenario 1: Serial vs parallel

The problem: A user asks "What's the weather in Madrid, Buenos Aires and Tokyo?". Your agent has a get_weather tool. Without parallel calling, the agent calls the tool three times in series:

get_weather("Madrid")     → 2 seconds
get_weather("Buenos Aires") → 2 seconds
get_weather("Tokyo")      → 2 seconds
─────────────────────────────────────
Total: ~6 seconds

Six seconds for three independent queries. The user waits. The UX suffers.

With parallel function calling:

get_weather("Madrid")        ┐
get_weather("Buenos Aires")  ├→ 2 seconds (simultaneous)
get_weather("Tokyo")         ┘
─────────────────────────────────────
Total: ~2 seconds

Same result, a third of the time. The parallel calling pattern doesn't add new functionality — it adds speed where it matters.

Scenario 2: No retry vs resilient

The problem: Your agent uses a web search tool that depends on an external API. The API has 99% uptime — sounds excellent, until you run the numbers. If your agent processes 1,000 requests a day, ~10 will fail. Without retry, those 10 users get an error or an incomplete answer.

With retry pattern + circuit breaker:

The agent retries with exponential backoff. If the API keeps failing after 3 attempts, the circuit breaker trips: it stops trying (so as not to overload an API that's already in trouble) and uses a fallback — cached results, an alternative tool, or an honest message to the user. Of those 10 failures, 7 get resolved by retry and the remaining 3 degrade gracefully.

Scenario 3: No streaming vs good UX

The problem: Your agent receives a complex question. Internally it runs 4 tools in sequence: it searches for information, filters results, queries a database, and generates a summary. The whole process takes 12 seconds. During those 12 seconds, the user sees... nothing. A spinner. Silence. Is it thinking? Did it hang? Should I reload?

With tool call streaming:

The user sees, in real time: "Searching for information..." → "Filtering results (23 found)..." → "Querying database..." → "Generating summary...". The same 12 seconds, but the experience is completely different. The user knows what's happening and trusts that the system works.

The common thread

In all three cases, the core functionality is the same — the tools don't change. What changes is how they're orchestrated. That's a pattern: a proven way of organizing tool execution to solve a specific production problem.

The 6 patterns and the problems they solve

Pattern	Problem it solves	Without the pattern	With the pattern
Parallel calling	Unnecessary latency	3 calls × 2s = 6s serial	3 calls × 2s = 2s parallel
Forced calls / routing	Wrong tool for the context	The model chooses freely and sometimes gets it wrong	You steer the model to the right tool
Structured extraction	Unstructured data	JSON parsing from prompts (fragile)	Precise extraction via schemas
Streaming	Waiting UX	12 seconds of silence	12 seconds with visible progress
Tool composition	Complexity in the orchestrator	The model coordinates 5 tools step-by-step	A composite tool encapsulates the logic
Retry / circuit breaker	External API failures	One failure breaks the agent	Retry, fallback, clean degradation

Each pattern has its own capsule where you'll go deep into implementation, trade-offs, and when NOT to use it (because yes, there are scenarios where a pattern is over-engineering).

Prerequisites

From Module 1 (Anatomy of an AI Agent)

Perceive-reason-act cognitive architecture: You understand the fundamental loop — perceive, reason, act
The concept of a tool_call: You know the model doesn't execute tools directly — it returns an instruction that you execute
Decision framework: You know when an agent is the right solution vs a chain or a workflow

From Module 2 (Tool Use Fundamentals)

@tool with Pydantic schemas: You know how to create tools with descriptions, constraints, and nested models
bind_tools and tool_choice: You know how to connect tools to the model and control selection (auto, any, specific)
Tool execution loop: You implemented the manual loop and understand every step — user → model → tool_call → execute → ToolMessage → model
Error handling: You know how to handle network failures, timeouts and rate limits without crashing the agent
InjectedToolArg: You know how to inject context into tools without exposing it to the model

If any of these points doesn't feel solid, go back to the corresponding module. This module assumes creating and executing individual tools is something you already master. Here the focus is exclusively on orchestration patterns.

Tools for this module

Python 3.11+
langchain v1.2+ and langchain-openai
langgraph v1.0+
An OpenAI API key (GPT-4.1 or GPT-4.1-mini — both support parallel function calling)
Module 2 tools working (web search, weather, calculator)

pip install langchain langchain-openai langgraph tavily-python python-dotenv

You don't need to install new dependencies if you already completed the M2 setup. The patterns in this module use the same libraries — the difference is in how you combine them, not in which libraries you need.

Objectives for Module 3

By the end of this module you'll be able to:

✅ Implement parallel function calling: Configure the model to generate multiple tool_calls in a single response, execute them simultaneously with asyncio.gather, and return all results to the model in one step
✅ Use forced tool calls and routing: Configure tool_choice to force the model to use a specific tool, and implement dynamic routing that adjusts which tools are available based on context or request type
✅ Do structured extraction via function calling: Use tool schemas as an extraction mechanism — define a tool whose only purpose is to receive structured data from the model, turning function calling into a precise extraction engine
✅ Implement tool call streaming: Accumulate ToolCallChunks during generation, show progress to the user while the tools run, and handle the "working..." logic that turns long waits into good UX
✅ Design tool composition and chaining: Create tools that internally orchestrate other tools — a "research" tool that calls search + filter + summarize — without circular dependencies and with clean error propagation
✅ Implement retry patterns and circuit breakers: Exponential backoff with jitter, configurable max retries, a circuit breaker that stops attempts after N consecutive failures, fallback tools as an alternative, and per-tool timeouts
✅ Combine patterns in a real system: Integrate parallel calls + routing + extraction + retry into a single working system — because in production patterns are never used in isolation
✅ Choose the right pattern for each situation: Use a decision framework that maps concrete problems (latency, resilience, UX, precision) to the pattern that solves them, with explicit trade-offs

Module map

#	Capsule	What you'll learn
02	Parallel Function Calling	The model generates N simultaneous tool_calls, you execute them in parallel, you return all the results. When to use parallel vs serial. Trade-off: speed vs error handling complexity
03	Forced Tool Calls and Routing	tool_choice to force specific tools. Dynamic routing: change which tools are available based on context. The "classify first, act second" pattern
04	Structured Extraction via Function Calling	Using tool schemas as an extraction mechanism — not to execute actions, but to extract structured data. Comparison with native structured output
05	Streaming Tool Calls	Accumulating ToolCallChunks, progress UI, handling multiple tools while streaming. The difference between "12 seconds of silence" and "12 informative seconds"
06	Tool Composition and Chaining	Tools that orchestrate other tools. Tool pipelines. How to avoid circular dependencies. Error propagation in chains
07	Retry Patterns and Circuit Breakers	Exponential backoff + jitter, max retries, circuit breaker, fallback tools, per-tool timeout. The full spectrum of resilience
08	Project: Extraction + Routing	A complete system that receives documents, extracts entities with function calling, and routes each type to a specialized processor

Learning flow

The module follows a progression that goes from individual patterns to production combinations.

You start with parallel function calling (capsule 02): the pattern with the most immediate impact. If your agent makes 3 calls that could be simultaneous and runs them in series, you're tripling latency for no reason. You'll learn when the model generates multiple tool_calls and how to execute them with asyncio.gather.

Then forced tool calls and routing (capsule 03): sometimes you don't want the model to choose freely — you want to steer it to a specific tool or restrict its options based on context. This pattern is the foundation of the "classify first, act second" approach you'll use in the project.

With structured extraction (capsule 04), you'll discover that function calling isn't only for executing actions — it's a powerful extraction mechanism. Defining a tool whose only purpose is to receive entities extracted from text is more precise than asking the model to format JSON in a prompt.

Capsule 05 (streaming) solves the UX problem: when the agent works internally for several seconds, the user needs to see progress. Streaming tool calls turns a frustrating experience into an informative one.

Capsules 06 and 07 handle complexity and resilience. Tool composition (capsule 06) lets you create high-level tools that internally orchestrate other tools — tool pipelines that encapsulate complex logic. Retry patterns and circuit breakers (capsule 07) give your agent the ability to survive real-world failures: APIs down, timeouts, rate limits.

Finally, the project (capsule 08) integrates everything. You build an extraction + routing system that combines at least 3 patterns into a working flow — just like in production, where patterns are never used in isolation.

Connection with the project

This module's project: Extraction + Routing system

You'll build a system that shows function calling patterns working together:

Component	Pattern it uses	What it demonstrates
Entity extractor	Structured extraction	Receives documents, extracts people, companies, dates, amounts using tool schemas
Router	Forced tool calls + routing	Classifies each extracted entity and directs it to a specialized processor
Parallel processors	Parallel function calling	Multiple entities processed simultaneously
Resilience layer	Retry + circuit breaker	The system handles slow or down APIs without losing data

The flow is: input document → entity extraction → classification by type → routing to processors → aggregated results. Each step uses a specific pattern, and the complete system shows how they combine in reality.

Why extraction + routing as the project? Because it's a real use case that shows up in production constantly: document processing, email intake, contract analysis, form parsing. It isn't an academic exercise. It's something engineering teams build and deploy. And the difference between doing it with regex and heuristics vs doing it with function calling patterns is the difference between a fragile system that breaks with every new format and one that adapts naturally.

Estimated duration: 45-60 minutes.

Connection with the evolving project

The patterns in this module are fundamental to everything coming in Phase 2 and Phase 3:

Module 3 (now):       Individual patterns + extraction/routing system
     ↓
Module 4:             State machines use conditional routing (M3 pattern)
     ↓
Module 5:             Planning uses tool composition for task decomposition
     ↓
Module 6:             Memory with retry patterns for reliable persistence
     ↓
Module 7:             MCP with dynamic tool loading (dynamic routing)
     ↓
Module 8:             Multi-agent uses parallel execution + routing between agents
     ↓
Module 9:             Testing evaluates that patterns hold under stress
     ↓
Module 10:            Production deployment applies circuit breakers + streaming

Practically every later module uses at least one M3 pattern. That's not a coincidence — function calling patterns are the orchestration primitives on which more complex architectures are built. State machines (M4) control the flow of decisions, but M3 patterns control how the actions inside each node get executed. They're complementary layers.

If in Module 8 a sub-agent needs to query 3 information sources simultaneously, it will use parallel calling (M3). If in Module 10 a production API fails under load, the circuit breaker (M3) will protect the system. If in Module 5 the agent needs to extract subtasks from a plan, it will use structured extraction (M3). The patterns you learn here aren't disposable — they're permanent tools in your repertoire.

What this module does NOT cover

❌ State machines or flow control — How the agent decides what step to take next, conditional routing between nodes, execution cycles with LangGraph StateGraph. That's all of Module 4. Here you control how the tools execute; there you control when and in what order
❌ Planning or task decomposition — Breaking a complex task into subtasks, creating an execution plan, reflecting on intermediate results. That's Module 5. Here you orchestrate tools with patterns; there the agent reasons about which tools it needs and in what sequence
❌ Multi-agent orchestration — Multiple agents coordinating, supervisor patterns, handoffs between specialized agents. That's Module 8. Here you work with a single agent and its tools
❌ MCP (Model Context Protocol) — Exposing tools as standardized servers, dynamic tool discovery, the MCP ecosystem. That's Module 7. Here the patterns operate on direct tools; there you operate on MCP tools
❌ Creating individual tools — You already covered that in Module 2. Here we don't re-teach @tool, Pydantic schemas, or basic error handling. Here you use those tools as pre-built blocks and focus on how to combine them
❌ Formal testing of patterns — Unit testing, integration testing, trajectory evaluation of function calling patterns. That's Module 9. Here you validate that the patterns work; there you test them with formal frameworks

The boundary is clear: Module 2 = create individual tools. Module 3 = orchestrate them with patterns. Module 4+ = use them inside architectures with state, planning and multi-agent.

How to use this module

If you're coming straight from Module 2

You're on the ideal progression. You already master tool creation and the execution loop. Now each capsule presents a new pattern with a consistent structure: problem → solution → implementation → when to use → when NOT to use → trade-offs. Read the capsules in order — the patterns build on each other and the final project combines them.

If you've already used parallel function calling or structured extraction

The individual capsules may go quickly for you. Focus on:

Capsule 06 (Tool composition) — the least intuitive pattern and the most powerful in the long run
Capsule 07 (Retry + circuit breakers) — the full spectrum of resilience, not just "try again"
Capsule 08 (Project) — validates that you can combine 3+ patterns into a coherent system

If you have experience with agents in production

Go straight to the project (capsule 08). If you can build the extraction + routing system in 45 minutes combining parallel, retry and streaming, you already master the patterns. If not, identify which patterns you were missing and review those capsules.

Estimated time

Reading and practice: ~60-75 minutes (capsules 02-07)
Project: ~45-60 minutes (capsule 08)
Total: ~1.7-2.3 hours

Evidence of success

By the end of this module, you'll know you succeeded if:

✅ You can configure a model to generate multiple tool_calls in a single response and execute them in parallel — reducing latency proportionally to the number of independent calls
✅ You can implement dynamic routing: given a request type, you decide programmatically which subset of tools the model can use, and you can force a specific tool when the situation calls for it
✅ You can use function calling as an extraction mechanism — define a tool that doesn't execute actions but extracts structured data from text with precision superior to a JSON prompt
✅ You can implement tool call streaming that shows progress to the user in real time, distinguishing between "searching...", "processing..." and "generating response..."
✅ You can create a composite tool that orchestrates 2-3 tools internally without circular dependencies, with clean error propagation
✅ You can implement retry with exponential backoff + jitter + circuit breaker, and explain why a simple retry (with no backoff and no circuit breaker) can make the problem worse
✅ Your extraction + routing system combines at least 3 patterns working together in an end-to-end flow
✅ Faced with a production problem (high latency, unstable API, waiting UX, imprecise extraction), you identify which pattern applies and why

Quick self-assessment

Ask yourself these questions after completing the module:

"If a user asks for data from 5 independent sources, can I make my agent query them simultaneously instead of sequentially?" → If yes, you master parallel calling
"If I receive a long email and need to extract the sender, date, companies mentioned and amounts, can I do it with function calling instead of regex?" → If yes, you master structured extraction
"If the search API fails, does my agent retry with backoff, and if it keeps failing, trip a circuit breaker and use a fallback?" → If yes, you master retry patterns
"Can I explain, with concrete trade-offs, why parallel calling adds error handling complexity?" → If yes, you understand patterns at the design level, not just implementation

If you answered yes to all four → ready for Module 4 (State Machines for Agents) and Phase 2. If you answered no to any → review the corresponding capsule before moving on.

What this module closes

By completing M3, you'll have finished all of Phase 1 (Agent Foundations). This is what you'll have:

Module completed	What you master
M1: Anatomy of an AI Agent	What an agent is, how it thinks, what types exist, when to use them
M2: Tool Use Fundamentals	Creating solid tools, schemas, execution loop, real APIs, error handling
M3: Function Calling Patterns	Orchestrating tools with production patterns: parallel, routing, extraction, streaming, composition, retry

With this base, Phase 2 asks you to build on solid foundations. You're not "learning the basics and then the advanced stuff" — you're building capability layers. Every Phase 2 module assumes you can already create tools (M2) and orchestrate them (M3), and focuses on the next layer: flow control (M4), reasoning (M5), memory (M6), standardized integration (M7).

Summary

Patterns are what separate a demo agent from a production one: not new tools, but advanced ways of orchestrating the ones you already have
Creating a tool is carpentry; orchestrating tools with patterns is architecture — this module makes you a function calling architect
Each pattern solves a specific, real problem: parallel → latency, routing → precision, extraction → structured data, streaming → UX, composition → complexity, retry → resilience
Patterns combine in production: the project integrates extraction + routing + parallel + retry into a working system
This module closes Phase 1 (Agent Foundations): with M1 (concepts), M2 (tools), and M3 (patterns), you have the complete toolkit to enter Phase 2
Every later module (M4-M10) uses at least one M3 pattern: they're the orchestration primitives on which state machines, planning, MCP and multi-agent are built
The trade-offs are explicit: parallel = speed + complexity, retry = resilience + latency, streaming = UX + overhead, composition = encapsulation + coupling

The most important concept in this module

If you take away only one idea from all of M3, let it be this: patterns are design decisions, not technical features. Parallel calling isn't "a function you invoke". It's a design decision that says "these operations are independent, so I run them simultaneously, and I accept the added complexity of handling partial errors." Routing isn't "an if/else". It's a design decision that says "the model shouldn't have access to every tool at all times because that reduces its precision."

Every pattern comes with a cost. The skill isn't knowing the patterns — it's knowing when the benefit justifies the cost. That ability to choose with judgment is what distinguishes you as an engineer, and it's what this module aims to develop.

Resources

OpenAI Function Calling Guide — Official function calling documentation including parallel tool calls and tool_choice
LangChain Tool Calling How-To — Reference for tools, bind_tools, and tool calling patterns in LangChain
LangGraph Streaming — Streaming guide for LangGraph including tool call chunks
Circuit Breaker Pattern — Martin Fowler — The classic circuit breaker pattern, applicable to tool execution
Building Effective Agents — Anthropic — Anthropic's perspective on orchestration patterns for agents
Structured Outputs — OpenAI — Structured outputs vs function calling for extraction: when to use each approach

← Previous8. Project: Agent with 5 Real External Tools Next →2. Parallel Function Calling