7. Comparing Reasoning Patterns

Overview

You already know the three reasoning patterns that define how an agent thinks. In capsule 02, you opened up the ReAct loop and saw how the agent interleaves reasoning with action on each iteration — fast, flexible, but with no global plan. In capsule 03, you separated planning from execution with Plan-and-Execute — the agent first decomposes the task and then executes each step, with the ability to re-plan when something fails. In capsule 05, you added reflection — the agent evaluates its own output with specific critique prompts and corrects itself before delivering.

Knowing each pattern separately isn't enough. The question that matters in production isn't "how does ReAct work?" but "which pattern do I use for THIS problem?" And that question isn't answered with intuition — it's answered with data. Latency, cost, number of tool calls, result quality. Concrete trade-offs that shift with the task's complexity, your latency tolerance, and your token budget.

This capsule takes the same use case and solves it with all three patterns. It measures everything. It gives you a comparison table and a decision framework you can use on any project. At the end, you'll also see how the patterns combine — because in production, the best agents don't use a single pattern, they use the right pattern for each phase of the task.

The Same Problem, Three Approaches

The use case: a Research Report

For the comparison to be valid, you need a use case complex enough that the differences between patterns are visible, but bounded enough that you can measure it. Here's the case:

Task: "Generate a report on the current state of AI Agents in production. Include: (1) the main frameworks and their adoption, (2) the most common architecture patterns, (3) deployment challenges, and (4) trends for the next 12 months. Cite sources."

This task has 4 interdependent sub-questions, requires multiple searches, needs synthesis of sources, and has a clear quality criterion (completeness, cited sources, coherence). It's the kind of task where the difference between patterns shows.

Shared setup

All three approaches use exactly the same tools, the same model, and the same base configuration:

import time, json
from typing import TypedDict, Annotated
from langchain.chat_models import init_chat_model
from langchain_core.tools import tool
from langchain_core.messages import BaseMessage, HumanMessage
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages

model = init_chat_model("openai:gpt-4.1-mini")

@tool
def web_search(query: str) -> str:
    """Search the web for current information."""
    from tavily import TavilyClient
    results = TavilyClient().search(query, max_results=3)
    return "\n".join(f"- {r['title']}: {r['content'][:200]}" for r in results["results"])

@tool
def get_page_content(url: str) -> str:
    """Get the full content of a specific web page."""
    from tavily import TavilyClient
    result = TavilyClient().extract(urls=[url])
    return result["results"][0]["raw_content"][:2000] if result["results"] else "No content"

tools = [web_search, get_page_content]

Metrics instrumentation

To measure consistently, you use a wrapper that captures everything:

from dataclasses import dataclass

@dataclass
class RunMetrics:
    tool_calls: int = 0
    llm_calls: int = 0
    total_tokens: int = 0
    start_time: float = 0
    end_time: float = 0
    
    @property
    def latency_seconds(self) -> float:
        return self.end_time - self.start_time
    
    @property
    def estimated_cost(self) -> float:
        return (self.total_tokens / 1_000_000) * 0.40

And a standardized quality evaluation function that scores the output on 4 dimensions (1-10):

def evaluate_quality(report: str, task: str) -> dict:
    eval_prompt = f"""Evaluate this report. Task: {task}
Report: {report}

Score these dimensions 1-10: completeness, depth, sources, coherence.
JSON: {{"completeness": N, "depth": N, "sources": N, "coherence": N}}"""

    response = model.invoke([HumanMessage(content=eval_prompt)])
    scores = json.loads(response.content)
    scores["average"] = round(sum(scores.values()) / 4, 1)
    return scores

ReAct: Results and Metrics

Implementation

The ReAct approach is direct: the agent takes the task and solves everything in a reactive loop. There's no plan up front and no evaluation afterward — the model decides on each iteration whether it needs to search more or whether it can already answer.

from langgraph.prebuilt import create_react_agent

react_agent = create_react_agent(
    model,
    tools,
    prompt="You are an expert researcher. Generate complete reports "
           "with cited sources. Be exhaustive in your research."
)

task = (
    "Generate a report on the current state of AI Agents in production. "
    "Include: (1) the main frameworks and their adoption, "
    "(2) the most common architecture patterns, "
    "(3) deployment challenges, and (4) trends for the next 12 months. "
    "Cite sources."
)

metrics = RunMetrics()
metrics.start_time = time.time()

result = react_agent.invoke(
    {"messages": [HumanMessage(content=task)]},
    config={"recursion_limit": 25}
)

metrics.end_time = time.time()
react_report = result["messages"][-1].content

How ReAct tackles the task

Look at the typical trace of a ReAct run for this task:

Iteration 1: Thought(implicit) → web_search("AI agents in production 2025 2026")
Iteration 2: Thought(implicit) → web_search("AI agent frameworks LangGraph CrewAI")
Iteration 3: Thought(implicit) → web_search("AI agent deployment challenges production")
Iteration 4: Thought(implicit) → web_search("AI agent architecture patterns")
Iteration 5: Thought(implicit) → Generates the final answer

The agent makes ad-hoc searches, one at a time. It doesn't decompose the task before starting — it discovers what it needs as it goes. Sometimes this works well; other times, the agent forgets a sub-topic or repeats similar searches because it has no visibility into the complete plan.

Typical results

┌──────────────────────────────────────────────┐
│ ReAct — Results                              │
├──────────────────┬───────────────────────────┤
│ Tool calls       │ 4-6                       │
│ LLM calls        │ 5-7                       │
│ Total tokens     │ ~8,000-12,000             │
│ Latency          │ 15-25s                    │
│ Estimated cost   │ ~$0.004                   │
├──────────────────┼───────────────────────────┤
│ Completeness     │ 6/10                      │
│ Depth            │ 5/10                      │
│ Sources          │ 5/10                      │
│ Coherence        │ 7/10                      │
│ Average quality  │ 5.8/10                    │
└──────────────────┴───────────────────────────┘

Analysis

Strengths:

• Fast. It's the approach with the lowest latency because there's no planning or evaluation overhead.
• Cheap. Fewer LLM calls = fewer tokens = less cost.
• Adaptable. If a search returns unexpectedly good results, the agent can pivot immediately.

Weaknesses:

• Inconsistent coverage. The agent frequently covers 3 of 4 sub-topics, forgetting one. Without an explicit plan, there's no checklist to guarantee completeness.
• Redundant searches. With no visibility into what it already searched vs what's missing, it sometimes repeats similar queries.
• Shallow depth. With no research strategy, the agent tends to take the first result and move on.
• No self-evaluation. The agent delivers the first draft without checking whether it meets the task's criteria.

Plan-and-Execute: Results and Metrics

Implementation

The Plan-and-Execute approach separates planning from execution. First the agent generates a research plan, then it executes each step of the plan with tools, and at the end it synthesizes the results. If a step fails, the agent can re-plan.

from langgraph.prebuilt import create_react_agent

class PlanExecuteState(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]
    task: str
    plan: dict | None
    step_results: list[str]
    current_step: int
    final_report: str

def plan_node(state: PlanExecuteState) -> dict:
    planner_prompt = f"""Break this task into 4-6 research steps.
Each step with concrete search queries. Answer in JSON:
{{"steps": [{{"description": "...", "search_queries": ["..."]}}], 
  "synthesis_instructions": "..."}}

Task: {state['task']}"""
    
    response = model.invoke([HumanMessage(content=planner_prompt)])
    return {"plan": json.loads(response.content), "current_step": 0, "step_results": []}

def execute_step_node(state: PlanExecuteState) -> dict:
    step = state["plan"]["steps"][state["current_step"]]
    executor = create_react_agent(
        model, tools,
        prompt=f"Research: {step['description']}. Suggested queries: {step['search_queries']}"
    )
    result = executor.invoke({"messages": [HumanMessage(content=step["description"])]})
    new_results = state["step_results"] + [f"## {step['description']}\n{result['messages'][-1].content}"]
    return {"step_results": new_results, "current_step": state["current_step"] + 1}

def should_continue_steps(state: PlanExecuteState) -> str:
    return "synthesize" if state["current_step"] >= len(state["plan"]["steps"]) else "execute_step"

def synthesize_node(state: PlanExecuteState) -> dict:
    all_research = "\n\n".join(state["step_results"])
    synth_prompt = f"""Synthesize this into a coherent report. Cite sources.
Task: {state['task']}
Research: {all_research}"""
    response = model.invoke([HumanMessage(content=synth_prompt)])
    return {"final_report": response.content}

graph = StateGraph(PlanExecuteState)
graph.add_node("plan", plan_node)
graph.add_node("execute_step", execute_step_node)
graph.add_node("synthesize", synthesize_node)
graph.add_edge(START, "plan")
graph.add_edge("plan", "execute_step")
graph.add_conditional_edges("execute_step", should_continue_steps,
    {"execute_step": "execute_step", "synthesize": "synthesize"})
graph.add_edge("synthesize", END)
plan_execute_agent = graph.compile()

How Plan-and-Execute tackles the task

PLANNING → A 5-step plan: frameworks, patterns, deployment, trends, metrics
EXECUTION → Each step executed by a ReAct sub-agent (1-2 searches per step)
SYNTHESIS → Combines the results into a structured report

The key structural difference: each sub-topic has a dedicated step. You can't forget a sub-topic because it's explicitly in the list.

Typical results

┌──────────────────────────────────────────────┐
│ Plan-and-Execute — Results                   │
├──────────────────┬───────────────────────────┤
│ Tool calls       │ 8-12                      │
│ LLM calls        │ 10-14                     │
│ Total tokens     │ ~18,000-25,000            │
│ Latency          │ 35-55s                    │
│ Estimated cost   │ ~$0.009                   │
├──────────────────┼───────────────────────────┤
│ Completeness     │ 9/10                      │
│ Depth            │ 7/10                      │
│ Sources          │ 7/10                      │
│ Coherence        │ 8/10                      │
│ Average quality  │ 7.8/10                    │
└──────────────────┴───────────────────────────┘

Analysis

Strengths:

• High completeness. The plan guarantees that every sub-topic gets researched. It's hard to forget something that's on an explicit list.
• Clear debugging. If the report falls short on "deployment challenges", you know exactly which step to check.
• Re-planning is possible. If step 3 fails (API down, no results), you can re-plan only the remaining steps.
• Predictable cost. The number of steps in the plan gives you an estimate of tool calls before you execute.

Weaknesses:

• Significantly higher latency. The planning LLM call adds 3-5 seconds, and having a ReAct sub-agent per step multiplies the calls.
• ~2x the cost. More LLM calls for the planner, the executors, and the synthesizer.
• A potentially rigid plan. If during step 2's execution you discover information relevant to step 4, the agent doesn't adjust — it executes linearly.
• No quality evaluation. The synthesizer generates the report and delivers it without checking whether it meets the criteria.

Plan + Reflection: Results and Metrics

Implementation

This approach combines Plan-and-Execute with a reflection step after the synthesis. The agent plans, executes, synthesizes, and then evaluates its own output. If the evaluation finds problems, the agent re-plans and iterates — but only over the identified gaps, not the whole task.

class PlanReflectState(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]
    task: str
    plan: dict | None
    step_results: list[str]
    current_step: int
    final_report: str
    reflection: dict | None
    iteration: int

def reflect_node(state: PlanReflectState) -> dict:
    reflect_prompt = f"""Evaluate this critically. Task: {state['task']}
Report: {state['final_report']}

Score 1-10: COMPLETENESS, DEPTH, SOURCES, COHERENCE.
If ALL >= 7: ACCEPTABLE. If not, identify the gaps.
JSON: {{"scores": {{...}}, "acceptable": bool, "gaps": [...], "improvement_queries": [...]}}"""
    
    reflection = json.loads(model.invoke([HumanMessage(content=reflect_prompt)]).content)
    return {"reflection": reflection, "iteration": state.get("iteration", 0) + 1}

def should_accept_or_improve(state: PlanReflectState) -> str:
    if state["reflection"]["acceptable"] or state.get("iteration", 0) >= 3:
        return "done"
    return "improve"

def improve_node(state: PlanReflectState) -> dict:
    gaps = state["reflection"]["gaps"]
    queries = state["reflection"]["improvement_queries"]
    improve_prompt = f"""Gaps: {gaps}. Queries: {queries}.
Research the gaps and improve this report: {state['final_report']}"""
    
    executor = create_react_agent(model, tools)
    result = executor.invoke({"messages": [HumanMessage(content=improve_prompt)]})
    return {"final_report": result["messages"][-1].content}

# Graph: plan → execute_step (loop) → synthesize → reflect → (improve → reflect) | done
graph = StateGraph(PlanReflectState)
graph.add_node("plan", plan_node)          # Reuses the planner
graph.add_node("execute_step", execute_step_node)
graph.add_node("synthesize", synthesize_node)
graph.add_node("reflect", reflect_node)
graph.add_node("improve", improve_node)
graph.add_edge(START, "plan")
graph.add_edge("plan", "execute_step")
graph.add_conditional_edges("execute_step", should_continue_steps,
    {"execute_step": "execute_step", "synthesize": "synthesize"})
graph.add_edge("synthesize", "reflect")
graph.add_conditional_edges("reflect", should_accept_or_improve,
    {"done": END, "improve": "improve"})
graph.add_edge("improve", "reflect")
plan_reflect_agent = graph.compile()

How Plan + Reflection tackles the task

PLAN + EXECUTE → (Identical to Plan-and-Execute: 5 steps, 8-12 tool calls)
SYNTHESIZE    → The report's first draft
REFLECT #1    → completeness=8, depth=6, sources=6, coherence=8 → NOT ACCEPTABLE
                Gaps: "depth on patterns", "few sources cited"
IMPROVE       → 2 additional searches focused on the gaps → improved report
REFLECT #2    → completeness=9, depth=8, sources=8, coherence=8 → ACCEPTABLE ✓

The reflection identified concrete gaps, suggested specific queries, and the improvement researched only what was missing — it didn't repeat the whole task.

Typical results

┌──────────────────────────────────────────────┐
│ Plan + Reflection — Results                  │
├──────────────────┬───────────────────────────┤
│ Tool calls       │ 12-18                     │
│ LLM calls        │ 14-22                     │
│ Total tokens     │ ~28,000-40,000            │
│ Latency          │ 55-90s                    │
│ Estimated cost   │ ~$0.015                   │
├──────────────────┼───────────────────────────┤
│ Completeness     │ 9/10                      │
│ Depth            │ 8/10                      │
│ Sources          │ 8/10                      │
│ Coherence        │ 8/10                      │
│ Average quality  │ 8.3/10                    │
└──────────────────┴───────────────────────────┘

Analysis

Strengths:

• Higher, more consistent quality. Reflection catches gaps that the other approaches hand straight to the user.
• Targeted improvement. The improvement only researches what's missing, it doesn't repeat the whole task. That's more efficient than iterating the entire loop.
• Self-awareness. The agent knows when its output is good and when it isn't. That's crucial for tasks where quality matters more than speed.
• An explicit quality gate. You can configure the threshold: for internal reports, >= 6; for client-facing reports, >= 8.

Weaknesses:

• ~2-4x the latency vs ReAct. Reflection adds evaluation LLM calls, and if it doesn't pass the quality gate, it adds a full improvement round.
• ~3-4x the cost vs ReAct. More tokens in the evaluation, more searches in the improvement.
• Risk of a loop. If the quality gate is too strict, the agent can iterate 3 times without reaching it. max_iterations is a necessary guardrail.
• Diminishing returns. The difference between the first and second reflection iteration is usually significant (+1.5 points on average). The third iteration rarely improves more than +0.3.

Head-to-Head Comparison

The complete table

Visualizing the trade-off

The quality/cost curve flattens after Plan-and-Execute:

• ReAct → Plan-Exec: +2.0 quality for +$0.005 (high ROI)
• Plan-Exec → Reflect: +0.5 quality for +$0.006 (low ROI)

What the data says

1. Plan-and-Execute has the best ROI for most tasks. It doubles ReAct's quality for ~2x the cost. Completeness jumps from 6 to 9 — that's the difference between a report that forgets a sub-topic and one that covers them all.

2. Reflection has diminishing returns. It adds +0.5 average quality points for ~1.7x the cost of Plan-and-Execute. It's only justified when quality must be high (client reports, critical decisions, published content).

3. ReAct wins when latency matters more than completeness. Chatbots, quick answers, simple tasks with 1-2 sub-questions. Not everything needs a plan.

4. Completeness is the biggest jump. The most important difference between ReAct and Plan-and-Execute isn't depth or coherence — it's completeness. An explicit plan is an implicit checklist.

Decision Framework

Decision diagram

Use this flowchart to pick the pattern before you implement:

Does the task have more than 3 independent sub-parts?
│
├── NO → Does it need an answer in < 5 seconds?
│         ├── YES → ReAct
│         └── NO → Is the output's quality critical?
│                   ├── NO → ReAct
│                   └── YES → ReAct + Reflection only on the final output
│
└── YES → Can you tolerate > 30 seconds of latency?
          ├── NO → ReAct (accept lower completeness)
          └── YES → Does quality need to be > 8/10?
                    ├── NO → Plan-and-Execute
                    └── YES → Plan + Reflection

Practical rules

Five heuristics you can memorize:

Decision by application type

Combining Patterns

Why the patterns are composable

The three patterns aren't mutually exclusive — they're layers you can combine. In production, the best agents use hybrid approaches:

• Plan-and-Execute uses a ReAct sub-agent to execute each step of the plan
• Reflection can be added to any pattern as a final step
• A router can pick the pattern based on the task's complexity

Hybrid 1: Complexity router

The most practical pattern: a classifier that analyzes the task and picks the right approach.

def complexity_router(state: dict) -> str:
    """Classify the task's complexity and pick a pattern."""
    task = state["task"]
    
    classification_prompt = f"""Classify this task's complexity:

Task: {task}

Criteria:
- SIMPLE: 1-2 sub-questions, a direct answer, < 3 tool calls
- MODERATE: 3-5 sub-questions, needs structure, 3-8 tool calls
- COMPLEX: 5+ sub-questions, needs deep research, quality is critical

Answer with ONLY: SIMPLE, MODERATE, or COMPLEX"""
    
    response = model.invoke([HumanMessage(content=classification_prompt)])
    complexity = response.content.strip().upper()
    
    routing = {
        "SIMPLE": "react_agent",
        "MODERATE": "plan_execute_agent",
        "COMPLEX": "plan_reflect_agent",
    }
    return routing.get(complexity, "plan_execute_agent")

This router adds ~1 second and ~200 tokens of overhead, but it saves significant cost when many tasks are simple. If 70% of your tasks are SIMPLE, the router keeps 70% of requests from paying the planning overhead.

Hybrid 2: Plan-and-Execute with selective Reflection

You don't need reflection on every step — only on the final synthesis:

def selective_reflect(state: dict) -> dict:
    """Quick check: if completeness >= 8, deliver without the improvement loop."""
    quick_check = f"""Does this report cover ALL the aspects?
Task: {state['task']}  Report: {state['final_report'][:1000]}...
JSON: {{"score": N, "gaps": [...]}}"""
    
    result = json.loads(model.invoke([HumanMessage(content=quick_check)]).content)
    if result["score"] >= 8:
        return {"quality_score": result["score"]}
    return {"quality_score": result["score"], "reflection": {"gaps": result["gaps"]}}

This hybrid saves ~30% of full reflection's tokens because it only runs the improvement loop when it detects gaps.

Hybrid 3: Reflection in the pipeline

For tasks where each step produces independent output (not just the final report), you can put reflection after each step of the plan:

Plan → [Execute Step 1 → Reflect → OK?] → [Execute Step 2 → Reflect → OK?] → Synthesize

This is useful when:

• Each step produces data that feeds the next one (errors propagate)
• The steps are expensive (better to catch errors early)
• The task decomposition created steps with independent quality criteria

The trade-off is clear: more LLM calls (one reflection per step) but errors caught before they propagate. For a 5-step plan, it adds 5 reflection LLM calls (~2,500 tokens), but it prevents an error in step 2 from invalidating steps 3-5.

When NOT to combine

Combining patterns adds complexity to the code and to debugging. Don't combine if:

• The task is simple. Router → ReAct is enough for 70% of tasks.
• The budget is limited. Every layer adds LLM calls.
• You have no metrics. If you don't measure quality, you can't know whether reflection helps.
• The team can't debug the flow. A graph with 8 nodes and 3 conditional edges is hard to maintain.

Connection to the Project

In this module's deliverable (capsule 08), you'll implement exactly what you saw here: the Research Agent with a complexity router that picks between the three patterns. Your implementation will use M4's StateGraph as the base — the planning, research, analysis, and synthesis nodes become the plan's executors, the reflection node gets added as a quality gate, and the router as the initial node:

1. Low complexity: Router → create_react_agent directly → answer
2. Medium complexity: Router → Planner → Execute (M4's StateGraph per step) → Synthesize → answer
3. High complexity: Router → Planner → Execute → Synthesize → Reflect → (improve if needed) → answer

Troubleshooting

Problem 1: The planner generates overly detailed plans (10+ steps)

Symptom: 10-15 steps for a task ReAct would solve in 3 tool calls.

Solution: An explicit constraint: "RULE: Maximum 6 steps. If the task can be solved in fewer, use fewer." in the planner's prompt.

Problem 2: Reflection enters a loop and never accepts the output

Symptom: The agent iterates 3 times and the score barely moves. The improvement doesn't add new information.

Solution: Diminishing returns detection — if the improvement between iterations is < 0.5 points, stop:

def should_accept_or_improve(state: dict) -> str:
    if state["reflection"]["acceptable"] or state.get("iteration", 0) >= 3:
        return "done"
    prev = state.get("prev_quality_score", 0)
    curr = state["reflection"]["scores"]["average"]
    return "done" if curr - prev < 0.5 else "improve"

Problem 3: The router misclassifies complexity

Symptom: Complex tasks get routed to ReAct (incomplete output) or simple ones to Plan+Reflection (unnecessary latency).

Solution: Few-shot classification with concrete examples in the prompt: "capital of France" → SIMPLE, "compare 3 frameworks" → MODERATE, "due diligence report" → COMPLEX.

Problem 4: The step_results are inconsistent in length

Symptom: The synthesizer receives steps with different levels of detail. An unbalanced report.

Solution: Standardize each executor's output: "REQUIRED FORMAT: FINDINGS (2-3 paragraphs), SOURCES (URLs), CONFIDENCE (High/Medium/Low)".

Problem 5: Cost blows up in production

Symptom: 100+ requests/day with Plan+Reflection drain the budget.

Solution: A complexity router. If 60% of tasks are SIMPLE: 60×$0.004 + 30×$0.009 + 10×$0.015 = $0.66/day vs $1.50/day without a router. Savings: 56%.

Exercises

Exercise 1: Predicting metrics

Before running it, predict the metrics for this task with each pattern:

Task: "What are the 3 best Python libraries for web scraping and why?"

Estimate: tool calls, approximate latency, and expected quality (1-10).

Exercise 2: Design a 4-level complexity router

The capsule's router has 3 levels (SIMPLE, MODERATE, COMPLEX). Design one with 4 levels that includes "CRITICAL" — tasks where errors have serious consequences. Define each level's criterion, the assigned pattern, and an example.

def four_level_router(task: str) -> str:
    prompt = f"""Classify complexity and criticality:
- SIMPLE: 1-2 steps, an error is tolerable. E.g.: "What time is it in Tokyo?"
- MODERATE: 3-5 steps, an error causes rework. E.g.: "Compare 3 databases"
- COMPLEX: 5+ steps, an error causes bad decisions. E.g.: "Market report"
- CRITICAL: An error has legal/financial consequences. E.g.: "Contractual obligations"

Task: {task}. Answer with ONLY: SIMPLE, MODERATE, COMPLEX, or CRITICAL"""
    
    level = model.invoke([HumanMessage(content=prompt)]).content.strip().upper()
    routing = {"SIMPLE": "react", "MODERATE": "plan_execute",
               "COMPLEX": "plan_reflect", "CRITICAL": "plan_reflect_with_human_review"}
    return routing.get(level, "plan_execute")

Exercise 3: Implement diminishing returns detection

Implement a function that takes the history of quality scores from the reflection iterations and decides whether another iteration is worth it. The function should return True if it should continue, False if it should stop.

def should_continue_reflecting(
    score_history: list[float],
    max_iterations: int = 3,
    min_improvement: float = 0.5,
    target_score: float = 8.0,
) -> bool:
    if not score_history:
        return True
    
    if score_history[-1] >= target_score:
        return False
    
    if len(score_history) >= max_iterations:
        return False
    
    if len(score_history) >= 2:
        improvement = score_history[-1] - score_history[-2]
        if improvement < min_improvement:
            return False
    
    return True

# Tests:
assert should_continue_reflecting([5.0]) == True
assert should_continue_reflecting([5.0, 7.0]) == True
assert should_continue_reflecting([5.0, 7.0, 7.2]) == False  # improvement < 0.5
assert should_continue_reflecting([5.0, 7.0, 8.5]) == False  # target reached
assert should_continue_reflecting([5.0, 6.0, 6.5]) == False  # max iterations

Exercise 4: Calculate the break-even point

Your application processes 200 requests/day. 65% are SIMPLE, 25% MODERATE, 10% COMPLEX. Calculate:

1. The daily cost if you use Plan+Reflection for everything
2. The daily cost with a complexity router
3. The monthly savings (30 days) from the router
4. In how many days does the cost of implementing the router "pay for itself" (assume 8 hours of development at $50/hour)?

Exercise 5: Design a benchmark for your use case

Design a benchmark that compares the 3 patterns for your specific use case. Define: (1) 5 test tasks with varied complexity, (2) the metrics you'll measure, (3) the quality evaluation criterion, and (4) how you'll decide which pattern to use based on the results.

benchmark_tasks = [
    {"id": "simple_1", "query": "What is FastAPI and what are its advantages?", "complexity": "SIMPLE"},
    {"id": "simple_2", "query": "REST vs GraphQL", "complexity": "SIMPLE"},
    {"id": "moderate_1", "query": "Compare LangGraph, CrewAI and AutoGen", "complexity": "MODERATE"},
    {"id": "complex_1", "query": "Report: AI agents in production", "complexity": "COMPLEX"},
    {"id": "complex_2", "query": "Deployment analysis: cloud vs edge", "complexity": "COMPLEX"},
]

def run_benchmark(tasks, patterns=["react", "plan_execute", "plan_reflect"]):
    results = {}
    for task in tasks:
        results[task["id"]] = {}
        for pattern in patterns:
            agent = get_agent(pattern)
            metrics = RunMetrics()
            metrics.start_time = time.time()
            output = agent.invoke({"task": task["query"]})
            metrics.end_time = time.time()
            quality = evaluate_quality(output, task["query"])
            results[task["id"]][pattern] = {"metrics": metrics.summary(), "quality": quality}
    return results

Summary

The three reasoning patterns don't compete — they complement each other across a spectrum of complexity:

• ReAct is your default. Fast, cheap, works well for 1-3 step tasks. Its weakness is completeness on complex tasks — with no plan, it forgets sub-topics.

• Plan-and-Execute is your upgrade for complexity. The explicit plan guarantees completeness and makes execution predictable and debuggable. The trade-off is ~2x the latency and cost. It's justified when the task has 3+ independent sub-parts.

• Plan + Reflection is your maximum-quality tier. Reflection catches gaps the other approaches deliver without review. The trade-off is ~3-4x the cost and latency. It's justified when the output's quality matters more than speed (reports, code, critical decisions).

The decision isn't static — a complexity router can pick the pattern per request, optimizing cost without sacrificing quality where it matters. And the patterns are composable: Plan-and-Execute with selective reflection, ReAct with reflection on the final output, or reflection in the pipeline for tasks where errors propagate between steps.

What matters isn't memorizing which pattern is "best" — it's having a data-driven decision framework (tool calls, latency, cost, quality) that lets you pick the right pattern for each specific task.

Additional Resources

1. ReAct: Synergizing Reasoning and Acting in Language Models — The original paper that defines the Thought→Action→Observation pattern
2. Plan-and-Solve Prompting — The academic foundations of separating planning from execution in LLMs
3. Reflexion: Language Agents with Verbal Reinforcement Learning — Self-reflection as a mechanism for iterative improvement
4. LangGraph Plan-and-Execute Tutorial — The official Plan-and-Execute implementation in LangGraph
5. LangGraph Reflection Tutorial — The official Reflection implementation in LangGraph
6. Building Effective Agents — Anthropic — Agent design principles and when to use each pattern
7. Cognitive Architectures for Language Agents (CoALA) — A theoretical framework for categorizing and comparing agent architectures