Module 7: Project — Build a Mini Coding Agent
Analysis and Documentation
Description
The mini-agent's code is complete. Now comes the most valuable part of the project: running the agent on real tasks, observing its behavior, and documenting your insights. The analysis document is worth 25% of the evaluation — as much as the functional code — because understanding how an agent reasons is the real goal of this project.
Part 1: Run the Analysis Tasks
Run your mini-agent on at least 3-5 varied tasks. For each one, save the complete log.
Task 1: Pure exploration
"Explore the workspace and write a summary of what's there,
what each file does, and how they relate to each other."
What to observe: How the agent builds understanding step by step. Does it use list_directory first and then read_file? Or does it search directly?
Task 2: Bug fix
"The calculator.py file has a bug in the divide function:
if I pass divide(10, 3) it returns 3 instead of 3.333.
Investigate the problem and fix it."
What to observe: Does the agent read the file before acting? Does it verify the problem by running the code? Is its diagnosis correct? (Note: the current divide function doesn't have this bug — observe how the agent reacts to a possibly incorrect premise.)
Task 3: Create something new
"Create a utils.py file with functions to:
1. Reverse a string
2. Count the vowels in a string
3. Check whether a string is a palindrome
Include docstrings and edge case handling."
What to observe: Does it plan before writing or go straight to coding? Does the generated code handle the edge cases? Does it verify by running?
Task 4: Multi-step task
"Read calculator.py, add a factorial(n) function that
computes the factorial of n, create a test file for
ALL the functions (including the new one), and run
the tests."
What to observe: Does it follow a logical order (read → modify → create tests → run)? How many iterations does it need? Does it get lost in the process?
Task 5: Ambiguous task
"Improve the project."
What to observe: How does it handle the ambiguity? Does it explore first? Does it ask for clarification or assume? Which "improvements" does it choose and why?
Part 2: Analysis Document Template
Create an analysis.md file in the root of your project. Use this structure:
# Mini Coding Agent Analysis
## 1. Agent description
- Model used: [Claude Sonnet / GPT-4o-mini / etc.]
- Tools implemented: [list]
- Iteration limit: [number]
## 2. Summary of tasks run
### Task 1: [name]
- Iterations: [number]
- Tools used: [sequence]
- Result: [correct / partial / incorrect]
- Tokens consumed: [input / output]
### Task 2: [name]
[same structure]
[... for each task ...]
## 3. Insights (minimum 5)
### Insight 1: [title]
**Observation:** [what you observed]
**Connection with the guide:** [which module/concept explains this]
**Practical implication:** [what it means for your workflow]
### Insight 2: [title]
[same structure]
[... minimum 5 insights ...]
## 4. Patterns observed
- Which tools does it use most frequently?
- In what typical order does it use them?
- When does it choose well and when badly?
- How many iterations does it need per type of task?
## 5. Limitations found
- Where did the agent fail?
- Why did it fail? (connect with concepts from the guide)
- How would you improve it?
## 6. Personal reflection
- What changed in your understanding of how coding agents work?
- What will you do differently when using Claude Code / Cursor after this?Part 3: Guide for the 5+ Insights
Types of insights to look for
TYPE 1: TOOL SELECTION
Example: "The agent always uses list_directory before read_file
when it explores. This matches the 'top-down' strategy from
Module 04. When I give it specific context in the prompt,
it skips the exploration — confirming that the curated context
from Module 04 saves iterations."
TYPE 2: REASONING
Example: "The agent's 'thinking' before each tool call
shows chain-of-thought (Module 03). When the task is
ambiguous ('improve the project'), the reasoning is less
focused and produces less useful actions — this explains
why prompt-and-pray (Module 06) produces bad results."
TYPE 3: HALLUCINATIONS
Example: "In task 2 (bug fix), the agent diagnosed a
'bug' that didn't exist and generated an unnecessary fix. This is
a hallucination (Module 02): the LLM generated the most probable
('if you say there's a bug, I fix something') instead of verifying
first whether the bug is real."
TYPE 4: LOOP BEHAVIOR
Example: "Simple tasks take 3-5 iterations. The multi-step
task took 12 iterations. But the ambiguous task took
15 iterations without clear progress — the 'iteration spiral' from
Module 06. It should have applied the 'rule of three'."
TYPE 5: VERIFICATION
Example: "The agent only ran tests in 1 of 5 tasks.
In the rest, it generated code and declared it 'done' without verifying.
This confirms that Verification (Module 06) isn't automatic
— even for an agent. The developer has to request it
explicitly or the agent skips it."
Alternative: Analysis Without an API Key
If you don't have an API key, you can do the analysis using public transcripts:
Sources of transcripts
1. METR Study Transcripts
→ https://metr.org/notes/2026-02-17-exploratory-transcript-analysis/
→ Real transcripts of developers with coding agents
→ They include tool calls, reasoning, and results
2. Public coding agent demos
→ Videos of Claude Code, Cursor, Copilot Agent
→ Observe the tool calls and decisions
→ Pause and analyze each step
3. Your own experience
→ If you use a coding agent daily
→ Document 3-5 sessions with detailed notes
→ Observe the patterns you now know how to identify
Template for transcript analysis
# Coding Agent Transcript Analysis
## 1. Source of the transcript
- Origin: [METR / demo / own experience]
- Agent: [Claude Code / Cursor / etc.]
- Task: [description]
## 2. Step-by-step analysis
For each iteration of the transcript:
- Iteration N: [tool call] → [result] → [decision]
- Was the decision correct? Why?
## 3. Insights (minimum 5)
[same structure as the version with an API key]
## 4. Patterns and limitations
[same structure]
## 5. Reflection
[same structure]Part 4: Delivery Checklist
BEFORE DELIVERING, VERIFY:
CODE:
□ mini_agent.py works (can be run without errors)
□ The agentic loop completes basic tasks
□ At least 3 tools implemented and functional
□ Logging shows each decision
□ Basic safety implemented (workspace + allowed commands)
□ .env is NOT in the repository
ANALYSIS:
□ analysis.md created with the complete structure
□ At least 3 tasks documented with results
□ A minimum of 5 insights with observation + connection + implication
□ Patterns identified
□ Limitations documented
□ Personal reflection
Part 5: What to Do Next
Optional extensions
If you want to go further, you can add:
EXTENSION 1: MORE TOOLS
→ search_in_files(query, path): search text in files
→ web_search(query): search the internet (with requests)
→ git_status(): see the repo's state
EXTENSION 2: MULTI-TURN
→ Add input() after each of the agent's responses
→ The developer can give follow-up instructions
→ Keep the history between turns
EXTENSION 3: BETTER LOGGING
→ Save logs to a JSON file
→ Calculate total tokens per session
→ Measure time per iteration
EXTENSION 4: DYNAMIC SYSTEM PROMPT
→ Read an AGENT.md file as the system prompt
→ Change the behavior without modifying the code
→ The equivalent of CLAUDE.md for your mini-agent
Guide Closing
YOU COMPLETED "HOW CODING AGENTS WORK":
Module 01: You understood the paradigm shift
Module 02: You understood how LLMs work
Module 03: You understood what an agent is and how it reasons
Module 04: You know the tools and how to handle the context
Module 05: You know how to direct the agent with judgment
Module 06: You have a workflow (R→P→E→V) to be productive
Module 07: You BUILT an agent and observed from the inside
WHAT YOU HAVE NOW:
→ A deep understanding of how coding agents work
→ A functional mini-agent you can extend
→ A mental framework transferable to ANY coding agent
→ An analysis document that solidifies your understanding
WHAT'S NEXT:
→ Apply these concepts with your favorite coding agent
→ Practice R→P→E→V in your daily work
→ Explore the specific-tool guides
(Claude Code Foundations, Cursor Essentials)
→ Keep calibrating your confidence with experience
Module Summary
MODULE 07: PROJECT — BUILD A MINI CODING AGENT
Capsule 01: What you'll build and why
Capsule 02: Setup — API key, dependencies, first call
Capsule 03: The agentic loop — the while loop that is the heart
Capsule 04: The tools — read, write, list, run with safety
Capsule 05: Analysis — observe, document, understand
DELIVERABLES:
→ mini_agent.py (~265 lines, functional)
→ analysis.md (5+ documented insights)
THE FINAL MESSAGE:
→ A coding agent is YOUR code + an LLM API
→ The LLM provides the reasoning, you provide the tools and the control
→ Building one, even a simple one, gives more understanding
than 100 hours of just using one