Verification: The Step Everyone Skips

Description

Of the four phases of R→P→E→V, Validate is the most important — and the one most often skipped. This capsule explains why verification is the most costly blind spot in development with AI, what happens when you skip it, and how to turn the agent from a code generator into a code verifier.

It's not a capsule about testing (that deserves its own guide). It's about the concept of verification as a mandatory phase of the workflow, and the specific techniques for verifying AI-generated code.

---

Why We Skip Verification

The psychology behind the skip

1. COMPLETENESS BIAS
   "It looks complete" → "It is complete"
   The brain confuses appearance with correctness.
   Well-formatted code that compiles LOOKS correct.

2. ILLUSION OF SPEED
   "I already spent 30 minutes, I don't want to spend more"
   Verifying feels like "extra work" after finishing.
   The irony: the 30 minutes of verification save 3 hours of debugging.

3. TRUST IN THE AGENT
   "The AI knows what it's doing"
   Stack Overflow 2025: only 3% of developers highly trust
   AI code. But behavior says otherwise: most
   accept without verifying.

4. DECISION FATIGUE
   After Research, Plan, and Execute, you're tired.
   Verification requires fresh attention.
   It's tempting to declare victory and move to the next task.

5. TIME PRESSURE
   "I have to deliver today"
   Verification is perceived as sacrificable.
   The reality: delivering without verifying is delivering with bugs.

The observable pattern

                    With a complete workflow
TIME ─────────────────────────────────────────▶

R     P     E     V
│     │     │     │
▼     ▼     ▼     ▼
15m   10m   40m   25m   = 90 minutes → DONE

                    Without Verification
TIME ─────────────────────────────────────────▶

R     P     E     (skip V)    Debug    Fix    Debug    
│     │     │                 │        │      │        
▼     ▼     ▼                 ▼        ▼      ▼        
15m   10m   40m   "Done!"     60m      30m    45m = 200 minutes

The 25 minutes of Validate you "save" cost you 135 minutes later.

---

What It Costs to Skip Verification

The four costs

1. BUGS IN PRODUCTION
   → Code that "works in dev" fails with real data
   → Unconsidered edge cases cause errors for users
   → The cost of a fix in production is 10-100x greater than in development

2. SILENT TECHNICAL DEBT
   → Code that works but violates the codebase's patterns
   → Implementations that solve the problem today but create problems tomorrow
   → Logic duplication the agent didn't detect

3. FALSE CONFIDENCE
   → "I did it with AI and it came out fast" → the team assumes AI accelerates
   → The bugs appear later and aren't attributed to the process
   → The pattern repeats on more critical tasks

4. GRADUAL QUALITY EROSION
   → Each unverified task adds a bit of degradation
   → The codebase becomes less maintainable incrementally
   → After months, the code is a minefield

Data that backs the cost

VERACODE 2025:
→ 45% of AI-generated code has security vulnerabilities
→ Most of it isn't detected without specific analysis

GITHUB COPILOT RESEARCH:
→ Developers accept Copilot suggestions ~30% of the time
→ Of those accepted suggestions, how many are verified? No data,
   but the acceptance speed suggests that few

METR STUDY:
→ The developers who were slower with AI shared a pattern:
   accepting output without verifying → extensive debugging afterward
→ The faster ones: verified in the moment → less debugging

---

The Three Levels of Verification

Not all verification requires the same effort. Connecting with the trust calibration from Module 05, each level of confidence requires a different level of verification.

Level 1: Complete verification

WHEN: Trust calibration → COMPLETE
→ New code in unknown areas
→ Critical business logic
→ Authentication, payments, sensitive data
→ Infrastructure or security configuration

WHAT TO DO:
→ Read every line of generated code
→ Run tests (existing + new)
→ Verify against the spec/plan
→ Look for uncovered edge cases
→ Security review if applicable
→ Test with real or realistic data

TIME: 25-40% of the total time of the task

Level 2: Focused verification

WHEN: Trust calibration → FOCUSED
→ Extension of existing patterns
→ Code in areas you know partially
→ Moderate tasks with some risk

WHAT TO DO:
→ Review the code's decision points
→ Run relevant tests
→ Verify integration with existing code
→ Review the edge cases defined in the plan
→ Spot-check the rest

TIME: 15-25% of the total time of the task

Level 3: Light verification

WHEN: Trust calibration → LIGHT
→ Trivial changes in familiar code
→ Direct extension of well-established patterns
→ Easily reversible code
→ Low-risk tasks

WHAT TO DO:
→ Run existing tests
→ Quick scan of the diff
→ Verify the general behavior didn't change
→ Check that it compiles and the types are correct

TIME: 5-15% of the total time of the task

Decision diagram

What level of verification do I need?

1. Can the error affect users in production?
   YES → COMPLETE verification
   
2. Do I know this part of the codebase well?
   NO → COMPLETE verification
   
3. Is it an extension of a pattern that already works?
   YES + low risk → LIGHT verification
   YES + some risk → FOCUSED verification
   
4. Is it easily reversible?
   YES → a lower level is acceptable
   NO → a higher level is necessary

---

Verification Techniques with Coding Agents

Technique 1: The agent as reviewer

Instead of just generating code, use the agent to review it:

AFTER the agent implemented it:

"Now switch roles. Instead of an implementer, be a reviewer.
Review the code you just generated and look for:
1. Logic errors
2. Uncovered edge cases
3. Violations of the codebase's patterns
4. Possible security problems
5. Code that could be simpler"

→ The agent frequently finds errors in its own code
→ Switching the "role" activates a different reasoning mode
→ It's not perfect, but it catches ~30-50% of the problems

Why it works

When the agent GENERATES code:
→ Constructive mode: "how do I implement this"
→ Bias toward completing the task
→ It can take shortcuts due to completeness bias

When the agent REVIEWS code:
→ Analytical mode: "what could be wrong"
→ Bias toward finding problems
→ It sees the code "with fresh eyes" (context reset)

Technique 2: Verification by contrast

Ask the agent to generate a second independent implementation and compare:

STEP 1: Original implementation already done

STEP 2: "Without looking at the previous implementation, propose
         an alternative approach to solve the same 
         problem. Don't copy — reimagine."

STEP 3: Compare the two implementations:
→ Do they handle the same edge cases?
→ Do they have the same business logic?
→ Where do they differ and why?
→ The differences reveal weak points

When to use verification by contrast

✅ Good for:
→ Complex business logic
→ Non-trivial algorithms
→ When you're not sure the approach is correct

❌ Not necessary for:
→ Basic CRUD
→ Extensions of known patterns
→ Trivial code

Technique 3: Tests as verification

Use the agent to write tests AFTER the implementation (when you didn't use Spec-first):

"Write tests for the code we just implemented.
Include:
→ Happy path with normal data
→ Edge cases: empty inputs, nulls, limits
→ Error cases: what happens when it fails
→ Integration: it works with the rest of the system"

Why does asking for tests AFTER work as verification?

→ If the agent can't write a test for a case,
  the implementation probably doesn't handle it
→ If a test fails, you found a bug
→ The process of writing tests reveals implicit assumptions

Technique 4: The "adversarial prompt"

Ask the agent to try to break its own implementation:

"Try to break the code you just wrote.
You're a malicious tester. Look for:
→ Inputs that cause crashes
→ Race conditions
→ Memory leaks
→ SQL injection, XSS, or similar
→ Unexpected behavior with extreme data"

The agent in adversarial mode:
→ Thinks differently than in constructive mode
→ Looks for failures instead of functionality
→ Frequently finds real problems

Technique 5: Verification against the plan

"Compare the implemented code with the original plan:

PLAN:
[paste the plan from phase 2]

QUESTIONS:
1. Was everything the plan specifies implemented?
2. Are there deviations? Are they justified?
3. Is the definition of done met?
4. Is something the plan included missing?"

This technique is simple but powerful:
→ It catches omissions (things forgotten in the implementation)
→ It catches deviations (things that changed without justification)
→ It closes the loop between Plan and Validate

---

The Concept of "Verification-First"

What it means

Verification-first doesn't mean verifying before coding (that would be Spec-first). It means that verification guides the implementation:

NORMAL APPROACH:
"Implement X" → [code] → "Does it work?" (verification at the end)

VERIFICATION-FIRST:
"How am I going to verify that X works?" → [define checks]
→ "Implement X" → [code] → [run the defined checks]

The difference:
→ You decide HOW to verify BEFORE implementing
→ The implementation is oriented toward being verifiable
→ There are no surprises at the end

Example

NORMAL APPROACH:
"Implement caching for the products API"
→ [agent implements Redis caching]
→ "Does it work?" → "Uh... how do I verify?"

VERIFICATION-FIRST:
"Before implementing caching, let's define how to verify:
1. Cache hit: the second call is faster than the first
2. Cache invalidation: after an update, fresh data
3. Cache miss: the first call works normally
4. TTL: after expiring, the data is refreshed
5. Memory: the cache doesn't grow without limit"

→ "Now implement caching knowing that we're going to verify
   these 5 points specifically"

→ [implementation]

→ "Let's verify the 5 points:
   1. Cache hit... ✅
   2. Cache invalidation... ✅
   3. Cache miss... ✅
   4. TTL... ⚠️ not implemented
   5. Memory... ⚠️ no limit configured"

→ Fix TTL and memory limit BEFORE declaring it done

Why Verification-First changes the result

Without Verification-First:
→ "Implement caching" → works on the happy path → "done"
→ TTL and memory limit weren't implemented
→ In production: gradual memory leak, stale data

With Verification-First:
→ You define what to verify BEFORE
→ The implementation is aware of the criteria
→ The agent has more context about what matters
→ The gaps are detected in development, not in production

---

Verification for Different Types of Code

Business logic code

VERIFY:
→ All the if/else/switch paths
→ Domain edge cases (negative amounts, empty strings)
→ Consistency with existing business rules
→ That the variable/function names reflect the domain

RECOMMENDED TECHNIQUE:
→ Tests + adversarial prompt
→ "What happens if the user has a negative balance?"
→ "What happens if the product has a price of 0?"

Infrastructure code

VERIFY:
→ Error handling (connection down, timeout, retry)
→ Configuration (environment variables, secrets)
→ Idempotency (running twice doesn't cause problems)
→ Logging (enough for debugging, not excessive)

RECOMMENDED TECHNIQUE:
→ Verification-first + complete review
→ "What happens if the database doesn't respond?"
→ "What happens if it runs twice?"

Security code

VERIFY:
→ Input validation (ALL inputs)
→ Authentication/Authorization on each endpoint
→ SQL injection, XSS, CSRF
→ No hardcoded secrets
→ Rate limiting
→ Error messages that don't reveal internal information

RECOMMENDED TECHNIQUE:
→ Complete verification + adversarial prompt
→ NEVER light verification for security code
→ "Try to bypass this authentication"
→ "Can SQL be injected in any input?"

UI code

VERIFY:
→ Works on different screen sizes
→ States: loading, error, empty, data
→ Basic accessibility (keyboard, screen reader)
→ No visual regressions

RECOMMENDED TECHNIQUE:
→ Focused verification + manual visual test
→ The agent can verify logic but not UX
→ Always verify visually what the agent generates

---

The Universal Verification Checklist

Use this checklist as a quick reference for any task:

MINIMUM VERIFICATION (everything passes through here):
□ Does it compile without errors?
□ Do the existing tests still pass?
□ Does the basic behavior work?

STANDARD VERIFICATION (most tasks):
□ All of the above +
□ Are there new tests for the new code?
□ Are the edge cases defined in the plan covered?
□ Does the code follow the codebase's patterns?
□ Is the definition of done met?

COMPLETE VERIFICATION (critical code):
□ All of the above +
□ Are there security vulnerabilities?
□ Is the error handling robust?
□ Is the implementation idempotent where it should be?
□ Are the error messages informative without revealing internals?
□ Does it work with extreme data (very large, empty, malformed)?

---

Practical Exercise

Exercise 1: The cost of NOT verifying

Think about a bug you found in production in the last 6 months:

1. What type of bug was it?
   _______________________________________________

2. Would it have been detected with verification in development?
   □ Yes, with tests    □ Yes, with review    □ No, it was unpredictable

3. How much time did it cost to fix it in production?
   _______________________________________________

4. How much would it have cost to verify it before deploy?
   _______________________________________________

5. Ratio: (production fix time) / (verification time) = ___x

Exercise 2: Design your verification protocol

For your main stack/project, define:

LIGHT VERIFICATION (low-risk tasks):
→ _______________________________________________
→ _______________________________________________

FOCUSED VERIFICATION (moderate tasks):
→ _______________________________________________
→ _______________________________________________
→ _______________________________________________

COMPLETE VERIFICATION (critical code):
→ _______________________________________________
→ _______________________________________________
→ _______________________________________________
→ _______________________________________________

WHICH TECHNIQUE WILL YOU USE MOST?
□ Agent as reviewer
□ Verification by contrast
□ Tests as verification
□ Adversarial prompt
□ Verification against the plan

Exercise 3: Practice the adversarial prompt

Take a fragment of AI-generated code (from your real work or from a previous exercise) and practice:

STEP 1: Paste the code to the agent
STEP 2: "Try to break this code. Look for all the possible
         failures: logic errors, edge cases, security,
         performance."
STEP 3: Evaluate the results
STEP 4: How many of the problems found would you have
        detected yourself without the adversarial prompt?

---

Common Mistakes in Verification

Mistake	Consequence	Correction
"Compiles = works"	Silent logic bugs	Tests + review
Verifying only the happy path	Edge cases in production	Adversarial verification
Skipping Validate due to time pressure	Debugging 5x longer afterward	Invest 20% of the time in Validate
Only automatic tests, no review	Code quality degrades	Combine tests + code review
Inconsistent verification	Some bugs get through, others don't	Standardized checklist

---

Summary

VERIFICATION is the most important step of the workflow
and the one most often skipped.

WHY IT'S SKIPPED:
→ Completeness bias (it looks complete = it is complete)
→ Illusion of speed (verifying feels like "extra")
→ Decision fatigue after R+P+E

WHAT IT COSTS TO SKIP IT:
→ 25 min of verification saves 135 min of debugging
→ Bugs in production, technical debt, false confidence

THREE LEVELS:
→ Complete: critical code, unknown areas
→ Focused: moderate risk, partially known patterns
→ Light: low risk, established patterns

TECHNIQUES WITH THE AGENT:
→ Agent as reviewer (switch roles)
→ Verification by contrast (second implementation)
→ Tests as verification (post-implementation)
→ Adversarial prompt (try to break it)
→ Verification against the plan (close the loop)

VERIFICATION-FIRST:
→ Decide HOW to verify BEFORE implementing
→ The implementation is oriented toward being verifiable
→ The gaps are detected in development, not in production

Next capsule: 05 - Common anti-patterns — the five most destructive errors and how to recognize them.

---

Resources

1. Veracode State of Software Security 2025 — Data on vulnerabilities in AI-generated code
2. Google: AI-Assisted Security Reviews — How Google uses AI for security verification
3. OWASP: AI Security Testing Guide — Framework for security verification with AI
4. Martin Fowler: Continuous Integration — Principles of continuous verification
5. Anthropic: Safe Coding Practices — Verification recommendations for code generated with Claude