8. Project: leaving a repository agent-ready

Description

By the end of this lesson you will have, on a real repository — not a toy exercise put together for the occasion — the harness's four pieces complete and working: an instructions file with verifiable rules, documented one-line commands to install, test, and review, versioned least-privilege permissions, and a loop of tests, types, and linter that runs in seconds. And you'll have something more, which no previous lesson in this module asked of you yet: proof that work actually mattered. Not the feeling that "it works better now," but the transcript, the diff, and the time it took to run the same real task, on the same repository, before and after the harness — saved as evidence, not as a memory.

This matters because in real work nobody's going to give you time to rewrite a CLAUDE.md and configure permissions just because "it feels like it should help." A tech lead who has to approve four hours of an engineer's time dedicated to this is rightly going to ask how real the improvement is — and "trust me, the agent performs better now" isn't an answer that survives that question. A measured before-and-after, with the transcript and the diff saved alongside the code, does.

Connection to the module: this lesson introduces no new harness component — it brings together, on a single repository, the four pieces you already built separately: context engineering (lesson 2), instructions the agent actually respects (lesson 3), least-privilege permissions (lesson 4), the automatic signal loop (lesson 5), reproducible commands and test data (lesson 6), and the session artifact that survives a restart (lesson 7). The only genuinely new thing is the methodology for proving the improvement is real and not an impression: exactly lesson 1's illustrative table — "the times that follow are illustrative, not a benchmark" — but this time actually measured, on your own repository.

A before-and-after that stands on its own, with you not there to explain it

Imagine a mechanic swearing the new impact wrench is way better than the old one. He tells a colleague, who asks how long it took to change a wheel's four lug nuts with each one. The mechanic didn't time it — it just "felt faster." That claim is worth nothing in a serious shop, because it doesn't rule out any alternative explanation: maybe this particular wheel's nuts were less rusted, maybe the mechanic had already done the change once that afternoon and the second time went faster just from practice, maybe he switched cars at the same time he switched tools. A shop that takes the comparison seriously does something else: the same wheel, the same nuts, the same mechanic, stopwatch in hand, once with each tool — and logs the time, not the impression.

That's exactly what you're still missing in this module. You configured the harness's four pieces, lesson by lesson, and it's reasonable to suspect your repository performs better now. But "suspecting it performs better" and "having measured it" are different things, and only the second survives a question from someone who wasn't watching over your shoulder. A valid before-and-after demands three conditions, none optional: the same real task in both runs (not a simplified version "to make it go fast"), the same repository except for the variable you're measuring — the harness — and saved evidence from each run — not a number from memory written down a week later. Without all three, what you have is an opinion shaped like a measurement.

Worked example

You're going to build this on the same scenario from lesson 1 of this module: the orders-service service, and the task "Add a limit of 100 requests per minute per IP to the POST /orders endpoint. Run the tests and confirm they pass." There, that comparison was an illustrative table, flagged as such. Here it's the same task, but actually measured.

Step 1 — Freeze the "before" state without losing the harness work. Before touching a single harness file, mark where you are:

git tag pre-harness-baseline
git worktree add ../orders-service-before pre-harness-baseline

git worktree add creates a second working copy of the same repository, pointing to the commit you just tagged, without moving or duplicating your history. That leaves you with two usable folders at the same time: orders-service/ — where you're going to build the new harness — and ../orders-service-before/ — frozen exactly as it was before you started, ready to run the "before" pass there whenever you want, with no back-and-forth git checkout risking mixing the two states.

Step 2 — Build the harness's four pieces (what you already know how to do, now all together, in orders-service/).

CLAUDE.md, with verifiable rules and the off-limits zone explicitly marked:

# CLAUDE.md — orders-service

## Commands
- Install dependencies: `just install`
- Run the full test suite: `just test` (it creates and destroys the test
  database itself — never use `pytest` directly, you'll skip that setup)
- Run linter and types: `just check`

## Project rules
- Every new route under `app/routers/` needs a test in the mirrored file
  under `tests/` — `app/routers/orders.py` → `tests/test_orders.py`.
- API middleware logic (rate limits, authentication, logging) lives in
  `app/middleware/`, never inside an individual route's handler.
  - ✅ `app/middleware/rate_limit.py`, imported in `orders.py`.
  - ❌ A request counter written directly inside `create_order()`.

## Out of scope without explicit approval
- `migrations/` — any schema change goes through human review.
- Git history: never rebase, never force-push.

.claude/settings.json, with the allowlist winning over the denylist, just like you saw in lesson 4:

{
  "permissions": {
    "allow": [
      "Bash(just test)",
      "Bash(just check)",
      "Bash(git status)",
      "Bash(git diff)",
      "Bash(git log*)"
    ],
    "deny": [
      "Bash(git push*)",
      "Bash(git branch*)",
      "Edit(./migrations/**)",
      "Read(./.env)",
      "Read(./secrets/**)"
    ]
  }
}

A Justfile with one command per intent, just like you saw in lesson 6:

install:
    uv sync --group dev

test: install
    uv run pytest tests/ -x -q

check:
    uv run ruff check .
    uv run mypy app/

And the concurrency fixture lesson 1 only mentioned in passing — built following lesson 6 — now actually written, in tests/test_orders.py:

from concurrent.futures import ThreadPoolExecutor

VALID_ORDER_PAYLOAD = {"customer_id": "cust_test_001", "items": [{"sku": "SKU-1", "qty": 1}]}


def test_rate_limit_blocks_after_100_requests_per_minute(test_client):
    def send_order(_):
        return test_client.post("/orders", json=VALID_ORDER_PAYLOAD)

    with ThreadPoolExecutor(max_workers=20) as pool:
        responses = list(pool.map(send_order, range(110)))

    status_codes = [response.status_code for response in responses]
    assert status_codes.count(429) >= 10

That test fires 110 concurrent requests against the same endpoint. A rate limit implemented with a simple counter, with no protection against race conditions, lets more than 100 through before blocking — or fails intermittently; a correct one blocks at least 10 with 429. It's exactly the concurrency bug lesson 1 described in the abstract; here you have the test that makes it visible in seconds instead of two hours later, in human review.

Step 3 — Run the "before" pass, in ../orders-service-before/. Exact same prompt, with none of step 2's pieces:

docs/harness-evaluation/before/timing.md
---
Start: 14:02:07
End: 14:53:15
Total duration: 51 min 8 s
Permission interruptions: 7
Retries from a failed command: 1 (tried `pytest` directly, that runner
doesn't exist)

docs/harness-evaluation/before/transcript-excerpt.md
---
[14:04] Agent: I can't find a documented test command. I'll try `pytest`.
[14:04] $ pytest
        ModuleNotFoundError: No module named 'pytest' (environment not
        activated)
[14:11] Agent: Can I install dependencies with `pip install -r requirements.txt`?
        → waiting for confirmation (1)
[14:19] Agent: I implemented the limit with an in-memory dictionary
        `request_counts: dict[str, int]` inside the handler.
[14:27] Agent: Can I run `pytest tests/test_orders.py`?
        → waiting for confirmation (4)
[14:44] Agent: existing tests pass. Marking the task as done.

Nobody ran a concurrency test there — it didn't exist yet — so the dictionary with no lock between threads went undetected. docs/harness-evaluation/before/diff.patch gets saved as-is, with that implementation.

Step 4 — Run the "after" pass, in orders-service/, with the complete harness. Exact same prompt:

docs/harness-evaluation/after/timing.md
---
Start: 15:10:00
End: 15:26:44
Total duration: 16 min 44 s
Permission interruptions: 2 (approach confirmation and final commit)
Retries from a failed command: 0

docs/harness-evaluation/after/transcript-excerpt.md
---
[15:10] Agent: CLAUDE.md says to use `just test` for testing and that
        middleware lives in app/middleware/. I'll create
        app/middleware/rate_limit.py.
[15:13] $ just check
        (3.9s) 0 linter errors, 0 type errors
[15:15] Agent: I ran `just test` — the new concurrency test
        (test_rate_limit_blocks_after_100_requests_per_minute) failed:
        status_codes.count(429) == 3, expected >= 10.
        My dict-based counter isn't thread-safe under these 20 concurrent
        connections. Fixing it with a threading.Lock around the increment.
[15:19] $ just test
        (2.1s) 14 passed
[15:26] Agent: tests and types are green. Confirm the commit?
        → waiting for confirmation (2)

docs/harness-evaluation/after/diff.patch saves the final implementation, threading.Lock included — fixed in the same session, before a human had to find it.

Step 5 — docs/harness-evaluation/RESULTS.md, the summary that brings the four folders together:

# Result: adding rate limiting to POST /orders

Same task, same prompt, same repository (pre-harness-baseline commit
vs. HEAD after completing the harness). Not a generalizable benchmark —
it's the measurement of a single real case, with evidence saved in before/
and after/ so anyone can review it without having been present.

| Metric                          | Before  | After  |
|----------------------------------|---------|--------|
| Total duration                   | 51 min  | 17 min |
| Permission interruptions         | 7       | 2      |
| Retries from a failed command    | 1       | 0      |
| Concurrency bug caught in        | Human review (didn't occur in this run) | The same session, 4 min after writing the test |

See before/transcript-excerpt.md and after/transcript-excerpt.md for the
detail of each run, and before/diff.patch, after/diff.patch for the code
each one produced.

What to expect from all this: the model behind the agent was the same in both runs — the only thing that changed was what it saw (CLAUDE.md), what it could execute without asking (settings.json), and whether something told it its first attempt was broken before you reviewed it (the concurrency test). That difference, measured and saved, is this module's complete project.

How to version the evidence so it's verifiable by someone else

The five folders and files you just saw aren't one afternoon's notes: they're the deliverable, and they live in your repository, versioned alongside the harness that produced them — not drafted from memory a week later to justify a decision already made. The same discipline you already applied when versioning the specification package in module 2's project: the value isn't in the moment you wrote it, it's that someone who didn't see the session can reconstruct what happened without asking you anything.

docs/harness-evaluation/
  RESULTS.md
  before/
    timing.md
    transcript-excerpt.md
    diff.patch
  after/
    timing.md
    transcript-excerpt.md
    diff.patch

This gives you a concrete operational test, instead of a nice-sounding phrase: hand the complete folder — not the final code, not your memory — to a colleague, or cheaper still, to a new session of your agent that has only read docs/harness-evaluation/. Ask it what changed between the two runs and why the concurrency bug showed up in one and not the other. If it answers correctly with nothing more explained to it, the evidence passed the test. If it needs you to clarify something "that was understood," that something is missing from RESULTS.md, not from whoever read it.

Common mistakes

Changing the task, the prompt, or the repository's state between the two runs without noticing (conceptual). What happens: the "before" run is done with one version of the request, and the "after" one with a slightly different one — shorter, more specific, or on a repository that already has the bug fixed by hand from an earlier session — and the "improved" result proves nothing, because it no longer isolates the harness as the only variable that changed. Why it happens: time usually passes between configuring the harness and running the comparison, and it's easy to "slightly improve" the prompt when rewriting it from memory, without noticing it's no longer the same. How to spot it: put the two prompts from before/ and after/ side by side — if they aren't exactly the same text, the comparison is contaminated. How to fix it: save the exact prompt to a file as soon as you write it the first time, and copy it verbatim for the second run; step 1's git worktree exists exactly so the "before" state stays frozen and you don't have to reconstruct it from memory.

Saving only the final number, with no diff or transcript (practical). What happens: someone notes "went from 51 to 17 minutes" in a team chat message and deletes the session, leaving no trace of what produced each run. Why it happens: the number seems like the important part, and saving transcripts feels like extra work once you've already seen the result you expected. How to spot it: if someone asks you "why exactly did the time go down?" and you can't point to a concrete file — the test that caught the bug, the permission that no longer interrupted — to answer, you didn't save evidence, you saved a conclusion. How to fix it: this lesson's docs/harness-evaluation/ folder exists exactly for this; a number with no diff or transcript explaining it isn't defensible to anyone who asks "why."

Confusing "I left the harness configured" with "the harness ended up agent-ready" (conceptual). What happens: the four files get written — CLAUDE.md, settings.json, the Justfile, the concurrency test — and the project is called done without running either of the two passes. Why it happens: writing the configuration feels like the real work, and running the full task twice feels like an extra, optional step, "to confirm something already known to work." How to spot it: if your repository has the four pieces but there's no docs/harness-evaluation/ folder with real evidence of at least one task run end to end, the harness is configured, but nobody tested it — exactly the mistake lesson 1 of this module already warned about, now in its most expensive version: never verifying whether it actually worked. How to fix it: none of the four pieces counts as done without at least one real run that exercised all of them together — this lesson's project is that run, not the configuration by itself.

Exercises

Exercise 1 — Spot the contaminated comparison. A colleague shows you their RESULTS.md: "Before: 38 minutes, running 'add email validation to the registration endpoint.' After: 9 minutes, running 'add email and phone validation to the registration endpoint, reusing the validator you already wrote last time.'" Using this lesson's three conditions for a valid before-and-after, what would you tell them?

See solution

You'd tell them the comparison doesn't isolate the harness as the variable: the "after" task isn't the same as the "before" one — it's bigger in scope (adds phone too) but at the same time cheaper to solve, because it reuses a validator the first run had to write from scratch. The time difference could be entirely due to the second task starting from more work already done, not to the harness having improved anything. For the comparison to mean something, it would need to run exactly the same request — "add email validation to the registration endpoint," word for word — in both runs, on the same repository state except for the harness.

Why it works: it applies the first condition of a valid before-and-after — the same real task in both runs — to a case where the task subtly changed, which is exactly the kind of contamination this lesson's first common mistake describes.

Exercise 2 — Audit a harness declared "ready." A repository has this CLAUDE.md, this settings.json, and no file under docs/harness-evaluation/. The team says: "we already left the repo agent-ready, good to go." What's missing for that claim to be defensible?

{
  "permissions": {
    "allow": ["Bash(pytest)", "Bash(ruff check .)"]
  }
}

See solution

At least two concrete things are missing. First, settings.json has no deny list at all, and Bash(pytest) with no coverage of the rest of the catalog means any other command — including git push, rm -rf, or reading a .env — stays in "ask about everything" mode or, worse, with no rule covering it at all; lesson 4's least privilege isn't applied, only two loose commands are allowed. Second, and more important for this lesson: there's no docs/harness-evaluation/ folder with a real, measured run. The team may have the pieces well written, but "good to go" is a claim that only holds up with a real task run end to end, its time and its diff saved — without that, it's exactly this lesson's third common mistake: confusing configured with tested.

Why it works: it separates two different questions the exercise deliberately blends — is the permissions file well designed? and does evidence exist that the whole set works? — because a harness can fail on either one separately.

Exercise 3 — The complete project, on your own repository. On a real repository of yours — work or practice, as long as you use it with some coding agent —: (1) tag the current state and create a worktree as in step 1 of the worked example; (2) build or finish the harness's four pieces (curated context, verifiable instructions, least-privilege permissions, measured signal loop); (3) choose a real task from your backlog, not invented; (4) run that task in the "before" worktree and in your "after" repository; (5) assemble docs/harness-evaluation/ with RESULTS.md and the before/, after/ folders.

See solution

There's no single answer because it depends on your repository and your real task — but your project passes self-verification if it meets these four points:

1. The "before" run's prompt and the "after" run's prompt are, literally, the same text — you can put them side by side and confirm it.
2. settings.json has both allow and deny, and you can name, without a second thought, which zone of your repository was explicitly left out of scope.
3. docs/harness-evaluation/RESULTS.md has concrete numbers — duration, interruptions, retries — and every number can be traced back to an evidence file (timing.md, transcript-excerpt.md, diff.patch), not just to your memory.
4. You handed the complete folder to someone who saw neither session — a colleague or a new session of your agent — and that person could explain, with nothing else clarified for them, what changed between the two runs and why.

Why it works: these four points are this lesson's three common mistakes turned into yes/no questions — if your project passes them, you left a repository agent-ready with proof that it is, not just the configuration suggesting it should be.

Summary and next step

What you built in this lesson isn't one more harness piece — it's the measured, saved proof that this module's previous seven lessons produced something real. You have the instructions file, the least-privilege permissions, the automatic signal loop, and the one-line commands, all together on a real repository; and you also have evidence that running the same task with the old harness and the new one isn't the same experience — with numbers, a diff, and a transcript anyone can review without having watched you work.

Before moving on you should be able to: name the three conditions for a valid before-and-after without looking at notes; explain why a number with no diff or transcript isn't evidence, just a conclusion; and have run, on a real repository of yours, at least one complete comparison with its docs/harness-evaluation/ folder versioned.

Now that the repository has automatic signal — tests, types, and linter that run in seconds and tell the agent whether its work worked — human review no longer has to waste time confirming what that signal already confirmed. Module 4 gets right into that: what to do with the part no test catches — unrequested scope, design decisions slipped in without anyone asking for them, code that passes every test but doesn't do what the specification asked — and how to turn that review into a repeatable procedure, not a generic suspicion of "I check everything just in case."

Resources

• Effective harnesses for long-running agents — Anthropic — this entire module's underlying reference; goes deeper into why measuring the harness, not just configuring it, is part of designing it.
• Claude Code — Best practices for agentic coding — Anthropic — the conceptual basis for "give Claude a way to verify its work," the mechanism behind the concurrency bug getting caught in the same session instead of in human review.
• Claude Code — Settings — official reference for allow/deny syntax in settings.json, used in this lesson's .claude/settings.json.
• git-worktree — official Git documentation for the command used in step 1 of the worked example, which freezes the "before" state without losing the new harness work.
• just — a command runner — official manual for the recipe syntax used in this lesson's Justfile.