Module 7: Documentation That Survives

Living documentation

Overview

The previous lesson installed the thesis with a measurement: a system's knowledge lives in heads that leave, and documentation that survives is the insurance against that. But there's a trap this lesson dismantles: most documentation doesn't survive, even when it's written. Not because no one writes it —there's almost always a wiki—, but because it rots: it's written once, becomes disconnected from the code, and as the code changes and the wiki stays still, the doc drifts out of sync with reality until it says false things. And a doc that says false things is worse than having no doc: it sends the new dev down the wrong path. This lesson explains why doc rots, and what the alternative means: living documentation —live documentation, that stays true because it lives glued to the code and the decisions, and updates in the same change that would make it obsolete—.

The concept of living documentation, popularized by Cyrille Martraire, has a simple central idea: the doc that survives is the one so close to the code that updating it is part of changing the code, not a separate step someone will remember to do later. The disconnected wiki rots because updating it is a separate act —it requires remembering, opening another tool, writing— and "later" never comes. Live doc doesn't rot because there's no "later": the change and its documentation travel together. This lesson measures the difference between the two over the life of a system, and discovers a phenomenon that makes the trap even worse than it seems: the trust chasm. When a doc's accuracy falls below a certain threshold, devs stop trusting all of it —they can't tell which part is still fine and which part is already false—, so its effective value collapses to zero even if a fraction is still correct. The doc doesn't degrade smoothly; it falls off a cliff.

Connection with the module. It's the first of the lessons that install how to make doc survive. Here we work the principle —what keeps doc alive: proximity to the code—; lesson 3 will work the concrete practice that forces that proximity (docs-as-code: doc in the repo, reviewed in PRs). It's the relationship between understanding why the previous owner's manual has to be synchronized with the house (this lesson) and the mechanism that guarantees it —gluing it to the house's wall, not leaving it in another city— (the next). Frontier with the rest: here we don't re-teach how to draw C4 or write an ADR; we talk about what makes any doc artifact —a diagram, an ADR, a README— keep being true over time.

An analogy: the label on the machine vs. the manual in another building's basement

Think of a factory with a complicated machine —a press, an industrial furnace— and two ways of documenting how to operate it safely.

The manual in another building's basement. The company wrote, years ago, a thick and complete manual of the machine: every procedure, every warning, every valve. It printed it, bound it, and stored it in an archive in the basement of the head offices —two buildings from the machine—. The day it was written, the manual was perfect. But the machine, over the years, changed: a technician added a new safety valve, another changed the startup sequence, a third recalibrated the maximum pressure. Each of those changes should have gone into the manual, but updating the manual meant going to another building, down to the basement, finding the page, rewriting it —and no one did it, because there was always something more urgent—. Today the basement manual describes a machine that no longer exists: it says the maximum pressure is a value that stopped being true three years ago. A new operator who trusts it can get hurt. The manual is complete, bound, and dangerous, because it's out of sync with the real machine.

The label stuck on the machine. The same factory, for the critical stuff, does something else: it sticks labels on the machine itself. Next to the startup lever, a label with the sequence; next to the valve, a label with the maximum pressure; on the panel, a label with the warning. And there's a golden rule: when a technician modifies the machine, they update the label in the same job —they can't close the maintenance order without repainting the affected label—. When someone changes the maximum pressure, the label ten centimeters from the valve is updated on the spot, because it's there, in plain sight, impossible to ignore. The result: the machine's labels always tell the truth, because they live on the thing they describe and update as part of changing it. There's no "later" in another building; the change and the label happen in the same place, at the same time.

Here's the point, and it's the heart of living documentation: the doc that survives isn't the most complete or the best written; it's the one that lives so glued to the thing it describes that updating it is part of changing the thing. The basement manual was more complete than the labels —and yet it was useless and dangerous, because the distance desynced it—. The labels were brief —and yet they were the only reliable ones, because the proximity kept them alive—. The wiki disconnected from the code is the basement manual: complete on day one, rotted within a year, dangerous when someone trusts it. Living documentation is the label on the machine: it lives in the repo, next to the code, and updates in the same change. This lesson measures why the label wins, and why the wiki doesn't degrade slowly but falls off a trust cliff.

Worked example: the wiki that rots vs. the doc that lives

We're going to measure what the analogy claims. We take a system with 12 documented facts —12 things the doc asserts about the system—. The system evolves: in each release, a fraction of the facts changes (we call it drift, here 20% per release). The difference between the two docs is in a single variable: the probability that a fact gets re-synchronized when it changes.

The wiki lives far from the code, so updating it is a separate act that almost never happens: probability of syncing, 10%.
The live doc lives glued to the code and updates in the same PR that changes the system: probability of syncing, 97%.

And we add the key phenomenon: a trust floor. When a doc's accuracy falls below 70%, devs stop trusting all of it —because they can't tell which 70% is still fine—, so its effective value drops to zero. We measure the accuracy of the two docs release after release, and whether at each point they're trusted:

# Living documentation: the doc that lives GLUED to the code updates in the same
# change and stays fresh; the wiki far from the code updates almost never and
# rots. We model 12 documented facts. In each release, a fraction of the
# facts changes (drift). The "live" doc (next to the code) re-syncs almost
# always; the wiki almost never. And there's a chasm: when accuracy falls below a
# threshold, devs stop trusting ALL the doc (they don't know which part is still fine),
# so its effective value collapses to zero even if part is still correct.
DRIFT = 0.20            # 20% of the facts are touched per release
SYNC_LIVING = 0.97      # the doc next to the code updates in the same PR
SYNC_WIKI = 0.10        # the wiki far from the code almost never updates
TRUST_FLOOR = 0.70      # below this accuracy, trust in ALL the doc is lost


def accuracy(sync, k):
    # Fraction of the facts that's still correct after k releases.
    decay = DRIFT * (1 - sync)
    return (1 - decay) ** k


def effective_value(sync, k):
    # A doc that's not trusted is worth 0, even if part is still correct.
    a = accuracy(sync, k)
    return a if a >= TRUST_FLOOR else 0.0


print(f"{'release':>8}{'wiki %':>9}{'trust?':>9}{'live %':>9}{'trust?':>9}")
print("-" * 44)
for k in range(0, 9):
    aw, al = accuracy(SYNC_WIKI, k), accuracy(SYNC_LIVING, k)
    tw = "yes" if aw >= TRUST_FLOOR else "NO"
    tl = "yes" if al >= TRUST_FLOOR else "NO"
    print(f"{k:>8}{aw * 100:>8.1f}%{tw:>9}{al * 100:>8.1f}%{tl:>9}")
print("-" * 44)
k = 8
print(f"At release {k}: the wiki is {accuracy(SYNC_WIKI, k) * 100:.0f}% accurate,")
print(f"but it's WORTH {effective_value(SYNC_WIKI, k) * 100:.0f}% (no one trusts it).")
print(f"The live doc is {accuracy(SYNC_LIVING, k) * 100:.0f}% accurate and WORTH "
      f"{effective_value(SYNC_LIVING, k) * 100:.0f}%.")
print("The wiki crossed the trust floor at release 2 and never came back.")

What to expect. Running the file, the output is exactly this:

 release   wiki %   trust?   live %   trust?
--------------------------------------------
       0   100.0%      yes   100.0%      yes
       1    82.0%      yes    99.4%      yes
       2    67.2%       NO    98.8%      yes
       3    55.1%       NO    98.2%      yes
       4    45.2%       NO    97.6%      yes
       5    37.1%       NO    97.0%      yes
       6    30.4%       NO    96.5%      yes
       7    24.9%       NO    95.9%      yes
       8    20.4%       NO    95.3%      yes
--------------------------------------------
At release 8: the wiki is 20% accurate,
but it's WORTH 0% (no one trusts it).
The live doc is 95% accurate and WORTH 95%.
The wiki crossed the trust floor at release 2 and never came back.

Read the table slowly, column by column, because it has two lessons stacked on top of each other.

The first lesson: the wiki rots and the live doc doesn't. Notice the wiki % column. It starts at 100% —freshly written, everything true—, but it collapses: by release 1 it already lost almost a fifth (82%), by release 4 it's below half (45%), and by release 8 only 20% of what it says is still true. Eighty percent of the wiki lies. Why? Because each release changes 20% of the facts, and the wiki only re-syncs 10% of what changes —the rest stays written describing a system that no longer exists—. Now look at live %: it barely moves. It starts at 100% and by release 8 it's still at 95%. The same amount of change hits both docs; the difference is that the live doc re-syncs 97% of what changes, because updating it is part of the same PR that makes the change. The wiki and the live doc receive identical drift; what separates them is proximity to the code, and that single variable is the difference between 20% and 95% accuracy.

The second lesson, the one that makes everything worse: the trust chasm. Look at the trust? columns. The live doc says "yes" in every release —it's always above the 70% floor—. The wiki says "yes" only in releases 0 and 1; in release 2, at 67.2% accuracy, it crosses the floor and says "NO", and never comes back. And here's the blow: in release 8, the wiki is still 20% accurate —a fifth of what it says is still true—, but its effective value is 0%. How can something that's 20% correct be worth zero? Because no one knows which 20%. When you open a doc you know is mostly wrong, you can't use any part with confidence: each statement could be from the true fifth or the false four-fifths, and you have no way to tell without verifying against the code —and if you're going to verify everything against the code, the doc saved you nothing—. A partially rotted doc isn't worth its correct fraction; it's worth zero, because the uncertainty poisons the whole document.

That's the chasm, and it's why doc doesn't degrade smoothly but off a cliff. While accuracy is high, the small percentage of error is tolerated (you verify a rare case and move on). But there's a threshold —here 70%— below which the doc stops being "mostly correct with some error" and becomes "unreliable", and at that moment it loses all its value at once, not proportionally. The wiki crossed that threshold in release 2 —barely half a year, perhaps— and since then it's a dead document taking up space: people open it, see they can't believe it, close it, and go ask the person who knows. The doc exists, but the bus factor didn't drop a single point.

As bars, the contrast at release 8 looks like this:

Accuracy and effective value at release 8
  wiki      accuracy |####                20%   value |                    0%
  live doc  accuracy |###################  95%   value |################### 95%
             ─────────────────────────────
  The wiki is 20% true but worth 0: no one knows which 20%. The trust cliff.

Deep dive: what keeps doc alive, and what kills it

The experiment isolated the variable that decides everything: the probability of re-syncing a fact when it changes. It's worth understanding what determines that probability in the real world, because that's where the recipe for the doc that survives lives.

What makes the sync probability high is a single thing: the distance between the doc and the code. Not physical distance, but distance in the workflow. If updating the doc is on the path of making the change —if the same PR that changes payments includes the payments doc file, and the reviewer sees it—, the doc updates almost always, because not updating it would be a visible gap in the work. If updating the doc is off the path —if it lives in another tool, requires remembering, opening Confluence, finding the page— then it updates almost never, because "later" doesn't come: there's always something more urgent, and no one audits whether the wiki stayed up to date. Proximity isn't aesthetic; it's what determines whether updating the doc is an act that happens by default or an act that requires heroic discipline sustained for years (which is never sustained).

From there comes the deepest idea of living documentation, the one Cyrille Martraire pushes to the extreme: the best documentation is the one that isn't maintained apart, because it's derived from the source of truth. Think of it in degrees of proximity. The poorest degree is the wiki: totally separate doc, manual sync, rots. A better degree is the doc in the repo, next to the code, reviewed in the same PR (docs-as-code, lesson 3): sync forced by the flow, survives. The optimal degree, when possible, is doc generated from the code or the tests: a diagram drawn from the code's real structure, an endpoint list that comes from the router itself, usage examples that are the tests running in CI. That doc can't desync, because it doesn't exist apart: it's a projection of the source of truth. When the code changes, the generated doc changes with it, without anyone having to remember. You can't always generate everything —the why of a decision isn't derived from the code, it has to be written—, but the rule is clear: the closer to the source of truth the doc lives, the more it survives, and the disconnected wiki is the farthest possible point.

Now, the honest nuance, so as not to fall into a silly absolutism. "Living documentation" does not mean "generate everything automatically and write nothing". There's knowledge that isn't derived from the code and that has to be written by hand: above all the why —why payments is separated, why a queue was chosen instead of direct write—. The code shows what the system does, never why it was decided that way; that why is exactly what an ADR captures and what survives most (we'll see it in lessons 4 and 5). The lesson of living documentation isn't "don't write doc"; it's "write the doc that has to be written as close as possible to the code and update it in the same change, and generate from the code everything that can be generated, so the part you maintain by hand is the minimum —the stable, the why—". The doc that survives is a combination: the generated (which can't rot) plus the written-close-to-the-code (which stays alive by proximity).

There's a cultural consequence worth naming. In a team with living documentation, "documenting" stops being a separate phase —that "let's document the system" meeting that gets postponed forever— and becomes part of "changing the system": the PR isn't complete until its doc is up to date, just as it isn't complete until its tests pass. That sounds like more work, but it's less: keeping a label up to date when your hands are already on the machine costs minutes; rebuilding a rotted basement manual costs weeks —and by the time you rebuild it, it's rotted again—. Live doc is cheap because it's done in the moment; the wiki is expensive because it pretends to be done later, and "later" either doesn't come or comes so late that everything has to be redone.

Common mistakes

The wiki disconnected from the code (the basement manual). What happens: the team documents in a tool separate from the code —Confluence, Notion, a Docs folder— because it's comfortable to write there. The code keeps changing, the wiki stays still, and in months the wiki describes a system that no longer exists. Why it happens: writing far from the code doesn't require touching the repo or going through review, so in the moment it's the path of least resistance; the cost (the desync) appears later and belongs to someone else. How to spot it: if your doc lives outside the repo, if updating it is an act separate from changing the code, or if the answer to "is this up to date?" is "probably not", your wiki is the basement manual. How to fix it: bring the doc close to the code —to the repo, reviewed in the same PR (lesson 3)— and generate from the code everything that can be generated. Proximity is the only thing that keeps it alive; the discipline of "remembering to update the wiki" isn't sustained over years.

Believing a partially correct doc is worth its fraction. What happens: the team tolerates a doc that's "about 60% up to date" thinking 60% of value is better than nothing, and lets it rot a bit more each release. But it crosses the trust floor and, all at once, stops being used entirely: no one can tell which 60% is the good one. Why it happens: doc accuracy is thought of as a continuous scale (more accurate, more useful), when in fact there's a chasm —below a certain threshold, the value doesn't drop proportionally, it falls to zero—. How to spot it: if your doc has known errors that "almost no one looks at anymore", or if people prefer to ask rather than read the doc, you already crossed the chasm even if part is still fine. How to fix it: treat doc accuracy as a threshold, not a scale —keep it very high (with proximity and generation) or assume it's worth nothing—; a doc at 95% is used, one at 60% isn't used even if it has more absolute truth than its reputation.

Documenting everything by hand "so it's complete". What happens: the team tries to maintain an exhaustive doc by hand —every endpoint, every diagram, every detail— and since it's impossible to keep so much synchronized, everything rots equally, including the stable stuff that was worth it. Why it happens: "living documentation" is confused with "complete documentation", and completeness by hand is exactly what guarantees the rot, because there's too much to maintain. How to spot it: if your team has a huge, out-of-date doc, or if "maintaining the doc" feels like an infinite burden, you're maintaining by hand what should be generated or not documented. How to fix it: generate from the code everything that can be (doesn't rot), maintain by hand only the stable and the why (the minimum, lesson 5), and accept that the volatile is read from the code, not from the doc. Live doc is small and true, not big and rotted.

Exercises

Exercise 1 — Why 20% is worth 0. In the example, at release 8 the wiki is 20% accurate but its effective value is 0%. An engineer protests: "20% accuracy isn't zero; one in five things it says is true, that has some value". Explain why, in practice, that 20% is worth zero, and what would have to be true for a partially correct doc to actually be worth its fraction.

See solution

The 20% is worth zero in practice because no one knows which 20%. When you open the wiki and read a statement —"payments charges by direct write to provider X"—, you have no way to tell whether that statement is from the fifth that's still true or from the four-fifths that are already false. To use it with confidence you'd have to verify it against the code; and if you're going to verify each statement against the code, the doc saved you nothing —you might as well have read the code directly—. Worse still: there's a negative cost, because a false statement that looks authoritative (it's written in the official doc) can send you down the wrong path before you verify. So a doc that's mostly wrong isn't worth its correct fraction: it's worth zero or less, because the uncertainty about which part is correct poisons the whole document.

For a partially correct doc to actually be worth its fraction, it would have to be true that you can tell which part is correct without verifying it against the code. For example, if the doc marked each statement with its last-verification date and its status (verified in CI / written by hand / possibly obsolete), you could trust the verified part and ignore the rest. That's exactly what doc generated from the code or the tests achieves: each generated statement is true by construction, so its value isn't contaminated by the hand-written parts. The lesson: a doc's value doesn't depend only on what fraction is correct, but on whether you can distinguish the correct part —and a rotted wiki doesn't let you distinguish, which is why it's worth zero—.

Exercise 2 — Same drift, different fate. In the model, the wiki and the live doc receive exactly the same drift (20% of the facts change per release) and yet they end at 20% and 95% accuracy. Explain, without code, which single variable produces that enormous difference, what it represents in the real world, and why it can't be compensated for with "more discipline" to update the wiki.

See solution

The single variable that differs is the probability of re-syncing a fact when it changes: 97% for the live doc, 10% for the wiki. The same drift hits both, but the live doc brings almost everything that changed back up to date, while the wiki leaves 90% of what changed unupdated —and that unupdated residue is what accumulates release after release until it rots the document—. In the real world, that probability represents the distance between the doc and the code in the workflow: the live doc updates in the same PR that makes the change (updating it is on the path, almost automatic), the wiki updates in a separate act, in another tool, that requires remembering (updating it is off the path, almost never happens).

It can't be compensated for with "more discipline" because the discipline of updating a disconnected wiki requires that every person, in every change, over years, remembers to do a separate task that no one audits and that blocks nothing. That's not sustained: it's enough for the sync to fail a bit in each release —which is normal when it depends on memory and goodwill— for the residue to accumulate and the doc to rot. The solution isn't to ask for more sustained heroism (which always fails), but to change the structure so that updating the doc is on the path of the change and not off it: bring the doc close to the code, review it in the same PR, generate it from the source. When updating the doc is part of changing the code, the high sync probability comes for free; when it's a separate act, no amount of discipline sustains it.

Exercise 3 — What to generate and what to write. Living documentation pushes you to generate from the code everything you can, and write by hand only the minimum. For Mercado's payments module, classify these four doc pieces into "can be generated from the code" or "has to be written by hand", and explain why: (a) the list of endpoints payments exposes; (b) why payments is separated from the core and charges by queue instead of direct write; (c) the diagram of which other modules call payments; (d) the business rules of which transactions can be refunded.

See solution

(a) List of endpoints → can be generated. The endpoints payments exposes are defined in the code itself (the router, the routes). An endpoint list written by hand rots on the first change; one generated from the router (for example, an OpenAPI that comes from the code) is true by construction and updates itself when an endpoint is added or removed. It's generated, not written.

(b) Why payments is separated and charges by queue → has to be written by hand. This is the why of a decision, and the why isn't in the code: the code shows that there's a queue, never why the queue was chosen instead of direct write (isolate the external traffic from the core, accept a delay in exchange for safety). That reasoning has to be written —it's exactly what an ADR captures—, and it's the most stable and what survives most. It's written by hand, close to the code, and barely changes.

(c) Diagram of who calls payments → can be generated (largely). The dependencies between modules —who imports or calls payments— are in the code and can be extracted with dependency-analysis tools, producing a diagram that updates with the code. A dependency diagram drawn by hand desyncs; one generated from the real structure doesn't. (The highest-level C4, with the intent and the boundaries, does need a hand; the dependency detail is generated.)

(d) Refund business rules → mixed case, tends to be written. The implemented rules are in the code (and ideally in the tests, which can serve as executable live doc of "what's refunded and what isn't"). But the business intent —why the rule is that way, what business case motivated it— is written by hand. The living-documentation ideal here is that the refund-rule tests are the doc of what the system does (they can't rot, because if the code changes and the test doesn't, the test fails), and that only the business why is documented by hand. They combine: the what is derived from the tests, the why is written.

The general pattern: the what (endpoints, dependencies, implemented rules) is generated from the code or the tests, because that's where the truth lives and it can't rot; the why (decisions, intent, boundaries) is written by hand, close to the code, because it's nowhere else and it's the most stable. That combination —generate the what, write the why— is documentation that survives.

Summary and next step

In this lesson you understood why most documentation doesn't survive: it rots because it lives far from the code and updating it is a separate act that almost never happens. You saw, with the label on the machine vs. the manual in the basement, that the doc that survives isn't the most complete but the one that lives so glued to the thing that updating it is part of changing it —living documentation—. And you measured it twice: first, that the same drift produces 20% accuracy in the wiki against 95% in the live doc, and that the single variable that explains it is proximity to the code; second, the trust chasm —when accuracy falls below 70%, the whole doc is worth zero even if part is still correct, because no one knows which part, and the wiki crosses that floor as early as release 2—. You learned the recipe: generate from the code everything you can (doesn't rot), write by hand only the stable and the why (the minimum), and keep it all close to the code.

Before moving on you should be able to: explain why a 20%-correct doc is worth zero (the trust chasm); name the single variable that separates the wiki from the live doc (proximity to the code, not discipline); and classify what's generated and what's written by hand.

Lesson 3 takes this lesson's principle —proximity keeps doc alive— and turns it into a concrete practice: docs-as-code. You'll see how to put the doc in the repo, in Markdown and PlantUML, reviewed in the same PRs as the code, and —most powerful— validated in CI: a test that checks the doc hasn't desynced from the code and breaks the build if an ADR references a module that no longer exists. With that, "the doc rotted" stops being an invisible problem discovered months later and becomes a red test that fires on the spot. This lesson's proximity, made mechanical.

Resources

Cyrille Martraire, Living Documentation: Continuous Knowledge Sharing by Design (Addison-Wesley, 2019), part I — the source of this lesson's concept: the doc that survives is the one that lives glued to the code and, when possible, is derived from the source of truth instead of maintained apart. In English.
Write the Docs — "Docs as Code" — the community that pushes treating doc as code: in the repo, in plain text, close to what it describes. A good bridge to lesson 3. In English.
Martin Fowler — "Living Documentation" and the architecture hub — on why useful doc is the one kept alive by proximity, not the one written once. In English.
Andrew Hunt and David Thomas, The Pragmatic Programmer, 20th Anniversary Edition (Addison-Wesley, 2019), topic "It's All Writing" and the DRY principle applied to doc — why duplicating knowledge between the code and a separate doc guarantees they desync. In English.

← PreviousModule introduction: documentation that survives Next →Docs-as-code