Module 6: Designing for Change
The YAGNI line
Overview
With the three pieces of the stance installed —flow, optionality, sacrificial— the second half of the module begins: the two abysses the architect who designs for change walks between. This lesson opens the first, the most seductive because it disguises itself as prudence: over-engineering, and the line that marks it, YAGNI —You Aren't Gonna Need It—. It's the mistake of building flexibility, abstractions, and layers "just in case" for futures the architect imagines but that almost never arrive. The generic plugin system that will support extensions no one asked for, the configurable rules engine for rules that only change once a year, the microservices "because someday we'll scale": each sounds responsible, and each pays in advance the cost of a problem it may never have.
YAGNI is counterintuitive because it contradicts an instinct that feels virtuous: "prepare for the future". Lesson 3 already planted the answer —an option costs a premium and is only worth it above its break-even point—; this lesson takes it to the portfolio extreme. When the anxious architect looks at all the futures they could imagine and decides to pre-build the flexibility for each one, they make the mistake of buying insurance against dragons: they pay many premiums for events that won't happen. And since the vast majority of imagined futures are low-probability, the cost of carrying all that unused flexibility far exceeds simply paying the rework of the few changes that really arrive. This lesson measures it, and the result is an order of magnitude.
Connection with the module. It's the first abyss (this one: over-engineering; lesson 6: under-engineering; lesson 7: the sensible point between the two). It rests directly on lesson 3: over-engineering is buying options below their break-even point —paying premiums for changes too improbable—. And it sets up the tension lesson 7 resolves: if this lesson said only "don't build anything in advance", you'd fall into the opposite abyss (under-engineering); that's why lesson 6 corrects and lesson 7 finds the middle. Frontier with the sister guide: here we don't measure technical debt or the carrying cost formally; we work the stance of the architect who resists the temptation to build for every imaginable future, and knows how to distinguish the probable change from the imagined one.
An analogy: the urban planner who paves twelve lanes for the town
Go back to the urban planner of lesson 1, but now focus on the second one —the one who makes the mistake of excess—, because they're the protagonist of this lesson.
A town of three thousand inhabitants needs, today, a couple of streets and a two-lane road connecting it to the city. That's what the traffic asks for. But the planner is terrified of the future: "what if the town grows? what if it becomes a metropolis? Better build the infrastructure for five million inhabitants right now". So they pave a twelve-lane avenue with interchanges, build an international airport, lay out an underground metro network, and install electrical capacity for an industrial city. All that for three thousand people.
What happens? First, the cost is astronomical: infrastructure for five million costs orders of magnitude more than what the town needs, and the town goes into debt for generations. Second, almost nothing gets used: twelve lanes with three cars passing, an airport with one weekly flight, an empty metro. Third —and this is what almost no one anticipates—, maintaining what isn't used costs: the avenue's concrete cracks in the sun without a single car to justify it, the airport needs staff and maintenance, the metro consumes electricity. Oversized infrastructure isn't free while it waits for a future that doesn't arrive; it drains the town's budget every year. And fourth, by the time (or if) the town grows —in thirty years—, the avenue will already be obsolete, the airport small, and it'll all have to be redone anyway. The planner paid a fortune in advance, maintained a white elephant for decades, and didn't even guess right the future they built for.
Contrast with the good planner of lesson 1: they don't pave twelve lanes; they reserve the right of way —a cheap strip where, if the traffic arrives, the avenue will be opened then—. The difference is enormous: reserving land costs almost nothing and requires no maintenance; paving twelve lanes costs a fortune and drains the budget waiting for traffic that may not come. Over-engineering is paving the twelve lanes today. YAGNI is the discipline of not doing it —of building what the traffic asks for and, at most, reserving the right of way for the probable, not paving in advance for every imaginable future—. This lesson measures the cost of paving the twelve lanes.
Worked example: pre-building all futures vs. paying the rework of the ones that arrive
We're going to measure the cost of over-engineering. An anxious Mercado architect looks at a portfolio of imagined futures —changes they fear and want to pre-build today, "just in case"—. Each future has three numbers: the real probability that it arrives (p_happens), what it costs to pre-build its flexibility today (flex_now), and what it would cost to adapt later if it arrives and you didn't pre-build it (rework).
We compare two stances:
over_engineered— the anxious architect: pre-builds the flexibility of all imagined futures. Pays all theflex_now, regardless of whether the future arrives or not. It's paving the twelve lanes.yagni— the discipline: pre-builds nothing. Only pays thereworkof the futures that really arrive —and since each has its probability, the expected cost isp_happens × reworksummed over all—. It's building what the traffic asks for, when it asks for it.
The key point that makes the difference: the "just in case" imagined futures are, almost by definition, low-probability. Notice the p_happens in the portfolio —all small—, because it's exactly what characterizes the anxious architect's fear: they fear many futures, but each one is improbable:
# YAGNI ("You Aren't Gonna Need It"): building flexibility "just in case" for
# imagined futures that almost never arrive. A portfolio of changes an
# ANXIOUS architect fears and wants to pre-build today. Each one:
# p_happens : real probability that that future arrives
# flex_now : what it costs to build the flexibility TODAY, in advance
# rework : what it would cost to adapt LATER if it arrives and you didn't pre-build it
FUTURES = [
# (name, p_happens, flex_now, rework)
("multi_currency_before_expansion", 0.05, 15000, 20000),
("plugin_system_for_unknown_addons", 0.03, 25000, 18000),
("swap_database_vendor", 0.04, 20000, 30000),
("generic_workflow_engine", 0.02, 30000, 22000),
("support_5_payment_providers", 0.08, 12000, 16000),
("white_label_multitenant", 0.03, 28000, 26000),
]
# over_engineered: pre-builds the flexibility of ALL imagined futures.
over_engineered = sum(flex for _, _, flex, _ in FUTURES)
# yagni: pre-builds nothing; only pays the (expected) rework of the ones that DO arrive.
yagni = sum(p * rework for _, p, _, rework in FUTURES)
expected_arrivals = sum(p for _, p, _, _ in FUTURES)
print(f"{'imagined future':<36}{'p':>6}{'flex_now':>10}{'p*rework':>10}")
print("-" * 62)
for name, p, flex, rework in FUTURES:
print(f"{name:<36}{p:>6.2f}{flex:>10,}{p * rework:>10,.0f}")
print("-" * 62)
print(f"{'over_engineered (pre-builds EVERYTHING)':<52}{over_engineered:>10,}")
print(f"{'yagni (pays rework only for the ones that arrive)':<52}{yagni:>10,.0f}")
print()
print(f"Of {len(FUTURES)} pre-built futures, {expected_arrivals:.2f} are expected to arrive.")
print(f"Over-engineering pays {over_engineered / yagni:.0f}x more to carry flexibility")
print("that, in its vast majority, will never be used. YAGNI: don't build it until you need it.")What to expect. Running the file, the output is exactly this:
imagined future p flex_now p*rework
--------------------------------------------------------------
multi_currency_before_expansion 0.05 15,000 1,000
plugin_system_for_unknown_addons 0.03 25,000 540
swap_database_vendor 0.04 20,000 1,200
generic_workflow_engine 0.02 30,000 440
support_5_payment_providers 0.08 12,000 1,280
white_label_multitenant 0.03 28,000 780
--------------------------------------------------------------
over_engineered (pre-builds EVERYTHING) 130,000
yagni (pays rework only for the ones that arrive) 5,240
Of 6 pre-built futures, 0.25 are expected to arrive.
Over-engineering pays 25x more to carry flexibility
that, in its vast majority, will never be used. YAGNI: don't build it until you need it.
Read the table column by column, because the contrast between flex_now and p*rework is the whole argument.
The flex_now column is what the anxious architect pays in advance, guaranteed. Pre-building the multi-currency system: 15000. The generic plugin system: 25000. The abstracted database-vendor swap: 20000. The configurable workflow engine: 30000. And so on. It totals 130000 dollars, and they pay it now, used or not. It's the cost of paving the twelve lanes: real, present, guaranteed.
The p*rework column is what it costs, in expected value, to not pre-build and adapt if the future arrives. Multi-currency: 5% probability times 20000 of rework = 1000 expected. The plugin system: 3% times 18000 = 540. And so on. It totals 5240 dollars. Notice why it's so small: each term is multiplied by a small probability, because these imagined futures almost never arrive. It's not that adapting is free; it's that most of the time there's no need to adapt, because the feared future doesn't arrive.
And there's the number that defines the lesson: 130000 against 5240, twenty-five times more expensive to pre-build everything. Over-engineering pays 25x to carry a flexibility that, in its vast majority, will never be used. The code line says it bluntly: of the 6 pre-built futures, 0.25 are expected to arrive. Zero point twenty-five. The anxious architect built six flexibilities for a future that, on average, will use a quarter of one. They paid for six insurance policies against dragons and, in expectation, no dragon came.
Notice the mechanism, because it's the heart of YAGNI: over-engineering pays the cost of the flexibility with certainty (guaranteed, today) in exchange for avoiding a rework that is uncertain (probable only for a few futures). It trades a sure cost for avoiding an improbable cost —the worst possible deal under uncertainty—. YAGNI does the opposite: it pays nothing in advance and takes on the rework only when —and if— the future materializes. Since most imagined futures don't materialize, YAGNI almost always wins in the aggregate. Not because the future never brings changes (some it does: the expected value of the rework isn't zero), but because pre-building all imaginable changes is paying for a mountain of flexibility of which a wisp will be used.
As bars, the disproportion looks like this:
Total cost (USD): over-engineering vs YAGNI
over_engineered |################################ 130,000
yagni |# 5,240
────────────────────────────────
25x more expensive to pre-build all imagined futures
than to pay the rework of the ~0.25 that really arrive.
Deep dive: why over-engineering disguises itself as prudence
Over-engineering is dangerous precisely because it doesn't feel like a mistake —it feels like responsibility—. It's worth understanding the disguises it uses, because recognizing them is half of resisting them.
The "prepare for the future" disguise. "A good architect thinks long-term" is true, but it distorts into "a good architect builds the long-term today". Those are very different things. Thinking long-term is reserving the right of way —leaving the cheap seam for the probable change, as lesson 3 taught—; building the long-term today is paving the twelve lanes for every imaginable future. The first is prudence; the second is anxiety disguised as prudence. The question that unmasks the disguise: "is this future probable or only imaginable?". Almost anything is imaginable —Mercado could become a social network, could need 40 languages, could want a plugin marketplace—; the imaginable is infinite and building all of it is impossible. Only the probable justifies building in advance, and even then, often it's enough to reserve the right of way, not to pave.
The elegant-generalization disguise. As engineers we're seduced by the general: instead of solving this problem, solving the whole class of problems this is a case of. "Let's not make a discount calculation; let's make a configurable rules engine that can express any future discount". It sounds smarter, more professional. But a rules engine for a rule that changes once a year is twelve lanes for three cars: all the complexity of the general, none of its benefits. The golden rule here is the "three appearances" one: don't generalize on the first need, nor the second; wait for the third, when the real pattern has revealed itself and the generalization rests on evidence, not on imagination. Generalizing before seeing the pattern is guessing the shape of the abstraction —and it's almost always guessed wrong, whereby the "flexible" abstraction turns out flexible in the wrong dimensions—.
The cost no one counts: the carrying cost. The example measured the cost of building the unused flexibility (130000), but there's a cost the model didn't include and that in practice is worse: the carrying cost. Each abstraction, each configuration layer, each "just in case" indirection not only costs to build; it costs to carry forever. It makes the code harder to read (you have to understand the generalization to make a simple change), harder to change (changes have to respect a flexibility nobody uses), harder to debug (more moving parts), slower to onboard (each new person has to learn the generic machinery). It's the maintenance of the empty airport: oversized infrastructure drains budget every year, not just the day it's built. That's why over-engineering is doubly expensive: you pay to build what you don't use and you pay to carry it for the whole life of the system. YAGNI saves both things.
The nuance that avoids the misunderstanding —and that lesson 6 develops—. YAGNI is not "never build anything in advance" nor "don't think about the future". That would be the opposite abyss: the under-engineering that paints itself into a corner. YAGNI is more precise: don't build the flexibility until the change is probable enough to justify it —until it crosses its break-even point (lesson 3)—. For probable changes, yes you leave the seam (you reserve the right of way); for the imaginable but improbable ones, no. The YAGNI line isn't at "zero preparation"; it's at "preparation proportional to probability". This lesson's mistake is over-preparing (for everything imaginable); the next lesson's mistake is under-preparing (for nothing). The art, which lesson 7 measures, is preparing just enough —for the probable—.
Common mistakes
Building the abstraction before seeing the pattern. What happens: on the first (or second) appearance of a need, the architect builds a general abstraction "to cover future cases" —the rules engine, the plugin system, the configurable layer— before the real pattern has revealed itself. The abstraction turns out flexible in the wrong dimensions, because its shape was guessed. Why it happens: the general seduces (it looks smarter than solving the concrete case) and "preparing" feels responsible. How to spot it: if there are abstractions with a single real use case, or configurable engines whose configuration never changes, it was generalized too early. How to fix it: the rule of three appearances —solve the concrete case the first two times, and generalize on the third, when the pattern has revealed itself and the abstraction rests on evidence, not on imagination—. Guessing the shape of the abstraction almost always fails.
Confusing "thinking long-term" with "building the long-term today". What happens: the architect justifies pre-building infrastructure for imagined futures by saying "we have to think long-term", and paves the twelve lanes in advance. Why it happens: "thinking long-term" is a real virtue, and it's easy to slide from there to "building the long-term now" without noticing they're different things. How to spot it: if the justification for building something is "someday" / "just in case" / "we might need", with no probable and near change backing it, it's building the long-term today. How to fix it: ask "is this future probable or only imaginable?" —the imaginable is infinite and isn't built; only the probable justifies preparation, and often it's enough to reserve the right of way (the cheap seam) instead of paving—. Thinking long-term is leaving the cheap possibility open, not building the expensive future in advance.
Ignoring the carrying cost of the unused flexibility. What happens: an abstraction "just in case" is justified looking only at its build cost ("it doesn't cost that much to add it now"), without counting what it will cost to carry over the whole life of the system —harder to read, change, debug, and learn—. Why it happens: the build cost is a single, visible event; the carrying cost is an invisible drip paid on every future change. How to spot it: if each new person takes time to understand generic machinery no one uses, or if simple changes require respecting flexibility that adds nothing, you're paying carrying cost. How to fix it: count the total cost —build plus lifetime carrying— when evaluating a flexibility, not just building it; most abstractions "cheap to add" are expensive to carry. The empty airport doesn't cost only to build; it costs to maintain year after year.
Exercises
Exercise 1 — Probable or imaginable? For each proposal at Mercado, say whether it's legitimate preparation for a probable change (go ahead) or over-engineering for an imaginable future (YAGNI, don't build it), and justify: (a) building a generic plugin system for third parties to extend Mercado, when no third party has asked and there's no platform strategy; (b) leaving an interface over the payment provider, when finance is already negotiating a second provider; (c) building support for 40 languages and 30 currencies, when Mercado operates in a single country with no expansion plans; (d) designing a configurable rules engine for discounts, when the discount rules change once a year and always in the same way.
See solution
-
(a) Over-engineering (YAGNI, don't build it). A plugin system for third parties nobody asked for and with no platform strategy is an imaginable future, not a probable one. It's the airport for the town: you build enormous generic machinery for extensions that don't exist. Besides, it has a high carrying cost (the whole system has to respect the plugin mechanics). Don't build it until there's a real platform strategy and concrete demand.
-
(b) Legitimate preparation (go ahead, but as a seam, not as pavement). Finance is already negotiating the second provider: the change is probable and near. Leaving an interface over the payment provider is reserving the right of way —cheap, proportional to a high probability—. Note that this is leaving a seam, not building both providers in advance; it's still proportional preparation, not over-engineering.
-
(c) Over-engineering (YAGNI, don't build it). One country, no expansion plans: 40 languages and 30 currencies are twelve lanes for three cars. Imaginable future, not probable. The complexity of a complete internationalization system would be carried over the whole life of the system without being used. Don't build it; if someday the expansion becomes probable, then yes prepare —and even then, maybe a seam will be enough—.
-
(d) Over-engineering (YAGNI, don't build it). A configurable rules engine for something that changes once a year and always the same way is the elegant generalization that isn't justified: all the complexity of the general, none of its benefits. Solve the concrete discount; if someday the rules start changing often and in varied ways (the pattern reveals itself, the third appearance), then generalize. Guessing the engine now will almost surely make it flexible in the wrong dimensions.
The pattern: (b) is probable → prepare (with a seam); (a), (c), (d) are imaginable → don't build. The deciding question is always "probable or only imaginable?".
Exercise 2 — Where the 25x comes from. The example shows that over-engineering costs 25 times more than YAGNI. An architect objects: "but that's because you chose low-probability futures; with probable futures, pre-building would win". Are they right? Explain what the example assumes, why that assumption is correct for characterizing over-engineering, and what would happen with high-probability futures.
See solution
The architect is right about the mechanics —the 25x depends on the probabilities being low— but wrong about the conclusion, because the low probability is the definition of over-engineering, not a trick of the example. Over-engineering, by definition, is building for imagined but improbable futures: the "just in case". An anxious architect fears many futures (that's why the portfolio has six), but each one individually is improbable (that's why the ps are small, summing 0.25 expected). Choosing low probabilities doesn't bias the example; it faithfully portrays the situation the lesson criticizes. If the futures were probable, it would no longer be over-engineering —it would be legitimate preparation—.
What would happen with high-probability futures: if a future has, say, 90% probability, its p × rework term is no longer small, and pre-building its flexibility (leaving the seam) starts to win —exactly the break-even calculation of lesson 3—. And that's the point that connects with the following lessons: for probable futures, you do want to prepare (leave a seam); refusing to do it is the opposite abyss, the under-engineering of lesson 6. Over-engineering's mistake isn't "preparing"; it's preparing for the improbable. The architect of the objection, without meaning to, described the solution: distinguish futures by probability, prepare the probable, leave the improbable. That's exactly the right-sizing of lesson 7. The 25x doesn't say "never prepare"; it says "don't prepare for the improbable" —which is a different thing—.
Exercise 3 — The cost the model didn't measure. The example measured the cost of building the unused flexibility (130000), but the text says there's a worse cost the model didn't include. Name it, explain why in practice it's usually greater than the build cost, and give a concrete Mercado example of the configurable rules engine from exercise 1.
See solution
The cost the model didn't measure is the carrying cost: what it costs to carry the unused flexibility over the whole life of the system, beyond building it. Each abstraction, configuration layer, or "just in case" indirection makes the system harder to read, change, debug, and learn —forever, not just the day it's built—.
Why it's usually greater than the build cost: the build cost is a single event (paid once, 130000 in the example); the carrying cost is a drip paid on every future interaction with the code —each change has to understand and respect the generic machinery, each new person has to learn it, each bug is harder to trace because of the extra moving parts—. Summed over years and over the whole team, that constant drip usually far exceeds the single outlay of building it. It's the empty airport: building it was expensive once, but maintaining it drains the budget every year it waits for passengers who don't arrive.
Concrete Mercado example with the configurable rules engine (from exercise 1d): building the engine costs, say, 30000 once. But then, every time someone touches the discounts —even for a trivial change— they have to understand the engine's configuration language, not a simple calculation; every new dev who arrives at Mercado takes days to understand how the generic engine works before they can change a discount; every bug in a discount is harder to trace because the logic lives in interpreted configuration, not in direct code; and all that for rules that change once a year. The carrying —that constant friction multiplied by years and by the whole team— ends up costing much more than the 30000 to build the engine, and all for a flexibility that's barely used. A simple if would have been cheaper to build and to carry.
Summary and next step
In this lesson you opened the module's first abyss: over-engineering, and the YAGNI line. You saw, with the urban planner who paves twelve lanes for a town, that building flexibility for every imaginable future isn't prudence but disguised anxiety —expensive to build, and worse still to carry—. And you measured it: pre-building the flexibility of six imagined futures costs 130000 against 5240 to pay the rework only of the ones that really arrive —25 times more—, because of the six futures barely 0.25 is expected to arrive. You understood the mechanism (trading a sure cost for avoiding an improbable one, the worst deal under uncertainty), the disguises of over-engineering (preparing for the future, the elegant generalization), the carrying cost no one counts, and the crucial nuance: YAGNI isn't "never prepare anything", it's "prepare proportional to probability".
Before moving on you should be able to: distinguish a probable future (prepare) from an imaginable one (don't build); apply the rule of three appearances before generalizing; and explain why the carrying cost usually exceeds the build cost.
But be careful: if you keep only "don't build anything in advance", you fall into the opposite abyss. Lesson 6 opens it: under-engineering, painting yourself into a corner. You'll see the mistake of refusing to leave a single seam and being left with no way out when the probable change arrives —the house with everything set in concrete—, and execute how much it costs to rewrite without a seam against having left the cheap seam. With numbers, so it's clear that YAGNI has a limit: "don't prepare for the improbable" isn't the same as "don't prepare for anything".
Resources
- Martin Fowler, "Yagni" (2015) — martinfowler.com/bliki/Yagni.html. The lesson's canonical text: why building a "presumptive capability" for an imagined future almost always comes out expensive, and why the cost includes the carrying cost, not just the build cost. Short and essential. In English.
- Martin Fowler, "Design Stamina Hypothesis" (2007) — martinfowler.com/bliki/DesignStaminaHypothesis.html. The complement that avoids the misunderstanding: there is a level of design that is worth it, and the trick is not to overshoot (over-engineering) nor fall short (under-engineering, lesson 6). In English.
- Andrew Hunt and David Thomas, The Pragmatic Programmer, 20th Anniversary Edition (Addison-Wesley, 2019), topics "Good-Enough Software" and "The Evils of Duplication" — on not over-building and on when generalization (DRY) helps and when it gets in the way. In English.
- Sandi Metz, "The Wrong Abstraction" (2016) — sandimetz.com/blog/2016/1/20/the-wrong-abstraction. The best essay on why generalizing before seeing the pattern (guessing the shape of the abstraction) comes out more expensive than the duplication it tried to avoid. The basis of the rule of three appearances. In English.