Module 2: Understanding and Stabilizing the Legacy System

The characterization test: pin what it does TODAY

Next →

Overview

You already know what a characterization test does (it pins the current behavior) and why you need it (without the net, blind change costs a fortune). This lesson teaches how you write it, and it does so by pausing on the trap that catches almost everyone the first time. When you sit down to write a test, your instinct —well trained— is this: first I think what the correct answer is, I write it as the expected value, and then I verify that the code produces it. That instinct is exactly right for the code you design, and exactly wrong for the legacy code you're characterizing. With the legacy you don't know what the "correct" answer is —for that you'd have to understand years of accumulated rules— and, worse, you don't care: you're not verifying that the price is correct, you're pinning what the legacy returns today. You don't calculate the expected answer: you capture it by running the code and reading what comes out.

That inversion —capture instead of guess— is the whole art of the characterization test, and it sounds easier than it is, because the instinct to "fix along the way" fights you at every line. If while reading the code you see a rounding that seems wrong and you write the expected value it should give, your test will come out red against the legacy —and the danger is that you'll then "fix" the legacy to pass your test, changing the production behavior without anyone having decided it—. A well-made characterization test does the opposite: it surrenders to what the legacy does, transcribes it with total fidelity, quirks included, and only afterward —as a separate and conscious decision— can someone ask whether that behavior is the desired one.

Connection with the module. Lesson 2 justified the net; this one teaches you to weave it. It's the module's central technique: the lessons that follow extend it (lesson 4 makes it possible when a dependency prevents it; lesson 5 scales it to hundreds of cases; lesson 6 faces what to do when what you pinned is a quirk someone depends on). Here we install the fundamental gesture: capture the current output, don't guess the ideal one. The boundary with the Testing ecosystem holds firm: we won't discuss how a suite is structured or what an assert is; we'll use the assertion as an instrument to pin behavior before migrating it.

An analogy: the court stenographer

In a courtroom there's a person whose job is to write, word for word, everything that's said: the stenographer. And their golden rule is one that would give an editor chills: transcribe exactly what was said, errors included. If a witness says "I seen it" instead of "I saw it," the stenographer writes "seen." If someone gets a date wrong, they write it wrong. If a lawyer stammers, the stammers stay. Why not "fix it along the way," if it would be neater? Because their job isn't to produce a correct text; it's to produce a faithful record of what actually happened. That record has legal value precisely because of its fidelity: if the stenographer "improved" what was said, the transcript would stop being proof of anything. Correction is the task of another moment and another person —the judge will decide what weight to give that "seen"—; the stenographer's is to capture, not judge.

The characterization test is the legacy's stenographer. You run the function, see that for a certain item it returns 106.88 —a number that comes from applying the loyalty after the tax, something that seems like an error to you— and you transcribe it as is: 106.88. You don't write 106.89, which is what it "should" give per your clean math. You write what the legacy actually said. Like the stenographer, your job here isn't to produce the correct price; it's to produce a faithful record of what the legacy does today. If you "fix it along the way," your record stops serving as a net —it no longer pins the real behavior, it pins your opinion about it—. And the question of whether 106.88 is right or wrong belongs to another moment and another person: the business will decide (lesson 6). The characterization test just takes stenography.

Keep this image, because it's the gesture to learn: facing the legacy, transcribe, don't edit.

Worked example: don't guess, capture

We're going to build a three-step characterization test over the catalog price, and the example will make the trap visible. First, for three items, we compare the price you'd "guess" with clean math against the one the legacy actually returns. You'll see they don't match. Then we capture the real output and pin it. And at the end we look at what the test pinned, on purpose, in the weird case.

import math

TAX_RATE = 0.16
CATEGORY_DISCOUNT = {"electronics": 0.05, "books": 0.10, "toys": 0.08, "grocery": 0.00}
GOLD_LOYALTY_DISCOUNT = 0.03


def _floor_cents(a):
    return math.floor(a * 100) / 100


def legacy_catalog_price(item):
    price = _floor_cents(item["unit_price"] * item["quantity"])
    price = _floor_cents(price * (1 - CATEGORY_DISCOUNT.get(item["category"], 0.0)))
    price = _floor_cents(price * (1 - item.get("coupon_percent", 0.0)))
    price = _floor_cents(price * (1 + TAX_RATE))
    if item.get("loyalty") == "gold":
        price = _floor_cents(price * (1 - GOLD_LOYALTY_DISCOUNT))
    return price


def guessed_price(item):
    # What it "should" cost per clean math: single rounding, gold pre-tax.
    price = item["unit_price"] * item["quantity"]
    price *= (1 - CATEGORY_DISCOUNT.get(item["category"], 0.0))
    price *= (1 - item.get("coupon_percent", 0.0))
    if item.get("loyalty") == "gold":
        price *= (1 - GOLD_LOYALTY_DISCOUNT)
    price *= (1 + TAX_RATE)
    return round(price, 2)


ITEMS = [
    {"sku": "A-1", "unit_price": 100.00, "quantity": 1, "category": "electronics"},
    {"sku": "D-4", "unit_price": 100.00, "quantity": 1, "category": "electronics",
     "loyalty": "gold"},
    {"sku": "E-5", "unit_price": 49.00,  "quantity": 2, "category": "books",
     "coupon_percent": 0.15},
]

print("Step 1 - do NOT guess the 'correct' answer. Compare guessed vs real:")
print(f"{'sku':<6}{'guessed':>12}{'real (legacy)':>16}{'equal?':>11}")
for it in ITEMS:
    g, r = guessed_price(it), legacy_catalog_price(it)
    print(f"{it['sku']:<6}{g:>12}{r:>16}{str(g==r):>11}")
print("A test written with the 'guessed' column would come out RED against TODAY's")
print("legacy. And 'fixing it' to pass would change the price in production.\n")

print("Step 2 - CAPTURE the real output and pin it as the expected value:")
# The technique: you run the legacy, READ what it gives, and paste that number as expected.
captured = {it["sku"]: legacy_catalog_price(it) for it in ITEMS}
for sku, value in captured.items():
    print(f"  assert legacy_catalog_price(item_{sku}) == {value}")
print()

print("Step 3 - Run the characterization_test with the captured values:")
fails = 0
for it in ITEMS:
    expected = captured[it["sku"]]
    actual = legacy_catalog_price(it)
    ok = actual == expected
    fails += 0 if ok else 1
    print(f"  {'PASS' if ok else 'FAIL'}  {it['sku']}  "
          f"expected={expected:<8} actual={actual}")
print(f"  -> {'GREEN' if fails==0 else 'RED'}: the test does NOT say the price is "
      f"'right'; it says it's still today's.\n")

print("What the test pins on purpose (the D-4 quirk):")
d4 = ITEMS[1]
print(f"  'clean' math would say:  {guessed_price(d4)} (gold before the tax)")
print(f"  the legacy charges:      {legacy_catalog_price(d4)} (gold AFTER the tax)")
print("  The characterization_test pins 106.88, the quirk included. If tomorrow")
print("  someone 'fixes' it, the test turns RED and forces them to decide it on purpose.")

What to expect. When you run the file, the output is exactly this:

Step 1 - do NOT guess the 'correct' answer. Compare guessed vs real:
sku        guessed   real (legacy)     equal?
A-1          110.2          110.19      False
D-4         106.89          106.88      False
E-5          86.97           86.96      False
A test written with the 'guessed' column would come out RED against TODAY's
legacy. And 'fixing it' to pass would change the price in production.

Step 2 - CAPTURE the real output and pin it as the expected value:
  assert legacy_catalog_price(item_A-1) == 110.19
  assert legacy_catalog_price(item_D-4) == 106.88
  assert legacy_catalog_price(item_E-5) == 86.96

Step 3 - Run the characterization_test with the captured values:
  PASS  A-1  expected=110.19   actual=110.19
  PASS  D-4  expected=106.88   actual=106.88
  PASS  E-5  expected=86.96    actual=86.96
  -> GREEN: the test does NOT say the price is 'right'; it says it's still today's.

What the test pins on purpose (the D-4 quirk):
  'clean' math would say:  106.89 (gold before the tax)
  the legacy charges:      106.88 (gold AFTER the tax)
  The characterization_test pins 106.88, the quirk included. If tomorrow
  someone 'fixes' it, the test turns RED and forces them to decide it on purpose.

Read step 1 carefully, because there the trap is made visible. For the three items, the price the "clean" math guesses and the one the legacy actually returns don't match —110.2 vs 110.19, 106.89 vs 106.88, 86.97 vs 86.96—. In all three, the difference is a cent, and in all three the legacy charges less, because it rounds down at every step while the clean version rounds only once at the end. Now imagine you fell into the trap: you wrote the test with the "guessed" column —the values that seemed correct to you—. That test would come out red against today's legacy. And here comes the dangerous part: the natural reaction to a red test is "fix the code to make it pass." If you did, you'd change the legacy's rounding so it returns 110.2 instead of 110.19... altering the production price to match your opinion of how it should be calculated. You'd have used a test as an excuse to change behavior without anyone deciding it. That's the disaster the trap produces.

Step 2 is the way out: instead of guessing, you capture. You run the legacy, read what it returns (110.19, 106.88, 86.96) and paste those numbers as expected values. Notice the three assert lines it prints: they're the literal characterization test. They don't say "A-1's price should be 110.19 because I calculated it that way"; they say "A-1's price is 110.19 today, and I want it to stay that way." The expected value came from the legacy itself, not from your head.

Step 3 runs those asserts and comes out green, obviously —we captured the values from the same legacy—. But the green here means something precise worth repeating: it doesn't say 110.19 is the correct price; it says the legacy still returns what it returned when we took the photo. It's the guarantee of no-change, not of correctness.

And the final block is the module's most delicate lesson in seed form. Look at D-4: clean math says 106.89, the legacy charges 106.88, and the characterization test pins 106.88 —the quirk included—. The test protects the "error" on purpose. Why would you protect an error? Because your job today isn't to correct it, it's to pin it, and because —as you'll see in lesson 6— it turns out someone depends on that cent. By pinning 106.88, the test does something very valuable: if tomorrow someone "cleans up" the calculation so it gives 106.89, the test turns red and forces them to realize they're changing the quirk. It doesn't forbid it; it makes it visible and conscious. That's the whole point: the characterization test doesn't prevent fixing bugs; it prevents fixing them by accident.

Deep dive: the characterization test as a 'learning test'

There's a practical way to write these tests that Feathers describes and that's worth having on hand, because it solves the problem of "I don't know what value to put." The trick is to let the test tell you the value. You write the assertion with a value you know is false —for example, zero— you run the test, and you read in the failure message the real value the legacy produced. Then you copy that real value into the assertion. The test just "learned" the current behavior, and from there it stays green and protects that behavior. In pseudocode of the flow:

1. assert legacy_catalog_price(item_A1) == 0        # deliberately false value
2. you run -> FAILS: "expected 0, got 110.19"       # the legacy dictates the value
3. assert legacy_catalog_price(item_A1) == 110.19   # you copy the REAL value
4. you run -> GREEN                                 # the test learned the behavior

Notice the inversion relative to a normal test. In a test you write for your code, you know the expected answer in advance (you designed it) and the code has to reach it. In a characterization test, the code knows the answer and you learn it from it. That's why they're sometimes called "learning tests": they don't impose an expectation, they extract it. And that's why there's no risk of "guessing wrong": the value never comes from your head, it always comes from running the legacy.

This connects with an important caveat. The value you capture is only as good as the inputs you chose. If you only characterize the happy path —a simple item, no coupon, no gold, quantity 1— your net will be blind right in the weird cases, which are the ones that break most (remember that D-4, the gold, was the only one the "clean" reimplementation broke). A good characterization test covers the happy path and the edge cases: quantities that cross the volume threshold (9, 10, 11), coupons at the limit, gold customers, categories with a 0 discount, prices that round ugly. Lesson 5 takes this to its conclusion: when the edge cases are too many to choose by hand, you sample the input space and record a golden master. But the principle is the same from now on: transcribe many cases, not just the pretty one.

One last precision about what gets pinned. Here we characterize the return value of a pure function, which is the cleanest case. But the observable behavior of a legacy sometimes includes more than the return: what it writes to the database, what email it triggers, what record it leaves in the log. Characterizing those effects is more laborious (you have to capture them), but the idea doesn't change: you pin what the system does observably, not what you think it should do. In Mercado's catalog the price is a pure return, so we focus on it; keep in mind the technique extends to effects when needed.

Common mistakes

Guessing the expected value instead of capturing it. What happens: when writing the test, you calculate mentally (or with a "clean" formula) what the result should be and put it as expected, instead of running the legacy and copying what it returns. Why it happens: it's the correct instinct for the code you design, and it's hard to turn off. How to spot it: if your characterization test comes out red the first time you run it against the legacy without having changed anything, you almost certainly guessed the value instead of capturing it —the legacy can't "fail" against its own output—. How to fix it: use the "learning test" flow: put a false value, run, read the real value from the failure message, copy it. The expected value has to come from the legacy, never from your head. Like the stenographer: you transcribe what was said, not what should have been said.

"Fixing" the legacy so it passes your test. What happens: you wrote the test with a guessed value, it comes out red, and instead of fixing the test you fix the code so it produces your value. Why it happens: "red test → fix the code" is such a strong reflex that it fires even when the "fix" changes production behavior. How to spot it: if your first commit on the legacy changes what it returns for existing cases, and you justify it with "the test wasn't passing," you inverted the relationship —the test was supposed to describe the legacy, not reform it—. How to fix it: in the characterization phase, the legacy is the truth and the test adapts to it, never the other way around. If there really is a bug you want to fix, that's a separate and later decision (lesson 6), made after having the net and consciously, not smuggled in under the excuse of "making the test pass."

Characterizing only the happy path. What happens: you pin a couple of simple, pretty cases —a normal product, no weird discounts— and consider the net done. Why it happens: the simple cases are the first that come to mind and the fastest to write, and they give a false sense of coverage. How to spot it: if your suite doesn't include gold customers, or quantities at the volume threshold, or coupons, or categories with a 0 discount, you're blind right where the legacy is weirdest. How to fix it: the legacy hides its dangerous behavior at the edges, not in the center —remember that the only regression the "clean" reimplementation produced was in the gold customer, a case that a happy path without gold would never have detected—. Deliberately cover the edges: values at the thresholds, discount combinations, special segments. And when the edges are too many to enumerate, sample (lesson 5). A net with holes at the edges is almost as dangerous as no net, because it gives you confidence where you shouldn't have it.

Exercises

Exercise 1 — The value you capture. You have this Mercado legacy function and you want to characterize it for the item {"unit_price": 10.00, "quantity": 3, "category": "books"}. The books category gives a 10% discount; the tax is 16%; the rounding is down to the cent at each step. Without running anything, describe the correct procedure to obtain the expected value of the characterization test (we're not asking for the exact number, but the method). Then explain why the "I calculate 10×3×0.9×1.16 and put that number" approach is wrong.

See solution

The correct procedure is to capture, not calculate: you run the real legacy function with that exact item, read the value it returns, and that value —the one that came out of the code— is the one you put as expected in the test. If you want, use the "learning test" flow: you write assert price == 0, run it, and the failure message tells you the real value, which you copy into the assertion. The point is that the expected number comes from executing the legacy, not from a formula you wrote.

The "I calculate 10×3×0.9×1.16 = 31.32 and put that number" approach is wrong for two reasons. First, it reproduces your mental model of the calculation, not the legacy's. Your formula rounds only once at the end (implicit in giving 31.32); the legacy rounds down at every step, so its real result may differ by cents. If you put 31.32 and the legacy returns 31.31, your test comes out red against a legacy that didn't change —false alarm— and it would push you to "fix" the legacy so it gives your number. Second, and more fundamentally: the characterization test doesn't verify that your formula is correct; it pins what the legacy does. Even if your formula were "more correct" accounting-wise, the test must capture the real behavior to be able to detect changes relative to it. Correctness is a later and separate question.

Exercise 2 — Green isn't correct. A catalog characterization test pins that a certain gold item costs 106.88 and comes out green. A developer argues: "the test passes, so applying the loyalty after the tax is the correct way to calculate the price." Explain the reasoning error, drawing on the stenographer analogy, and say what would have to happen to be able to assert that 106.88 is the correct price.

See solution

The error is confusing "the test transcribes the behavior faithfully" with "the behavior is correct." The characterization test that pins 106.88 does exactly what the court stenographer does: it recorded that the legacy says 106.88, with the same neutrality with which the stenographer writes "seen" when the witness says "seen." The green means "the legacy still says what it said when we transcribed it" —not "what it says is true"—. A stenographer who transcribes a lie without errors doesn't turn it into truth; they only leave a faithful record that it was said. Likewise, a green characterization test over a bug doesn't turn the bug into correct; it only certifies that the bug is still there, identical.

To assert that 106.88 is the correct price you'd need something the test doesn't contain: the business specification of how a gold customer's price must be calculated —does the loyalty go before or after the tax?—, and confirmation that 106.88 corresponds to that desired rule. That's a product/business decision (and, as lesson 6 shows, it can have consequences: a partner already depends on the 106.88). The characterization test is neutral about that question by design: its job is to detect changes, not to arbitrate correctness.

Exercise 3 — Design the input battery. You're going to characterize the catalog price and you only have time to choose 6 items by hand. Based on the rule "cover the edges, not just the happy path," propose 6 items that together exercise the function's dangerous behavior (category, volume with a threshold at 10, coupon, gold loyalty, rounding). For each one, say in one sentence what edge or combination you're covering, and explain why a battery of "6 normal products without discounts" would be a net with holes.

See solution

A reasonable battery (the exact values can vary; what matters is which edge each one covers):

  1. {unit_price: 100, quantity: 1, category: "electronics"} — simple base path, a single discount (category), no combinations. The reference.
  2. {unit_price: 100, quantity: 9, category: "electronics"} — quantity just below the volume threshold (10): verifies that the volume discount is not applied.
  3. {unit_price: 100, quantity: 10, category: "electronics"} — quantity right at the threshold: verifies that the volume discount is applied. Cases 2 and 3 together pin the threshold edge, a classic place for bugs.
  4. {unit_price: 100, quantity: 1, category: "electronics", loyalty: "gold"} — gold customer: exercises the quirk (loyalty after the tax), the case that broke the "clean" reimplementation.
  5. {unit_price: 49, quantity: 2, category: "books", coupon_percent: 0.15} — combines category and coupon: verifies the order of application of two discounts and the accumulated rounding.
  6. {unit_price: 7.50, quantity: 12, category: "toys", coupon_percent: 0.10, loyalty: "gold"} — the "all together": volume + coupon + gold + a price that rounds ugly. The richest case, where the step-by-step rounding shows most.

A battery of "6 normal products without discounts" would be a net with holes because it would only exercise the calm center of the function —subtotal, category, tax— and leave unpinned exactly the branches where the legacy is weird and fragile: the volume threshold, the discount combination, the gold quirk, the rounding in hard cases. Since those branches wouldn't be in the photo, a change that broke them would pass the characterization test green —the net would give a green light to a regression—. And those edges aren't exotic museum cases: the gold customer and the coupon are real Mercado customers every day. The net has to cover where the legacy does weird things, which is exactly where the happy path isn't.

Summary and next step

In this lesson you learned to write the net, and the central gesture that makes it work: capture the legacy's current output, don't guess the ideal one. You saw, with the stenographer, that your job when characterizing is to transcribe with total fidelity what the legacy does —quirks and "errors" included—, not to produce the correct result; correctness is a later question and someone else's. And you measured it: step 1 showed that "clean" math guesses values that don't match the real ones (110.2 vs 110.19), so a test made with guessed values would come out red against a healthy legacy and would push you to alter production; step 2 captured the real values; and step 3 pinned them, including the gold quirk (106.88, not 106.89), so that any future "fix" has to be conscious.

Before moving on you should be able to: explain why the expected value is captured and not calculated; use the "learning test" flow to extract the value from the legacy itself; distinguish "green" (didn't change) from "correct" (meets the specification); and design an input battery that covers the edges and not just the happy path.

Lesson 4 attacks the obstacle that in practice stops characterization most often: what do you do when you can't pin the behavior because a dependency makes it unpredictable? The catalog price consults the real clock to know whether a coupon expired, so its output changes depending on the day you run the test —impossible to pin deterministically—. You'll learn to find the seam: the point where you divert that dependency (the clock) without rewriting the function, so the test can govern it.

Resources

  • Michael Feathers, Working Effectively with Legacy Code (Prentice Hall, 2004), ch. 13 "I Need to Make a Change, but I Don't Know What Tests to Write" — the chapter that defines the characterization test and the "learning test" technique (put a false value and let the code dictate the real one). The direct source of this lesson. In English.
  • "Characterization test" — Michael Feathers's term (Working Effectively with Legacy Code); summary on Wikipedia: en.wikipedia.org/wiki/Characterization_test. The distinction, in one page, between pinning behavior and verifying correctness —the misunderstanding this lesson combats—. In English.
  • Emily Bache, "The Gilded Rose Kata" — github.com/emilybache/GildedRose-Refactoring-Kata. The classic exercise for practicing characterization tests over a tangled legacy function before refactoring it; the practical counterpart of Mercado's catalog. In English.
  • Nicolas Carlo, "How to add a test to legacy code (characterization tests)" — understandlegacycode.com. A step-by-step, modern guide with examples of the same capture flow we use here. In English.
← PreviousWorking with code that scares you and has no testsNext →Finding the seam