Module 7: Measuring Migration Progress

The migration fitness function

Overview

The burn-down of lesson 3 measures one thing: how many calls to the legacy remain, dropping toward zero. But it has a blind spot. It measures the legacy shrinking at the front —the requests you stop sending it—, and it doesn't see whether the legacy is growing at the back —new code someone builds on top of the monolith you're supposedly killing—. You can have a burn-down that drops beautifully while, in parallel, a team adds a new feature straight onto the legacy, reintroducing dependencies that took work to cut. The burn-down wouldn't notice; it would drop the same. You need a second instrument, one that watches specifically that the legacy doesn't grow back. That instrument is the migration fitness function.

A fitness function, as you learned in architecture-decisions, is an automated test that guards a property of the architecture and breaks when that property degrades. Here we apply it to a very concrete migration objective: the number of the code's references to the legacy module must never grow. The fitness function counts those references and compares them against a baseline (the count from the last time it was healthy). If the count dropped or held, PASS: the legacy shrank or at least didn't grow. If the count rose, FAIL: someone added new code that calls the legacy, and the test breaks the CI so that change doesn't get in until it's fixed. It's an automatic alarm against the worst habit during a migration: keep feeding the monster you want to kill.

Connection with the module. This lesson builds the dashboard's second instrument, complementary to the burn-down: the burn-down measures the advance (how much is left), the fitness function protects that advance (that it doesn't roll back). Lesson 5 turns it into a ratchet —a baseline that only drops—, making the progress irreversible, and connects with the discipline of not adding features to the legacy. Lesson 6 will use "references to the legacy at zero" as one of the done conditions. Notice the boundary: the fitness function as a concept —what it is, its types, how it integrates in an evolutionary architecture— belongs to architecture-decisions (M6 of that guide) and to Ford & Parsons's book; here we do not re-explain it, we apply it to a migration end. The new thing isn't the idea of a fitness function; it's using one so the legacy doesn't grow while you strangle it.

An analogy: the elevator's weight limit that refuses to move

Imagine an elevator with a weight limit —say 600 kilos—. When too many people get in and the weight passes the limit, the elevator doesn't move: an alarm sounds, a light turns on, and the doors stay open until someone gets off. The elevator doesn't trust the passengers' good judgment ("surely we fit"); it has a sensor that measures the weight and a hard rule that blocks the movement if it's exceeded. Nobody argues with the elevator: either the weight drops, or the elevator doesn't start.

That's exactly the role of a fitness function in a migration. The "weight" is the number of references to the legacy. The "limit" is the baseline —the maximum allowed, set at the last healthy state—. And the "elevator that doesn't move" is the CI turning red: if a change makes the references to the legacy pass the baseline, the test fails and the PR doesn't get in —the change can't get on the elevator—. It doesn't depend on someone remembering the rule "don't add code to the legacy" or reviewing it by hand on every PR; the sensor measures and the rule blocks, automatically, every time.

And like the elevator, the fitness function doesn't judge why the weight rose —whether it was carelessness, haste, or because it was really needed—; it only observes that it was exceeded and blocks. That's its virtue: it's an objective limit, not an opinion. The conversation stops being "is it OK to add this dependency to the legacy?" (which can be rationalized) and becomes "the CI is red, we have to lower the references for it to get in" (which can't be rationalized). The elevator doesn't move until the weight drops.

Worked example: the fitness function that counts and fails

We're going to write the fitness function and run it against two versions of the catalog's code. We model the "code" as a dictionary of files, each one with the lines that import from the legacy monolith (mercado.monolith). The count_legacy_refs function counts how many of those lines there are in total, and migration_fitness fails with an assert if that count exceeds the baseline. We run two scenarios: a healthy sprint where modules were migrated out of the legacy (references dropping → PASS), and a sprint where someone added a new feature on the legacy (references up → FAIL, with the literal AssertionError).

# The migration fitness function: counts the catalog's references to the legacy
# monolith and FAILS (assert) if they grew relative to the baseline. Catches the
# new code someone added to the module we're killing.

# The catalog's "code": each file with its lines that import from the monolith.
codebase_baseline = {
    "catalog/pricing.py":   ["from mercado.monolith import tax_table",
                             "from mercado.monolith import discount_rules"],
    "catalog/inventory.py": ["from mercado.monolith import stock_ledger"],
    "catalog/search.py":    ["from mercado.monolith import legacy_index"],
}

def count_legacy_refs(codebase):
    return sum(1 for lines in codebase.values()
               for line in lines if "mercado.monolith" in line)

def migration_fitness(codebase, baseline):
    """Fails if the references to the legacy GREW relative to the baseline."""
    refs = count_legacy_refs(codebase)
    assert refs <= baseline, (
        f"migration_fitness FAILED: legacy_refs={refs} > baseline={baseline} "
        f"(new code was added to the legacy)")
    return refs

baseline = count_legacy_refs(codebase_baseline)
print(f"Migration fitness function (baseline = {baseline} references to the legacy)\n")

# --- Scenario 1: a HEALTHY sprint. inventory and search were migrated out of the
#     monolith; only pricing's 2 references remain. ---
codebase_good = {
    "catalog/pricing.py":   ["from mercado.monolith import tax_table",
                             "from mercado.monolith import discount_rules"],
    "catalog/inventory.py": [],   # migrated: no longer calls the monolith
    "catalog/search.py":    [],   # migrated: no longer calls the monolith
}
print("Scenario 1 - healthy sprint (inventory and search were migrated):")
refs = migration_fitness(codebase_good, baseline)
print(f"  legacy_refs={refs}  baseline={baseline}  ->  migration_fitness PASS\n")

# --- Scenario 2: someone added a NEW feature on the legacy: a promo.py file
#     that imports from the monolith again. The references RISE. ---
codebase_bad = dict(codebase_baseline)
codebase_bad["catalog/promo.py"] = ["from mercado.monolith import banner_config"]
print("Scenario 2 - someone added catalog/promo.py that imports from the monolith:")
try:
    migration_fitness(codebase_bad, baseline)
    print("  (shouldn't reach here)")
except AssertionError as e:
    print(f"  AssertionError: {e}")
    print("  -> the CI goes RED: the PR doesn't merge until the new reference is removed.")

What to expect. When you run the file, the output is exactly this:

Migration fitness function (baseline = 4 references to the legacy)

Scenario 1 - healthy sprint (inventory and search were migrated):
  legacy_refs=2  baseline=4  ->  migration_fitness PASS

Scenario 2 - someone added catalog/promo.py that imports from the monolith:
  AssertionError: migration_fitness FAILED: legacy_refs=5 > baseline=4 (new code was added to the legacy)
  -> the CI goes RED: the PR doesn't merge until the new reference is removed.

Read the two scenarios, because they're the only two things the fitness function distinguishes: the legacy shrinks (or stays the same) or the legacy grows.

The baseline is 4: the catalog's code, in its last healthy state, had four references to the legacy monolith —two in pricing.py, one in inventory.py, one in search.py—. That number is the line that mustn't be crossed upward.

In scenario 1, a sprint did its work: it migrated inventory.py and search.py out of the monolith (their import lists became empty), and only pricing.py's two references survived. count_legacy_refs returns 2. Since 2 <= 4, the assert passes: migration_fitness PASS. The legacy shrank from 4 to 2 references —exactly what should happen during a migration—. This is what the fitness function wants to see, and it makes no noise: it passes silently.

In scenario 2, someone added catalog/promo.py, a new file that imports banner_config from the monolith —a new feature built on top of the legacy—. Now the code has the 4 original references plus 1 new one: count_legacy_refs returns 5. Since 5 > 4, the assert fails, and the output shows the literal AssertionError: "migration_fitness FAILED: legacy_refs=5 > baseline=4 (new code was added to the legacy)". In the CI, this turns the build red, and the promo.py PR can't be merged until that legacy dependency is removed (building the feature on the modern instead). The elevator doesn't move: the weight passed the limit.

Notice the asymmetry, which is the whole point: the fitness function doesn't require the references to drop each sprint (scenario 1 would have passed the same if pricing.py hadn't changed and they were still 4); it only requires that they don't rise. It's a ceiling, not a floor. You can have a sprint where nobody migrated anything (references stay at 4, PASS) —that's fine, the burn-down will take care of measuring whether you advance—; what the fitness function doesn't tolerate is someone adding to the legacy (references rise to 5, FAIL). Its job is a single thing: that the legacy doesn't grow. Lesson 5 will make it stricter by turning the baseline into a ratchet that drops.

Deep dive: what to count, and why to automate

The example's fitness function counts import lines that mention mercado.monolith. It's an approximation —direct and sufficient for the module—, but it's worth understanding what's really being measured and how it would be done in a real system.

What you want to count is any way the new code depends on the legacy: imports of the legacy module, calls to its functions, references to its tables, endpoints that route back to the monolith. In a real repository, this is implemented in several ways depending on the language: a grep of the legacy package's imports, an analysis of the dependency graph (tools that tell you which module imports which), an architecture rule in a linter (ArchUnit in Java, import-linter in Python, dependency-cruiser in JavaScript) that declares "the catalog package must not import the monolith package", or a test that queries that graph and fails if a forbidden edge appears. They're all the same idea: count the dependencies toward the legacy and fail if they grow.

        new code (catalog)                  legacy (monolith)
        ┌────────────────────┐              ┌──────────────────┐
        │ pricing.py  ───────┼──ref────────>│ tax_table        │
        │ pricing.py  ───────┼──ref────────>│ discount_rules   │   baseline = 4
        │ inventory.py       │  (migrated)  │ stock_ledger     │
        │ search.py          │  (migrated)  │ legacy_index     │
        │ promo.py    ───────┼──ref──✗─────>│ banner_config    │ <- FAIL: rises to 5
        └────────────────────┘              └──────────────────┘
                                  the fitness function counts the arrows
                                  and breaks the CI if a new one appears

The key question is why automate this instead of trusting code review. The answer is that "don't add code to the legacy" is a rule that everyone approves and nobody remembers under pressure. In a PR review, when someone is in a hurry to ship a feature and the fastest way is to hang it off the monolith ("it's just an import, the function is already there"), the rule gets rationalized: "it's temporary", "we'll migrate it later", "it's a special case". And each of those exceptions makes the legacy grow back that cost so much to reduce. A fitness function doesn't get tired, isn't in a hurry, and doesn't accept rationalizations: it counts and blocks. It turns a good intention (that people forget) into a system constraint (that people can't skip). It's the difference between "we should" and "you can't".

There's an honest nuance about false positives and negatives. Counting import lines is a heuristic: there can be references to the legacy that aren't imports (an HTTP call to the monolith, a query to a shared table) that this count doesn't see —false negatives—; and there can be imports of the legacy package that are legitimate and transitory (the anti-corruption layer has to talk to the legacy by design) that the count flags —false positives—. In practice it's refined: the ACL is excluded from the count (it's the authorized boundary), and the import count is complemented with other signals (calls, queries). What matters is the idea: an objective measure of "how much the new depends on the old", watched automatically, that can't grow. The precision of the measure is adjusted; its role as guardian isn't.

Common mistakes

Trusting code review to keep the legacy from growing. What happens: the team agrees "we don't add code to the legacy during the migration" but leaves the enforcement to the manual review of each PR. Why it happens: writing a fitness function is extra work, and "we'll review it in the PR" seems enough. How to spot it: sprint after sprint, new imports to the legacy module appear that passed the review "because they were urgent" or "temporary"; the count of references to the legacy rises instead of dropping. How to fix it: automate the rule with a fitness function that breaks the CI. The manual review fails exactly when it's most needed —under pressure, in a hurry, when the exception feels justified—; an automatic test doesn't give in to pressure. The cost of writing the fitness function (a few lines that count imports) is minuscule against the cost of letting the legacy grow back one dependency at a time.

Putting in the fitness function but not connecting it to the CI. What happens: the team writes the fitness function and runs it by hand every once in a while, or leaves it as a script nobody executes. Why it happens: integrating it into the pipeline is one more step, and "it's already written" feels like enough. How to spot it: the fitness function exists in the repository but the builds never fail because of it; the references to the legacy grow without anything turning red. How to fix it: a fitness function only works if it blocks. It has to run on every PR and fail the build when the count rises, just as the unit tests fail. A fitness function that isn't connected to the CI is like the elevator's weight sensor disconnected from the brakes: it measures, but the elevator moves anyway. Its value is in the automatic blocking, not in the existence of the code.

Counting wrong: including the anti-corruption layer or ignoring the dependencies that aren't imports. What happens: the fitness function flags FAIL because of the ACL (which must talk to the legacy) or, the other way around, gives PASS while the code calls the monolith over HTTP without importing it. Why it happens: counting imports is the simplest heuristic, but the ACL is a legitimate exception and not all dependencies are imports. How to spot it: the fitness function fails because of code that's correct (the ACL), or passes while the burn-down doesn't drop (there are real dependencies it doesn't count). How to fix it: exclude the ACL from the count (it's the authorized boundary between old and new, by design it's going to reference the legacy until the end) and complement the import count with other dependency signals (HTTP calls to the monolith, queries to shared tables). The measure is a heuristic that's refined; what doesn't change is its role: count the new's dependency on the old and fail if it grows. A badly calibrated fitness function either makes noise (false FAILs the team learns to ignore) or gives false comfort (PASS while the legacy grows on another side).

Exercises

Exercise 1 — PASS or FAIL. With the baseline at 4, say whether each state of the code gives PASS or FAIL and why: (a) 4 references (nobody migrated or added anything this sprint); (b) 2 references (two modules were migrated); (c) 6 references (two features were added on the legacy); (d) 0 references (everything was migrated).

See solution

(a) 4 references → PASS. 4 <= 4: the count didn't rise relative to the baseline. The fitness function is a ceiling, not a floor: it doesn't require you to drop each sprint, only not to grow. A sprint without migrating anything passes (though the burn-down won't advance —that's another metric—).
(b) 2 references → PASS. 2 <= 4: the legacy shrank. It's the ideal case, and it passes silently.
(c) 6 references → FAIL. 6 > 4: someone added two new dependencies to the legacy. The test breaks the CI; those changes don't get in until the new references are removed.
(d) 0 references → PASS. 0 <= 4: the code no longer depends on the legacy at all. This is the state that, combined with legacy_calls == 0, allows declaring done (lesson 6).

The rule is always the same: PASS if the count is less than or equal to the baseline (the legacy didn't grow), FAIL if it's greater (the legacy grew).

Exercise 2 — Why automate. The text says that "don't add code to the legacy" is a rule that "everyone approves and nobody remembers under pressure". (a) Describe a realistic situation where a developer in a hurry would add an import to the legacy with good justification. (b) Why doesn't code review always catch it? (c) What does the fitness function do differently than the review?

See solution

(a) A typical situation: a promotions feature has to ship for Black Friday in two days. The function that calculates the discounts already exists... in the legacy monolith (discount_rules). Building the new version in the modern service would take a week; importing the legacy's takes five minutes. The developer, with the deadline looming, imports discount_rules from the monolith "for now, we'll migrate it after the campaign". The justification is real: the feature is needed, there's not enough time, the function already exists.

(b) Because code review is done by a person, under the same pressures: the reviewer also knows the campaign is in two days, also sees that the function already exists in the legacy, and the justification "it's temporary, we'll migrate it later" sounds reasonable in the moment. The rule gets rationalized case by case, and each individual exception seems defensible —the problem is that the sum of defensible exceptions makes the legacy grow—. The manual review is weaker exactly when it's most needed: under pressure.

(c) The fitness function doesn't reason or give in: it counts the references, sees they rose from 4 to 5, and turns the CI red, regardless of the justification. It turns the conversation from "is this exception OK?" (which can be won with a good argument) into "the CI is red, we have to lower the references to merge" (which can't be won with arguments, only by removing the dependency). It doesn't prevent the feature from being made —it prevents it from being made on the legacy—, forcing it to be built on the modern or to migrate discount_rules first. It's a system constraint, not an intention that's remembered.

Exercise 3 — Design the measurement. You're the one implementing the fitness function in a real Python repository where the catalog is migrating out of mercado.monolith. (a) What would you count and how? (b) What would you exclude from the count and why? (c) Where would you connect it so it really blocks?

See solution

(a) I'd count the dependencies of the catalog package toward the mercado.monolith package: the imports (from mercado.monolith import ... or import mercado.monolith), detected with a grep/static analysis or with an architecture tool like import-linter declaring the rule "catalog must not import monolith". I'd complement with the non-import dependencies if there were any: HTTP calls to the monolith's endpoint and queries to the still-shared tables. The total number of dependencies is what's compared against the baseline.

(b) I'd exclude the anti-corruption layer from the count. The ACL is the authorized boundary between the old model and the new (M5 lesson); by design it has to reference the legacy until the migration ends, so counting it would give false FAILs. I isolate it in its own module (for example catalog/acl/) and exclude it from the rule, so the fitness function watches only the unauthorized dependencies —the ones that shouldn't exist—.

(c) I'd connect it to the CI pipeline, as a step that runs on every PR along with the unit tests, and that fails the build if the count exceeds the baseline. The baseline is stored in the repository (a versioned file with the current allowed number), so that lowering it is an explicit commit (lesson 5, the ratchet). Without the connection to the CI that blocks the merge, the fitness function only measures; connected, it really prevents the legacy from growing —the weight sensor connected to the elevator's brakes—.

Summary and next step

In this lesson you built the dashboard's second instrument: the migration fitness function, which covers the burn-down's blind spot —watching that the legacy doesn't grow at the back while you reduce it at the front—. You saw, with the elevator that refuses to move when the weight passes the limit, that the fitness function is an objective and automatic limit: it counts the references to the legacy, compares them against a baseline, and blocks (breaks the CI) if they grew. And you executed it: PASS when the code shrank from 4 to 2 references (modules were migrated), and FAIL —with the literal AssertionError— when someone added promo.py importing from the monolith and the references rose to 5. You learned that it's a ceiling, not a floor (it doesn't require dropping each sprint, only not rising), why automating it beats manual review (it doesn't give in under pressure), and how to calibrate what to count (exclude the ACL, complement the imports with other dependencies).

Before moving on you should be able to: write a fitness function that counts references to the legacy and fails if they exceed a baseline; explain why it's a ceiling and not a floor; argue why automating it in the CI beats trusting code review; and calibrate what to count and what to exclude.

Lesson 5 makes the fitness function more powerful with a simple and profound idea: turning the baseline into a ratchet —a number that can only drop, never rise—. Each sprint that reduces the references to the legacy tightens the baseline to the new minimum, fixing the progress so it can't roll back. With that, the migration becomes irreversible: each advance is locked in, and the legacy can't grow back even one import above its best mark. You're going to see the ratchet tighten the baseline sprint by sprint (4→3→2→1→0) and block an attempt to add code to the legacy when the baseline already reached zero —and you'll see how this sustains the discipline of not building new features on the monolith you're killing—.

Resources

Neal Ford, Rebecca Parsons, and Patrick Kua, Building Evolutionary Architectures (O'Reilly, 2nd ed., 2022) — the book that defines the fitness functions as tests that guard architectural properties and integrate into the pipeline. The source of the concept we apply here to the migration. In English.
Martin Fowler, "StranglerFigApplication" (2004) — martinfowler.com/bliki/StranglerFigApplication.html. Fowler underlines that the legacy must shrink until it disappears; a fitness function that keeps it from growing is what protects that shrinking from rolling back. In English.
Sam Newman, Monolith to Microservices (O'Reilly, 2019), ch. 3 — on the discipline of not continuing to add functionality to the monolith being decomposed; this lesson's fitness function is how that discipline is enforced automatically. In English.
Martin Fowler, martinfowler.com — the bliki with the entries on evolutionary architecture and fitness functions that frame the general concept of which we take a concrete instance here. In English.

← PreviousThe legacy-call burn-down Next →Keeping the legacy from growing back