Module 7: Measuring Migration Progress
Avoiding the eternal migration
Overview
This lesson covers the worst possible outcome of a modernization project, and it has a name: the eternal migration. It's the state in which a migration advances well for months, reaches 90% or 98%, and there it stays forever —the legacy is never turned off, the team maintains two systems indefinitely, and the migration's prize is never reaped—. It's worse than not having started, because not starting leaves you with one system to maintain; the eternal migration leaves you with two, forever, plus the complexity of having them coexisting. And the most painful thing is that it almost always arrives after the hard part is already done: the team built all the infrastructure, moved 98% of the traffic, and then didn't finish the last 2% —nailing the project at the exact point where it already paid all the costs and is one push away from reaping all the benefits—.
The eternal migration is almost never a technical failure. The burn-down dropped, the fitness function worked, the data was migrated. It's a failure of termination: there was no done criterion that forced closing the last stretch, and without that obligation, the last stretch —the most boring, the one with the rare cases, the one that no longer has urgency because "98% is already on the new"— never rises in the list of priorities. There's always something more urgent than turning off a legacy that "is barely used anyway". This lesson shows, executed, how two migrations identical in their technical part end in opposite ways depending on whether or not they have a done criterion that forces them to close —and how much the one that doesn't finish costs, in numbers—.
Connection with the module. This lesson is the consequence of not having what we built in the previous lessons: without the definition of done (lesson 6), without the burn-down that detects the stall (lesson 3), the migration becomes eternal. It's also the underlying argument of the whole module —the reason why measuring the progress matters—: you measure in order to finish, and not finishing is the disaster the measurement exists to avoid. Lesson 8 will integrate everything into a dashboard that does take the migration to its close. Notice the boundary: M3 (lesson 7) already named the eternal migration for an individual slice (leaving the legacy on with zero traffic). Here we treat it at program scale —a complete migration that gets stuck in the last stretch— and add what avoids it: the done criterion and the forcing function that force the close.
An analogy: the remodel that's "almost" and the three years among rubble
You know someone —or you are that someone— who remodeled their house and "is almost done". The kitchen turned out, the rooms turned out, they painted almost everything. But missing: a bathroom without tiles, a room that ended up as storage with the leftover materials, a door they never hung, the patio with the work's rubble piled in a corner. And it's been like that for three years. It's not that they can't finish —very little is missing, a weekend of work—; it's that nobody has the urgency anymore, the contractor left for another job, and "anyway, you can already live in the house". They live among rubble, with the work at 95%, indefinitely, because the last 5% never rises in life's list of priorities.
And that half-done remodel costs, even if nobody keeps the tally. It costs the room they can't use (they lost a bedroom). It costs stumbling over the materials every day. It costs the discomfort of an unfinished bathroom. And if they rented an apartment "while the work is finished", it costs two rents at once —the temporary apartment's and the house that's almost—, a double expense that was supposed to be for a few months and has lasted three years. The cost of "almost" isn't zero; it's a constant drip that accumulates, month after month, because the work never crosses the finish line.
The eternal migration is that remodel. The 98% of traffic on the new is the house where "you can already live". The legacy on is the patio with the rubble and the storage room —the work's remnants nobody removes—. And the two rents are the cost of maintaining two systems: you pay for the modern's operation and the legacy's, the complexity of both, the team divided between both, a double expense that was supposed to be temporary and became permanent. The difference between the finished house and the one of three years among rubble isn't the contractor's skill —they both know how to lay tiles—; it's that someone set a delivery date and enforced it. Without a date, "almost" is forever.
Worked example: two migrations, one finishes and the other doesn't
We're going to execute two migrations side by side, with the same speed the first six sprints, and see how they end in opposite ways. Migration A has no done criterion: it drops well to 98.5% and gets stuck there (it stays at 15 calls to the legacy forever). Migration B has a forcing function (a done deadline): it keeps pushing the last stretch until reaching zero and deleting the legacy. In each sprint we count the accumulated cost of maintaining the two systems: while the legacy is alive, you pay; when it's deleted, the cost stops.
# The eternal migration vs the one that finishes. Two migrations, 12 sprints.
# A (no done criterion) gets stuck at 98.5% forever.
# B (with a forcing function) reaches 0 and DELETES the legacy.
# The cost of maintaining two systems accumulates while the legacy stays alive.
COST_PER_SPRINT = 10 # what it costs to have both systems on for a sprint
# legacy_calls per sprint (sprints 1..12)
migration_A = [1000, 700, 400, 150, 40, 15, 15, 15, 15, 15, 15, 15] # plateau
migration_B = [1000, 700, 400, 150, 40, 15, 0, 0, 0, 0, 0, 0] # finishes
def run(name, calls):
print(f"{name}")
print(f" {'sprint':>7}{'legacy_calls':>14}{'legacy alive?':>14}{'cost accum.':>13}")
print(" " + "-" * 48)
cost = 0
deleted_at = None
for sprint, legacy_calls in enumerate(calls, start=1):
alive = legacy_calls > 0 or deleted_at is None
# the legacy is deleted the first sprint it reaches 0 (and there the cost stops)
if legacy_calls == 0 and deleted_at is None:
deleted_at = sprint
if deleted_at is not None and sprint > deleted_at:
alive = False
if alive:
cost += COST_PER_SPRINT
status = "yes" if alive else "DELETED"
print(f" {sprint:>7}{legacy_calls:>14}{status:>14}{cost:>13}")
print(" " + "-" * 48)
if deleted_at:
print(f" -> legacy DELETED in sprint {deleted_at}. Total cost: {cost}. "
f"Stops paying.\n")
else:
final = calls[-1]
pct = (1000 - final) / 1000 * 100
print(f" -> never reaches 0 (stuck at {pct:.1f}%). Cost at 12 sprints: "
f"{cost}, and KEEPS paying forever.\n")
print("Eternal migration vs migration that finishes\n")
run("A - no done criterion (stuck in the last 1.5%):", migration_A)
run("B - with a forcing function (done deadline):", migration_B)
print(" Same speed the first 6 sprints. The difference isn't technical:")
print(" it's having (or not) a done criterion that forces closing the last stretch.")What to expect. When you run the file, the output is exactly this:
Eternal migration vs migration that finishes
A - no done criterion (stuck in the last 1.5%):
sprint legacy_calls legacy alive? cost accum.
------------------------------------------------
1 1000 yes 10
2 700 yes 20
3 400 yes 30
4 150 yes 40
5 40 yes 50
6 15 yes 60
7 15 yes 70
8 15 yes 80
9 15 yes 90
10 15 yes 100
11 15 yes 110
12 15 yes 120
------------------------------------------------
-> never reaches 0 (stuck at 98.5%). Cost at 12 sprints: 120, and KEEPS paying forever.
B - with a forcing function (done deadline):
sprint legacy_calls legacy alive? cost accum.
------------------------------------------------
1 1000 yes 10
2 700 yes 20
3 400 yes 30
4 150 yes 40
5 40 yes 50
6 15 yes 60
7 0 yes 70
8 0 DELETED 70
9 0 DELETED 70
10 0 DELETED 70
11 0 DELETED 70
12 0 DELETED 70
------------------------------------------------
-> legacy DELETED in sprint 7. Total cost: 70. Stops paying.
Same speed the first 6 sprints. The difference isn't technical:
it's having (or not) a done criterion that forces closing the last stretch.
Look at the two migrations in parallel. The first six sprints are identical: both drop 1000, 700, 400, 150, 40, 15 —the same speed, the same technical part, the same capable team—. Up to sprint 6, there's no way to tell them apart: both are going well, both reached 98.5%, both did the hard work. If you judged them at sprint 6, you'd say both are a success.
The difference appears at sprint 7, and it isn't technical. Migration A gets stuck: it stays at 15 calls to the legacy, sprint after sprint, from 7 to 12 and beyond. That 15 is the last 1.5% —the rare cases, the flows nobody documented, the forgotten nightly process—, and without a done criterion that forces closing it, it never rises in the list of priorities. There's always something more urgent than turning off a legacy that "is barely used". The legacy alive? column says "yes" in every row, and the cost accum. keeps rising: 70, 80, 90, 100, 110, 120 at sprint 12 —and the final line finishes it off: "KEEPS paying forever"—. Migration A did 98.5% of the work and will never reap the prize, because the legacy is never turned off.
Migration B finishes. In sprint 7, the forcing function —a done deadline, the decision that the last stretch closes in such-and-such sprint— pushes the last 15 calls to zero. legacy_calls == 0, done is met, and the legacy is DELETED. From sprint 8, the column says "DELETED" and the cost accum. freezes at 70: it stopped paying. Migration B reaped the prize —a single system, the monolith one slice smaller, the team free for what's next—.
Now compare the costs: A paid 120 and counting (forever); B paid 70 and stopped. And the gap grows every sprint: at sprint 12 it's 120 against 70, but at sprint 20 it would be 200 against 70, at sprint 50 it would be 500 against 70. The eternal migration doesn't have a fixed cost; it has a cost that accumulates without limit, because it never stops paying the two rents. And all for a last 1.5% that cost one sprint to close. The final line says it bluntly: the difference isn't technical —both had the same speed the first six sprints—; it's having, or not, a done criterion that forces closing the last stretch.
Deep dive: why the last stretch doesn't close on its own, and how to force it
The eternal migration has a structural cause worth understanding, because it explains why the last stretch needs a force to close and doesn't close by itself.
The problem is that the incentives invert in the last stretch. At the start of the migration, each sprint cuts a lot of traffic (200, 300 calls) and the progress is visible, motivating, celebrated —there's urgency and reward—. At the end, the last 1.5% are the hardest and least visible cases: the flow a single customer uses once a month, the nightly batch process nobody remembers, the integration with a forgotten partner. Closing those cases is a lot of work for very little visible burn-down —a whole sprint to go from 15 to 0—, and the benefit (being able to delete the legacy) is real but deferred. Meanwhile, the legacy "barely bothers anymore" (98.5% of the traffic is already on the new), so the pressure to finish it is minimal. The combination —much effort, little visible progress, little urgency— makes the last stretch always lose against anything else on the list of priorities. It doesn't close on its own because nothing pushes it to close.
Visible progress / urgency
│
high│ ● ● ●
│ ● ●
│ ●
│ ●___________________ <- the last stretch:
low│ much effort, little
│ burn-down, zero urgency
└────────────────────────────────────
sprint 1 "almost" forever
(here the migration dies)
The solution is a forcing function: a mechanism that forces the close of the last stretch despite the lack of natural urgency. There are several forms, and good migrations use at least one:
- A done deadline, committed and visible. "The
catalog's legacy is turned off on the 30th of such-and-such month" —a public date, in the plan, with an owner—. The date creates the urgency the last stretch doesn't have on its own. It's what migration B had and A didn't. - The legacy's cost, made visible. Putting on a dashboard, each sprint, how much it costs to keep the legacy alive (operation, maintenance, the divided team). When the cost of "almost" is visible and accumulates in everyone's view, the pressure to close it appears. The 120-and-counting number of the example is that visibility.
- The definition of done as the project's gate. The migration project doesn't close —isn't declared finished, the team isn't released, there's no celebration— until the legacy is deleted. If "finished" requires the deletion (lesson 6), there's no way to declare victory at 98.5%; the project stays officially open, with its pressure, until the real close.
- Attacking the last stretch first, not at the end. A preventive variant: identify the rare and difficult cases early (when there's energy and urgency) instead of leaving them for the end (when there's neither). Migrate the nightly process in sprint 3, not in sprint 11.
The common point of all of them is that the last stretch needs an external force, because the natural incentives abandon it. Measuring the progress —having the burn-down that shows the stall, the done criterion that requires the zero, the visible cost that accumulates— is what provides that force. A migration without instruments doesn't see it's getting stuck until it's already been a year at 98.5%; a migration with this module's dashboard sees the velocity fall in sprint 7 and acts. That is, ultimately, the why of the whole module: you measure to finish, and the eternal migration is what happens to whoever doesn't measure.
An important clarification so as not to overcorrect: forcing the close does not mean turning off the legacy with cases still depending on it. The forcing function pushes to complete the last stretch (really close the 15 calls by migrating their cases to the modern), not to turn off the legacy leaving those cases unserved. The difference between "closing" and "abandoning" is that closing migrates the cases that are missing; turning off without migrating them breaks them. Done requires a real legacy_calls == 0, not legacy_calls == 0 from having turned off the legacy and left 15 requests failing. The forcing function speeds up the last stretch's work; it doesn't skip it.
Common mistakes
Leaving the last stretch for the end "because it's almost". What happens: the team migrates the easy 90% fast and leaves the rare and difficult cases for the end, where they never get done. Why it happens: the easy cases give a visible and satisfying burn-down; the hard ones are a lot of work for little apparent progress, so they're postponed. How to spot it: the burn-down drops fast and then flattens far from zero (the "stuck one" shape of lesson 3); the remaining legacy_calls are always the same rare cases, sprint after sprint. How to fix it: attack the last stretch early, when there's energy and urgency, not at the end when there's neither. Identify the rare and difficult cases at the start and migrate some in the first sprints. And when you reach the last stretch, give it a forcing function (deadline, visible cost) that pushes it, because it won't close on its own —the incentives abandon it right there—.
Not having a done criterion, so the migration is never declared finished. What happens: the migration reaches 98% and stays there because there's no defined finish line that forces closing the remaining 2%. Why it happens: without a written done (lesson 6), "finished" is a feeling, and at 98% the feeling is "almost, we'll close it later" —a "later" that doesn't come—. How to spot it: nobody can say when the migration will be finished; the legacy has been at the same percentage for months; the project isn't "closed" or "open", just floating. How to fix it: define the done (lesson 6) and treat it as the project's gate —the migration doesn't close until the legacy is deleted—. Without a done criterion, there's no way to know you got stuck or pressure to get unstuck; with it, the 98.5% is visibly "not finished" and the project stays officially open until the real close. The done criterion is the remodel's delivery date: without it, "almost" is forever.
Not measuring (or making visible) the cost of maintaining two systems. What happens: the team doesn't keep the tally of how much the legacy alive costs, so the eternal migration doesn't feel expensive and nobody pushes to close it. Why it happens: the cost of operating the legacy is spread out (servers, licenses, maintenance, the team's attention) and doesn't appear as a single visible figure. How to spot it: when you ask "how much does having the legacy on cost us?", nobody has the number; the cost is real but invisible, so it generates no pressure. How to fix it: make the cost visible —a dashboard, each sprint, with what it costs to keep the legacy alive (operation, the divided team, the complexity of two systems)—. When the cost of "almost" accumulates in everyone's view (like the 120-and-counting of the example), the pressure to close it appears on its own. What isn't measured isn't felt, and what isn't felt stays at 98.5% forever; the visible cost is what turns the eternal migration from an abstract problem into a figure that hurts every month.
Exercises
Exercise 1 — Read the two costs. In the output, migration A paid 120 at sprint 12 and B paid 70. (a) Why are they equal up to sprint 6 and differ after? (b) How much would each pay at sprint 20? (c) What does it represent that B's cost "freezes" and A's doesn't?
See solution
(a) They're equal up to sprint 6 because the two migrations have exactly the same speed in that part (they drop 1000→700→400→150→40→15), so both pay the cost of maintaining the two systems alive the same 6 sprints: 60 each at sprint 6. They differ from sprint 7 because there B reaches 0 and deletes the legacy (stops paying), while A gets stuck at 15 (keeps paying). The difference isn't in the technical part (identical), but in whether the last stretch is closed.
(b) At sprint 20: B would still be at 70 —it deleted the legacy in sprint 7 and hasn't paid a cent more since—. A would be at 200 —it paid 10 per sprint over the 20 sprints, and continues—. The gap, which at sprint 12 was 50 (120 vs 70), at sprint 20 would be 130 (200 vs 70), and grows without limit.
(c) That B finished and A didn't. B's cost freezes because it deleted the legacy: it stopped maintaining two systems, reaped the prize, the expense stopped. A's cost doesn't freeze because it never finished: it keeps maintaining two systems indefinitely, paying the "two rents" forever. A cost that freezes is the signature of a closed migration; a cost that rises without stopping is the signature of the eternal migration —the constant drip of "almost" that accumulates endlessly—.
Exercise 2 — Why the last stretch gets stuck. The text says that "the incentives invert in the last stretch". (a) Why does the first 90% get done on its own and the last 2% doesn't? (b) Give a concrete example of a rare case that would be left in that last stretch in Mercado's catalog. (c) Why is "the legacy is barely used" exactly what prevents finishing it?
See solution
(a) Because at the start each sprint cuts a lot of traffic (200-300 calls): the progress is visible, satisfying, and urgent, so it gets done with energy. At the end, the last 2% are the hardest and least visible cases, and closing them is a lot of work for very little burn-down (a whole sprint to go from 15 to 0). The high effort, the low visible progress, and the minimal urgency make the last stretch lose against any other priority. The first 90% has the incentives in its favor; the last 2% has them against.
(b) A concrete example: a nightly process that generates the catalog's monthly inventory report, which runs at 3 a.m. on the last day of the month and still queries the legacy's tables. Nobody sees it day to day, nobody remembers it exists, and it only runs once a month —so it's among the last cases to be discovered and the easiest to postpone—. As long as that process keeps touching the legacy, legacy_calls doesn't reach zero, but since it's invisible the rest of the month, it generates no urgency. Other examples: an endpoint only a B2B partner uses, or a returns flow that triggers rarely.
(c) Because "barely used" eliminates the urgency to finish it, which is the only thing that would push closing the last stretch. If the legacy served 50% of the traffic, turning it off would be urgent (half the business depends on it). But at 1.5%, the legacy "doesn't bother" —98.5% is already on the new—, so there's always something more important to do, and the 1.5% is postponed indefinitely. It's the paradox of the eternal migration: the more you advanced, the less urgent finishing becomes, right when you're closest to reaping the prize. That's why the last stretch needs an external force (the forcing function): the natural urgency is already exhausted.
Exercise 3 — Design the forcing function. Mercado's catalog migration has been stuck for three sprints at legacy_calls = 15 (98.5%). You're asked to unstick it. (a) Propose two concrete forcing functions. (b) How would you make sure not to "force the close" by turning off the legacy with cases still depending on it? (c) What would have prevented this stall from the start?
See solution
(a) Two concrete forcing functions: (1) a committed and visible done deadline —"the catalog's legacy is deleted in sprint 10, date in the plan, with an assigned owner"—, which creates the urgency the last stretch lost; and (2) making the cost visible —a line in the team's dashboard, each sprint, with what it costs to keep the legacy alive (operation + the divided team), accumulating in everyone's view—, so that "almost" stops feeling free. A third option: treat the done as the project's gate —not officially close the migration or release the team until the legacy is deleted—, so the 98.5% counts as "not finished" and the project keeps its pressure.
(b) I'd make sure the forcing function pushes to complete the last stretch, not to skip it: before the deadline, identify exactly what those 15 calls are (with observability: which cases, which flows, which customers generate them) and migrate those cases to the modern, so that when the deadline arrives legacy_calls is really zero —because the cases already live on the new—, not zero from having turned off the legacy leaving them failing. Done requires a real legacy_calls == 0; turning off the legacy with 15 cases still depending on it isn't finishing, it's breaking 15 flows. The forcing function speeds up the work, it doesn't skip it.
(c) What would have prevented the stall from the start: (1) define the done on day one (lesson 6), with the deletion as the criterion, so the 98.5% was visibly "not finished"; (2) attack the last stretch early —identify the rare and difficult cases in the first sprints, when there was energy and urgency, and migrate some then— instead of leaving them for the end; and (3) measure the progress with the burn-down (lesson 3), which would have shown the velocity falling to zero in sprint 7 —the early sign of the stall— in time to act, instead of discovering the problem three sprints later. In one phrase: have this module's dashboard from the start. The eternal migration is what happens to whoever doesn't measure to finish.
Summary and next step
In this lesson you faced the worst outcome of a migration: the eternal migration, which gets stuck in the last stretch and maintains two systems forever. You saw, with the remodel that's "almost" and the three years among rubble paying two rents, that the cost of not finishing is a drip that accumulates endlessly. And you executed it: two migrations with the same speed the first six sprints, one without a done criterion that got stuck at 98.5% and paid 120-and-counting, and another with a forcing function that reached zero, deleted the legacy, and froze its cost at 70. You learned why the last stretch doesn't close on its own (the incentives invert: much effort, little visible progress, zero urgency), the forcing functions that close it (done deadline, visible cost, done as the project's gate, attacking the last stretch early), and that forcing the close is completing the last stretch, not turning off the legacy with cases still depending on it.
Before moving on you should be able to: recognize the eternal migration by the shape of its burn-down and its accumulated cost; explain why the last stretch gets stuck even though the technical part is solved; propose concrete forcing functions to unstick a migration; and argue why measuring the progress is what avoids the eternal migration.
Lesson 8 is the capstone: integrating the module's three instruments —the burn-down, the fitness function with the ratchet, and the done criterion— into a single MigrationTracker, and running it as a journal of Mercado's catalog migration that does finish. You're going to see the complete dashboard work together: the burn-down dropping, the fitness function catching someone who adds code to the legacy in sprint 3, the ratchet tightening the baseline, and the done criterion triggering the deletion on reaching zero. It's the whole module in a single executed simulation, and the guide's close.
Resources
- Martin Fowler, "StranglerFigApplication" (2004) — martinfowler.com/bliki/StranglerFigApplication.html. Fowler warns that the strangler's danger is staying half done —two systems coexisting—: the eternal migration is exactly that danger, and finishing (retiring the legacy) is what avoids it. In English.
- Sam Newman, Monolith to Microservices (O'Reilly, 2019), ch. 3 — Newman describes the risk of migrations that never finish decomposing the monolith and end up with the worst of both worlds; the source of this lesson's argument. In English.
- Neal Ford, Rebecca Parsons, and Patrick Kua, Building Evolutionary Architectures (O'Reilly, 2nd ed., 2022) — on measuring and governing the state of an evolving architecture so the change reaches its end instead of staying indefinite; the framework for having the forcing functions that close a migration. In English.
- Martin Fowler, martinfowler.com — the bliki with the entries on strangler fig and modernization that ground why a migration must finish with the legacy retired and not coexist forever. In English.