Expand-contract: never a destructive change all at once

Overview

Lesson 1 gave you the map of the five pieces of data migration. This lesson sets the law under which they all operate, the one that makes it possible to change data without shutting down the system: expand-contract. Said in one sentence: you never remove the old and add the new in the same step; first you expand (add the new, leaving the old intact), then you migrate (move those who used the old toward the new), and only at the end you contract (remove the old, when nobody uses it anymore). A destructive change does both things at once —removes and adds in an instant— and that's why it breaks everyone who still depended on the old. Expand-contract separates those two things in time, and that separation is what keeps the system alive.

The name says it: the schema grows (expand) before it shrinks (contract), and never the other way around. Between those two moments there's a phase of coexistence —the old and the new live together— which is uncomfortable (you have two shapes of the same thing at once) but is exactly the price of not shutting anything down. Martin Fowler also calls it parallel change: the change happens in parallel, with both versions alive, instead of in an atomic jump.

This idea isn't only for columns of a database. It's the same for renaming a field, changing a type, splitting a table in two, or changing the format of a value. And it's the same one that governs the dual-write, the backfill, and the read-switch of the following lessons: each of them is an application of "add before removing". The dual-write adds the new store without removing the old; the read-switch moves the reads before retiring the old. The whole module is expand-contract applied, so understanding it here, in its pure form, is understanding the rest.

Connection with the module. This lesson gives the framework; the following ones instantiate it. Lesson 3 (dual-write) is the "expand" of the writes: adding the second destination without removing the first. Lesson 4 (backfill) fills what the expand left empty. Lesson 7 (read-switch) is the "migrate" and the "contract" of the reads: moving the readers to the new and only afterward retiring the old. Notice the boundary: here we work the framework at the schema/field level (migrating price_cents to price_usd), with in-memory reads; the migration of the content of the rows between two different stores is what lessons 3 to 7 build. Expand-contract is the principle; they are the mechanics.

An analogy: rewiring the house with the light always on

You're going to renovate the electrical installation of your house —the old wiring can no longer keep up—. But you set a condition: the house isn't left without light for a single minute. There are people living there, a refrigerator running, someone working from home. Cutting all the power over the weekend to redo everything at once isn't an option; that would be the big-bang.

The destructive way would be: you cut the old wire of a room and, in that gap of time, you run the new one. During those minutes —or those hours— the room (or half the house, if the old wire fed several) is left without light. And if the new wire turns out to be faulty and doesn't work, you already cut the old: you're left in the dark with no quick way to go back. You removed before the new was tested.

The expand-contract way is how a good electrician who can't cut the power does it:

1. Expand. Run the new wire next to the old, without touching the old. The two wires run in parallel along the wall. The house is still fed by the old; the new is in place but not yet connected to the outlets. Nobody noticed anything; there's just more wire.
2. Migrate. Room by room, move the outlets from the old wire to the new. When a room is finished, that room now draws its power from the new wire —and keeps having light the whole time, because the new was already run and tested before connecting it—. Move to the next room. At no moment is a room left in the dark.
3. Contract. Only when all the rooms draw their power from the new wire, and you confirmed it by checking that everything works, do you remove the old wire from the wall. Nobody uses it anymore, so removing it turns nothing off.

Notice the order and what it guarantees. The new wire is added before anything depends on it (expand). The outlets are moved little by little, with both options available (migrate). And the old wire is removed only when nobody uses it anymore (contract). The light is never cut because there's never an instant when the old is no longer there but the new isn't yet. A destructive change creates exactly that instant —the gap in the dark—; expand-contract eliminates it by putting the new first and removing the old at the end.

In data terms: the old wire is the price_cents field, the new is price_usd. Running it alongside is adding price_usd without removing price_cents. Moving the outlets is changing the readers to use price_usd. And retiring the old wire is deleting price_cents when nobody reads it anymore. Let's execute exactly that.

Worked example: migrate the price field in a live system

We're going to compare the two ways —destructive and expand-contract— measuring the only thing that matters: how many reads break while the system is alive. The scenario: a catalog row stores the price in price_cents (the old field) and we want to migrate it to price_usd (the new). While we make the change, the system keeps serving reads —100 of them—. An old reader expects price_cents; a new reader expects price_usd. The key, and what makes the example realistic: the code and the schema don't change in the same instant. When you change the schema, the deployed reader is still the one that was there; it updates afterward, in another deployment. That gap is where a destructive change does harm.

# --- A catalog row. Starts with the old field: price_cents. ---
def seed_row():
    return {"sku": "ssd-1tb", "name": "SSD 1TB", "price_cents": 8999}

# --- Deployed readers. The old expects price_cents; the new, price_usd. ---
def old_reader(row):
    return row["price_cents"]          # KeyError if the field no longer exists

def new_reader(row):
    return round(row["price_usd"] * 100)

# --- A batch of reads: serves N reads and counts how many BREAK. ---
def serve_reads(n, row, reader):
    broken = 0
    for _ in range(n):
        try:
            reader(row)
        except KeyError:
            broken += 1
    return broken


# ---------------------------------------------------------------------------
# SCENARIO A: destructive change all at once.
#   In a single step we remove price_cents and add price_usd. But the deployed
#   reader is still the old one (code and schema don't change atomically).
# ---------------------------------------------------------------------------
row = seed_row()
broken_a = 0
broken_a += serve_reads(50, row, old_reader)          # reads 1..50: all fine

# --- The destructive change: removes the old AND adds the new, at once. ---
row["price_usd"] = row.pop("price_cents") / 100       # price_cents DISAPPEARS
broken_a += serve_reads(50, row, old_reader)          # reads 51..100: KeyError

# ---------------------------------------------------------------------------
# SCENARIO B: expand-contract (additive, in three steps).
#   EXPAND: adds price_usd WITHOUT removing price_cents (both coexist).
#   MIGRATE: changes the reader to new_reader (price_usd already exists).
#   CONTRACT: only now removes price_cents (nobody reads it anymore).
# ---------------------------------------------------------------------------
row = seed_row()
broken_b = 0
broken_b += serve_reads(25, row, old_reader)          # reads 1..25: old reader

# EXPAND: add the new field, leave the old. Nothing breaks.
row["price_usd"] = row["price_cents"] / 100
broken_b += serve_reads(25, row, old_reader)          # reads 26..50: old still OK

# MIGRATE: now switch the reader to the new. price_usd is there, no failure.
broken_b += serve_reads(25, row, new_reader)          # reads 51..75: new reader

# CONTRACT: nobody reads price_cents now -> can remove without breaking anything.
del row["price_cents"]
broken_b += serve_reads(25, row, new_reader)          # reads 76..100: without the old


print("Migrate the price field in a LIVE system (100 reads each)\n")
print(f"{'strategy':<22}{'broken reads':>16}")
print("-" * 38)
print(f"{'destructive change':<22}{broken_a:>16}")
print(f"{'expand-contract':<22}{broken_b:>16}")
print("-" * 38)
print("\n  Destructive: removing and adding at once broke every reader that still")
print("  expected the old field (50 reads down).")
print("  Expand-contract: additive first, remove at the end -> 0 broken reads.")

What to expect. When you run the file, the output is exactly this:

Migrate the price field in a LIVE system (100 reads each)

strategy              broken reads
--------------------------------------
destructive change                  50
expand-contract                      0
--------------------------------------

  Destructive: removing and adding at once broke every reader that still
  expected the old field (50 reads down).
  Expand-contract: additive first, remove at the end -> 0 broken reads.

The two numbers say it all: 50 broken reads with the destructive change, 0 with expand-contract.

In the destructive scenario, the first 50 reads go fine: the old reader reads price_cents, which exists. Then the destructive change happens —row.pop("price_cents") removes the old field and creates price_usd in the same step—. But the deployed reader is still old_reader: in real life you can't update the schema and all the reader code in the same microsecond. So reads 51 to 100 do row["price_cents"] on a row that no longer has that field: KeyError, 50 times. Each of those 50 reads is an error served to a user. That's the gap in the dark: the old is no longer there, but the reader still asks for it.

In the expand-contract scenario, the change is split into three moments and that gap is never created:

• Expand (after 50 healthy reads with the old reader): we add price_usd without removing price_cents. Now the row has both fields. The old reader keeps reading price_cents, which is still there: 0 broken reads. The schema grew, nothing broke.
• Migrate: we switch to the new_reader, which reads price_usd. That field already exists (we put it in during the expand), so the new reader works immediately: 0 broken. The readers move to the new field while the old is still available.
• Contract: now that nobody reads price_cents (all the readers are new), we delete it. The new reader uses price_usd, which is still there: 0 broken. The schema shrank, and since nobody depended on what we removed anymore, nothing broke.

Total: 0 broken reads. The entire difference between 50 and 0 is the order: destructive removes and adds at the same time (creating the gap); expand-contract adds first, migrates in the middle, and removes at the end (there's never a gap).

Deep dive: why the three phases and in that order

The core of expand-contract is a rule of dependencies. A reader depends on the field it reads. If you remove a field someone depends on, you break them. The only safe way to remove something is to make sure first that nobody depends on it —and that takes time, because you have to move everyone who depended on it to the new—. The three phases are exactly that process:

        expand              migrate               contract
   ┌──────────────┐   ┌──────────────────┐   ┌──────────────┐
   │ add the new; │   │ move those who   │   │ remove the   │
   │ leave the    │──▶│ used the old     │──▶│ old (nobody  │
   │ old field    │   │ toward the       │   │ depends      │
   │ intact       │   │ new              │   │ now)         │
   └──────────────┘   └──────────────────┘   └──────────────┘
   old and new        coexistence:           only the new
   coexist            both available         remains

Why expand first. If you're going to move readers to the new field, the new field has to exist before the first reader asks for it. Adding is a safe operation: nobody breaks because an extra field appears (the old readers ignore it). That's why the schema's growth can always go first, without risk.

Why migrate in the middle. You can't move all the readers in an instant. In a real system there are several instances of the service, staggered deployments, maybe clients that update at their own pace. The coexistence phase —both fields alive— is what gives time for all the readers to move to the new without any of them being left without their field. The more distributed the system, the longer and more necessary this phase.

Why contract at the end. Removing is the dangerous operation: it breaks everyone who still depends. That's why it's last, and it's only done when you can prove that nobody depends on the old. That proof —"nobody reads price_cents anymore"— is the one that in the following lessons becomes the green parallel-run and the burn-down of calls to the old: you don't remove on a hunch, you remove on evidence.

There's a useful symmetry with module 3. The strangler fig also adds the new (the modern) alongside without removing the old (the legacy), diverts the traffic little by little (coexistence), and retires the old only when the burn-down reaches zero. Expand-contract is the same skeleton —add, coexist, remove— applied to the data and the schema instead of the traffic. If you mastered the strangler, you already have the mental shape; here only the object it acts on changes.

An important nuance: expand-contract has a temporal cost, the coexistence phase. During it you keep two fields (or two stores) synchronized, which is more work and more complexity than having just one. That cost is real, but it's temporal (it disappears when you contract) and bounded (you know it ends when nobody uses the old). The mistake isn't paying that cost; it's not paying it (doing the destructive change) or paying it forever (never contracting, staying with both fields indefinitely —the eternal migration that module 7 fights—).

Common mistakes

The destructive change all at once: removing and adding in the same step. What happens: the team renames a column, changes a type, or replaces a field in a single schema migration —ALTER TABLE ... RENAME, or removing and adding at once—. Why it happens: it's what looks like "the change", a single clean operation; splitting it into three phases feels bureaucratic. How to spot it: the schema migration removes something (a DROP COLUMN, a RENAME) in the same step it adds or changes; the reader code and the schema deploy as if they were atomic. How to fix it: split every schema change into expand (add, additive, safe) and contract (remove, in another later deployment, when nobody uses the old), with the code migration in the middle. Never RENAME in a live system: it's ADD the new, migrate, and DROP the old in three separate steps. The mechanical rule: if a schema migration removes something the deployed code still uses, it's destructive —and it's going to break reads in the gap between deploying the schema and deploying the code—.

Contracting before everyone has migrated. What happens: the team does the expand and the migrate, sees that "the system already uses the new field", and deletes the old right away —but there was a lagging reader (a nightly job, an old instance, a client that didn't update) that still used the old field—. Why it happens: it's easy to believe "we already migrated" when what you see migrated, forgetting the less visible readers. How to spot it: after deleting the old field, errors appear in peripheral components —reports, batch jobs, integrations— that nobody touched in the migration. How to fix it: the contract is only safe when you can prove that nobody reads the old, not when you believe nobody reads the old. That proof is measurable: instrument the old field to count its reads and wait for the count to reach and stay at zero (it's module 7's burn-down). Contracting is the dangerous operation; it's done on evidence of zero use, not on the feeling that the migration is already done.

Staying in coexistence forever (never contracting). What happens: the team does the expand and the migrate, everything works with the new field, and... leaves the old field there "just in case". Months later the table has both fields, the code writes to both out of habit, and nobody remembers which is the truth. Why it happens: contracting gives no visible value (the system already runs on the new) and is a little scary (what if something still uses it?). How to spot it: there are "old" fields that have gone months without anyone reading them but are still there, and code that keeps both synchronized for no reason. How to fix it: expand-contract ends in contract. The coexistence phase is a means, not a destination: it's expensive (two things to keep synchronized) and confusing (two sources of truth). Set from the start an exit condition —"when the read count of the old field is zero for N days, it gets deleted"— and honor it. An expand without its contract is a debt that grows; module 7 treats this trap of the migration that never ends in depth.

Exercises

Exercise 1 — The gap in the dark. In the destructive scenario, exactly 50 reads broke. (a) Why 50 and not 100, nor 0? (b) What concrete event, in the code, created the "gap in the dark"? (c) With the wiring analogy, what does that gap represent?

See solution

(a) Because the destructive change happened right in the middle, after 50 reads and before the other 50. The first 50 ran with price_cents still present (old reader, old field: fine). The last 50 ran after pop removed price_cents, but with the reader still old (it still asks for price_cents): KeyError, 50 times. If the change had happened at read 30, 70 would have broken; at 90, only 10. The number depends on how long the old reader survives the schema change —and in reality that gap is the time between deploying the schema and deploying the new code, which is never zero—.

(b) The event was row["price_usd"] = row.pop("price_cents") / 100. The pop removes price_cents in the same instant it creates price_usd. That "remove" is the destructive part: in the next microsecond, any reader that still expects price_cents breaks. The operation mixed adding (safe) and removing (dangerous) into a single step, which is the definition of the destructive change.

(c) That gap represents the instant when you cut the old wire before the room drew power from the new. The room (the old reader) was left in the dark because its power source (the price_cents field) disappeared while it still depended on it. Expand-contract eliminates that gap by running the new wire first and cutting the old only when nobody uses it anymore.

Exercise 2 — Order the phases. A team wants to split the name field in two: first_name and last_name. They have these five actions. Order them into a safe expand-contract sequence, and mark which is expand, which migrate, and which contract: (i) delete the name column; (ii) deploy the code that reads first_name/last_name; (iii) add the first_name and last_name columns; (iv) fill first_name/last_name from name for the existing rows; (v) confirm (with metrics) that nobody reads name anymore.

See solution

The safe order is iii → iv → ii → v → i:

1. (iii) add the first_name and last_name columns — EXPAND. Additive and safe: the old readers that use name ignore it. The schema grows first.
2. (iv) fill first_name/last_name from name — part of the expand/migrate of the data: the new columns exist but are empty for the old rows; they have to be filled (this is a backfill, lesson 4). It's done after adding the columns and without touching name.
3. (ii) deploy the code that reads first_name/last_name — MIGRATE. Moves the readers from the old field to the new, which already exists and is already filled. The coexistence (name + the two new) gives time for all the instances to update.
4. (v) confirm that nobody reads name — the evidence required before contracting. It's not deleted on belief, but on measurement of zero use.
5. (i) delete the name column — CONTRACT. Last, and only after confirming that nobody depends on it.

Key points: filling (iv) goes after adding (iii) —you can't fill columns that don't exist—; deploying the new code (ii) goes after the columns exist and are filled —otherwise, the new code would read empty columns—; and deleting name (i) goes at the end, after the evidence (v). Inverting any of those orders creates a gap in the dark.

Exercise 3 — Safe and dangerous changes. Classify each schema change as safe (additive, can be done in one step without breaking readers) or dangerous (destructive, requires expand-contract), and explain why: (a) adding a new column with a default value; (b) renaming an existing column; (c) adding an index; (d) changing a column's type from int to string; (e) making a constraint stricter (e.g., NOT NULL on a column that had nulls).

See solution

• (a) Adding a new column → SAFE. It's pure expand: the old readers ignore the new column; none breaks. It can be done in one step. (With a default value, moreover, the old rows are immediately consistent.)
• (b) Renaming a column → DANGEROUS. A RENAME is removing the old name and putting the new at the same time: every reader that uses the old name breaks in the instant of the rename. It requires expand-contract: add the new column, copy the data, migrate the readers, and delete the old —never a direct rename in a live system—.
• (c) Adding an index → SAFE. It doesn't change the schema the readers see nor remove anything; it only speeds up queries. (In large databases it may require creating it without locking the table, but it doesn't break readers.)
• (d) Changing the type int → string → DANGEROUS. The readers that expect an int may break on receiving a string, and the type change is usually destructive of the old value. It requires expand-contract: add a new column of the new type, migrate the data and the readers, and contract the old.
• (e) Making a constraint stricter (NOT NULL) → DANGEROUS. If the column had nulls, imposing NOT NULL all at once fails or breaks existing rows, and the writers that inserted nulls break. It requires the inverse but analogous process: first ensure (through data migration) that there are no more nulls and that nobody writes nulls, and only then impose the constraint.

The mechanical rule: adding is safe; removing, renaming, changing type, and restricting are dangerous. Everything dangerous decomposes into an expand (add the new) and a contract (remove the old) separated in time, with the migration in the middle.

Summary and next step

In this lesson you set the law that governs the whole module: expand-contract, never a destructive change all at once. You saw, with the house wiring changed room by room with the light always on, that safety comes from the order —add the new (expand), move those who use the old (migrate), and remove the old only when nobody depends on it (contract)—, and that a destructive change breaks because it does the "remove" and the "add" in the same instant, creating a gap where the old is no longer there but the readers still ask for it. And you executed it: migrating price_cents to price_usd in a live system broke 50 reads at once and 0 with expand-contract. The entire difference was the order.

Before moving on you should be able to: name the three phases (expand, migrate, contract) and why they go in that order; distinguish a safe schema change (additive) from a dangerous one (destructive); explain what the "gap in the dark" is and how expand-contract eliminates it; and recognize the two ways to fail with the framework (contracting too early, or never contracting).

Lesson 3 instantiates the "expand" of the writes: the dual-write. You're going to add a second destination to each write —the new store— without removing the first —the old—, and you're going to see that this keeps the new up to date with everything written from now on. But you're also going to see, executed, the exact limit of the dual-write: it does not touch what was written before turning it on. That historical gap —what the expand left empty in the old rows— is what lesson 4 will fill with the backfill. Expand-contract has just given you the framework; the dual-write is its first piece in motion.

Resources

• Martin Fowler, "ParallelChange" — martinfowler.com/bliki/ParallelChange.html. The canonical entry of the expand-contract pattern (also called parallel change): the three phases (expand, migrate, contract) to change an interface or a schema without breaking its clients. The foundational reading of this lesson. In English.
• Pramod Sadalage and Martin Fowler, Refactoring Databases: Evolutionary Database Design (Addison-Wesley, 2006) — the catalog of database refactorings, each designed as a small, reversible change that preserves behavior. The book's "Rename Column" is exactly this lesson's expand-contract applied to a column. In English.
• Pramod Sadalage, "Evolutionary Database Design" — martinfowler.com/articles/evodb.html. The article that sums up how to evolve a database's schema continuously and incrementally, with versioned migrations and downtime-free transitions. The framework of the whole module. In English.
• Sam Newman, Monolith to Microservices (O'Reilly, 2019), ch. 4, "Decomposing the Database" — the patterns to change and separate a monolith's schema in steps, including how to add before removing when splitting tables and moving data between services. In English.