Module 3: Gradual Rollout

Blast radius grows with each stage

Next →

Description

Module 1 measured blast radius (blastRadius()) by comparing three different ways of launching recommendations: 100% at once, 10%, and a 1% canary. This lesson takes that same function, unchanged, and applies it to the full ramp's four stages at once — not as alternatives being compared against each other, but as consecutive steps of the same rollout. And it adds a new question module 1 didn't answer: when you move from one stage to the next, how many new people end up exposed in that specific transition?

Connection to the module. This lesson directly connects module 1 with this module 3: it reuses blastRadius() exactly as it stood in that module's lesson 4, and wraps it in blastRadiusPerStage(), applied to the same four-stage ramp this module's lesson 2 defined. It introduces no new rollout concept — it puts an exact number on something lesson 2 mentioned in passing: that the ramp exists, in part, because the jumps between stages aren't all the same size.

An analogy: opening a dam's gates, one at a time

A dam doesn't release all its contained water by opening every gate at the same time — it opens them one by one, and each gate that opens releases more flow than the previous one, not because the gates are bigger, but because the volume built up behind them is already greater. Opening the first gate, with the dam nearly full, releases a manageable flow. Opening the last one, when little water is left contained, barely changes anything. Sequence matters, and the size of each released flow "jump" isn't uniform — it depends on how much was contained up to that point.

recommendations' rollout has the same shape, with user exposure standing in for flow. Going from 1% to 10% releases a certain amount of new exposure. Going from 10% to 50% releases a lot more — not because that stage is "more dangerous" in itself, but because the jump in total users is simply bigger. Understanding where the big jumps are is as important as understanding the base's total size.

Worked example: blastRadiusPerStage() on the full ramp

We apply blastRadius() — unchanged from module 1 — to the ramp's four stages, and calculate how many new people get exposed at each transition:

// blastRadiusPerStage: reuses blastRadius() exactly as it stood in module 1
// (lesson 4), but applied to the FOUR stages of the full ramp at once, and
// adds the exposure INCREMENT between one stage and the next -- how many NEW
// people end up exposed when moving from one stage to the other.
function blastRadius({ totalUsers, rolloutPercent, regressionRate }) {
  const exposedUsers = Math.round(totalUsers * rolloutPercent);
  const affectedUsers = Math.round(exposedUsers * regressionRate);
  return { rolloutPercent, exposedUsers, affectedUsers };
}

function blastRadiusPerStage({ totalUsers, regressionRate, stages }) {
  let prevExposed = 0;
  return stages.map((s) => {
    const { exposedUsers, affectedUsers } = blastRadius({ totalUsers, rolloutPercent: s.percent, regressionRate });
    const newlyExposed = exposedUsers - prevExposed;
    prevExposed = exposedUsers;
    return { label: s.label, percent: s.percent, exposedUsers, affectedUsers, newlyExposed };
  });
}

const totalUsers = 250000; // same buyer base as module 1
const regressionRate = 0.05; // same known regression rate

const stages = [
  { label: 'canary 1%', percent: 0.01 },
  { label: 'rollout 10%', percent: 0.10 },
  { label: 'rollout 50%', percent: 0.50 },
  { label: 'rollout 100%', percent: 1.00 },
];

console.log('=== blastRadiusPerStage: recommendations\' full ramp at Mercado ===\n');
const perStage = blastRadiusPerStage({ totalUsers, regressionRate, stages });
perStage.forEach((s) => {
  console.log(s.label.padEnd(14) + 'exposedUsers=' + s.exposedUsers.toLocaleString('en-US').padStart(7) +
    '  affectedUsers=' + s.affectedUsers.toLocaleString('en-US').padStart(6) +
    '  newlyExposed=+' + s.newlyExposed.toLocaleString('en-US').padStart(7));
});

const biggestJump = perStage.reduce((max, s) => (s.newlyExposed > max.newlyExposed ? s : max));
console.log('\nThe biggest jump in new exposure happens at "' + biggestJump.label + '" (+' +
  biggestJump.newlyExposed.toLocaleString('en-US') + ' new people) -- not at the first canary.');

What to expect. Running the file with Node, the output is exactly this:

=== blastRadiusPerStage: recommendations' full ramp at Mercado ===

canary 1%     exposedUsers=  2,500  affectedUsers=   125  newlyExposed=+  2,500
rollout 10%   exposedUsers= 25,000  affectedUsers= 1,250  newlyExposed=+ 22,500
rollout 50%   exposedUsers=125,000  affectedUsers= 6,250  newlyExposed=+100,000
rollout 100%  exposedUsers=250,000  affectedUsers=12,500  newlyExposed=+125,000

The biggest jump in new exposure happens at "rollout 100%" (+125,000 new people) -- not at the first canary.

Look at the newlyExposed column from top to bottom: 2,500, then 22,500, then 100,000, then 125,000. Every jump is bigger than the last — and not by a little. The jump from 50% to 100% (+125,000 people) is fifty times bigger than the initial canary's jump (+2,500). This has a direct consequence for how you should think about each stage's risk: the ramp's most delicate stage isn't the first one — the canary, with the smallest exposure of all — it's the last one, precisely because that's where most of the contained "flow" gets released.

Why this changes how you think about each stage's advance criteria

This observation has a direct practical implication for lesson 4: if the exposure jump grows with each stage, the advance criterion — and the confidence you need before applying it — should get stricter, not more relaxed, as you climb the ramp. It's tempting to reason backward: "we already reached 50%, we've already confirmed several times that everything's fine, let's move faster from here on" — but the jump from 50% to 100% is, in terms of newly exposed people, the biggest of the entire ramp. Confidence accumulated at previous stages is real, valuable information (you got this far for a reason), but it shouldn't translate into less care exactly at the moment where an error's blast radius grows the most.

This same pattern — jumps that grow, not stay constant — is also an argument for adding more intermediate stages toward the top of the ramp, not just the bottom. A ramp like 1% → 10% → 30% → 60% → 100%, with an extra rung between the original 50% and 100%, would reduce the size of the biggest final jump — at the cost, as you saw in lesson 2, of one more stage to coordinate.

Common mistakes

Assuming blast radius grows evenly between stages, instead of accelerating. What happens: someone plans the same level of review and care for the jump from 1% to 10% as for the jump from 50% to 100%, treating all four transitions as if they were equivalent. Why it happens: the ramp looks, in the list of percentages, like an even progression (1, 10, 50, 100) — it isn't obvious at a glance, without calculating it, that the number of new people behind each jump grows very unevenly. How to spot it: compare the newlyExposed column across stages, as in today's example — if the team can't say from memory which transition exposes the most new people, it's probably treating all stages as equal. How to fix it: use blastRadiusPerStage() to see each transition's real increment, and adjust the level of care (review, lesson 7's dwell time) to be greater, not less, at the biggest jumps.

Relaxing vigilance as you climb the ramp, because of confidence accumulated at previous stages. What happens: after the 10% and 50% pass with no problems, the team reviews the guardrail less rigorously before advancing to 100%, reasoning that "if it held up this far, it'll hold up the rest." Why it happens: accumulated confidence is a real signal, and it's psychologically natural to relax as a plan keeps going without a hitch. How to spot it: the time or effort spent reviewing the guardrail at the last stage is less than what was spent at the initial canary, even though the last stage exposes far more new people. How to fix it: remember the jump from 50% to 100% is, in this example, the ramp's biggest (+125,000) — review rigor should, if anything, increase at the final stages, not decrease.

Ignoring a stage's blast radius because it was already calculated once, at a previous stage, and "shouldn't change much." What happens: someone calculates blastRadius() for the 1% canary at the start of the rollout, and doesn't recalculate it for the following stages, assuming the logic has already been proven. Why it happens: recalculating at each stage feels redundant if the model (blastRadius()) has already been validated once. How to spot it: if you ask "how many new people get exposed at the stage we're about to start?" and the answer is a shrug or a number from a previous stage, the calculation isn't being repeated where it matters. How to fix it: as blastRadiusPerStage() does in today's example, calculate the blast radius — total and increment — for each specific stage, every time, before deciding to advance to it.

Exercises

Exercise 1 — Calculate the increment for a ramp with more stages. Go back to lesson 2's exercise 2's five-stage ramp: 1% → 5% → 10% → 50% → 100%. Calculate exposedUsers for the new 5% stage (with the same totalUsers: 250000 and regressionRate: 0.05), and compare its newlyExposed (relative to the previous 1%) against the original 1%-to-10% jump in the four-stage ramp.

See solution

exposedUsers at 5% = round(250000 * 0.05) = 12,500. newlyExposed relative to 1% (2,500) = 12,500 - 2,500 = 10,000. Compared to the original 1%-to-10% jump in the four-stage ramp (+22,500), adding the intermediate 5% stage splits that big jump into two smaller ones: +10,000 (from 1% to 5%) and +12,500 (from 5% to 10%, since 25,000 - 12,500 = 12,500). No individual jump disappears — total exposure by the time you reach 10% stays the same — but each individual transition ends up more contained, giving the team one more chance to catch a problem before it grows.

Exercise 2 — Find the biggest jump with a different regressionRate. If regressionRate changed from 0.05 to 0.02 (a rarer regression), would which transition has the biggest newlyExposed change? Justify your answer without needing to recalculate everything.

See solution

It wouldn't change. newlyExposed is calculated over exposedUsers (which depends only on totalUsers and rolloutPercent), not over affectedUsers (which does depend on regressionRate). Since regressionRate doesn't appear at all in exposedUsers's calculation, the transition with the biggest exposure jump — from 50% to 100%, in this ramp — stays the same no matter how common or rare the regression is. What would change, proportionally, is affectedUsers at each stage: with regressionRate: 0.02 instead of 0.05, every affectedUsers value would drop to 40% of the original (125 → 50, 12,500 → 5,000, and so on), but the pattern of where the biggest exposure jump is doesn't change.

Exercise 3 — Connect this lesson with lesson 4. In 3-4 sentences, explain why this lesson's finding — that the jump from 50% to 100% is the ramp's biggest — is an argument for having an especially strict advance criterion (lesson 4) right before that transition, and not only before the initial canary.

See solution

Lesson 4's advance criterion is the only thing standing between "the guardrail is broken" and "exposing more people" at each stage. Since the jump from 50% to 100% exposes, in one shot, 125,000 new people — half of Mercado's entire base — a weak or poorly verified criterion right at that transition has the worst possible cost if it fails: it's exactly the moment where a wrong decision (advancing when it shouldn't) affects the largest number of new people in the whole ramp. That's why the criterion's rigor — how well-measured the guardrail is, how much evidence is required before trusting it — should, if anything, be greater at that final transition, not less just because "we got this far with no problems."

Summary and next step

In this lesson you reused module 1's blastRadius(), wrapped in blastRadiusPerStage(), over recommendations' full four ramp stages: you saw blast radius doesn't grow evenly — the jump from 50% to 100% (+125,000 new people) is fifty times bigger than the initial canary's (+2,500) — and why that implies the care applied to the advance criterion should increase, not decrease, as you climb the ramp.

Before moving on you should be able to: calculate the exposure increment (newlyExposed) between any two stages of a ramp; and explain why a ramp's last stage, not its first, is usually the one exposing the biggest blast radius.

Lesson 6 adds a dimension the percentage alone doesn't cover: who, specifically, sees the feature first at each stage — deployment rings, from the internal team to the general public.

Resources

← PreviousAdvance criteria: what makes a stage "advance," concretelyNext →Deployment rings: who sees the feature first