Where `expectedLift` comes from

How this connects to the module. Lesson 4 showed that reachableUsers and baselineRate are measured —they live on a dashboard, today, verifiable by anyone—. expectedLift is different: the change doesn't exist yet, so there's no way to measure it directly. This lesson is about how to justify that number anyway, without it being a figure pulled out of thin air — and about the real cost of not doing so.

An everyday analogy: the appraiser and the comparable houses

When a real estate appraiser has to tell someone how much their house is worth, they don't close their eyes and imagine a number that "feels right". They look for comparables —similar houses, in the same neighborhood, sold in the last few months— and adjust from there: this house has one more room, that street is noisier, that other one sold just two weeks ago so the price is still representative of today's market. The final number is still an estimate, not a fact —the house hasn't sold yet—, but it's an estimate with backing: anyone can ask to see the comparables and judge whether the adjustment makes sense.

An appraiser who instead says "I think it's worth $300,000, it looks like a $300,000 house" isn't necessarily lying —they might have good intuition—, but nobody can audit that number, and "it looks like" intuition tends to inflate when the appraiser also wants a quick sale. expectedLift works the same way: you can estimate it with comparables (similar tests, industry benchmarks, other companies' cases) or you can say "I feel like it's going to go up a lot". The first option is auditable. The second is an opinion with decimals.

Worked example: the hunch against the benchmark

Let's go back to "faster mobile checkout". An engineer on the team, without comparing against anything, feels that speeding up checkout should move the needle "quite a bit" — they propose expectedLift: 0.30. Someone else on the team remembers something: last quarter, Mercado ran a similar experiment speeding up the search page —from 2.8 to 1.0 seconds—, and that test, measured with a real A/B, raised search conversion by a relative 15%. It isn't the exact same change, but it's the closest comparable that exists. Let's size both versions to see the difference.

function opportunitySize({ reachableUsers, baselineRate, expectedLift }) {
  const baselineOutcome = Math.round(reachableUsers * baselineRate);
  const newRate = baselineRate * (1 + expectedLift);
  const newOutcome = Math.round(reachableUsers * newRate);
  const extraOutcome = newOutcome - baselineOutcome;
  return { baselineOutcome, newRate, newOutcome, extraOutcome };
}

function toGMV(extraOutcome, avgOrderValue) {
  return Math.round(extraOutcome * avgOrderValue);
}

const reachableUsers = 100000;
const baselineRate = 0.04;
const avgOrderValue = 45;

// The engineer, without comparing against anything, "feels" that faster
// checkout should raise conversion by a good stretch.
const gutFeelLift = { reachableUsers, baselineRate, expectedLift: 0.30 };

// Mercado's team already ran a similar test last quarter (speeding up the
// search page) and that test raised conversion by a relative 15%. It is
// the closest benchmark that exists, not a promise.
const benchmarkedLift = { reachableUsers, baselineRate, expectedLift: 0.15 };

console.log('=== Where expectedLift comes from: intuition vs benchmark ===\n');

const gutResult = opportunitySize(gutFeelLift);
const gutGMV = toGMV(gutResult.extraOutcome, avgOrderValue);
console.log('-- "by feel" (expectedLift = 0.30, no backing) --');
console.log('extra conversions expected:', gutResult.extraOutcome);
console.log('extra GMV expected:', gutGMV);

const benchResult = opportunitySize(benchmarkedLift);
const benchGMV = toGMV(benchResult.extraOutcome, avgOrderValue);
console.log('\n-- internal benchmark (expectedLift = 0.15, last quarter\'s search test) --');
console.log('extra conversions expected:', benchResult.extraOutcome);
console.log('extra GMV expected:', benchGMV);

console.log('\ndifference between defending the bet "by feel" and with a benchmark:', gutGMV - benchGMV, 'USD extra, with no backing');

What to expect. When you run the file with Node, the output is exactly this:

=== Where expectedLift comes from: intuition vs benchmark ===

-- "by feel" (expectedLift = 0.30, no backing) --
extra conversions expected: 1200
extra GMV expected: 54000

-- internal benchmark (expectedLift = 0.15, last quarter's search test) --
extra conversions expected: 600
extra GMV expected: 27000

difference between defending the bet "by feel" and with a benchmark: 27000 USD extra, with no backing

The hunch doubles the benchmark's result: 54,000 against 27,000 dollars. Neither number is "the correct one" in an absolute sense —both are assumptions about the future—, but only one of the two can be defended in front of someone who asks "why that number?". "Because a similar test, with real data, gave 15%" is an answer. "Because it feels like a big improvement" isn't — and the difference between the two answers, in this case, is worth $27,000 of unsupported expectation.

Going deeper: the source hierarchy for expectedLift

Not every expectedLift source is worth the same. From most to least reliable:

1. Your own similar test, already run. Like the example's benchmark: Mercado already tested something comparable and measured the result with a real experiment. It's the strongest source because it comes from the same product, the same user base, the same context.
2. An industry benchmark, for the same type of change. Public studies on load speed and conversion, or published cases from similar companies that made a similar change. Weaker than your own test —the context isn't identical—, but better than nothing: at least there's a citable source, and you can adjust the number downward if the context differs a lot.
3. A small, cheap experiment, run now, just to estimate the lift. If there's neither your own test nor an applicable benchmark, sometimes a small experiment is worth it —a prototype, a test with a fraction of traffic— just to have a real number before committing to the full sizing. This starts crossing into "validate with evidence" territory, which is the whole subject of product-discovery-and-prototyping-guide — here it's enough to know it's an option when the two above don't exist.
4. The expert judgment of someone with a track record on similar decisions. The last resort, not the first: the opinion of someone who's already watched enough similar bets play out and has intuition calibrated by real results, not by optimism. It's still weaker than data, and should be explicitly declared as such ("this is expert judgment, not a benchmark").

The order matters, and the most common mistake is starting at 4 without trying 1 or 2 first. You won't always have a similar test of your own —that's fine—, but the right habit is to ask "does anything exist, in any of the three sources above, before I fall back on my intuition?" before writing a number. And when there really is no other option but expert judgment, lesson 6 gives you the tool for not faking a precision you don't have: a range, not a single number.

Common mistakes

An optimistic expectedLift with no basis at all. What happens: someone proposes a number that "sounds reasonable" for the size of the bet they want to justify, without looking for any comparable first. Why it happens: whoever proposes a bet is usually convinced it's going to work well —that's part of why they proposed it—, and that conviction leaks straight into the number, with no external filter. How to spot it: if you can't name your expectedLift's source in one sentence ("it comes from X"), it's unsupported intuition. How to fix it: before writing the number, go through this lesson's source hierarchy — start by looking for your own test or a benchmark, and only fall back on intuition if there truly is nothing else, declaring it explicitly.

Copying the lift from another company's case study without adjusting for context. What happens: an article is found saying "company X raised its conversion 40% by speeding up checkout" and that 40% gets used directly, without asking whether X's context resembles Mercado's. Why it happens: a published number feels objective because it comes from "outside", even though the context —company size, starting point, product type— can be completely different. How to spot it: if your only justification is "I read that another company achieved X%", with no adjustment for context differences, you're copying a comparable without the adjustment part an appraiser always does. How to fix it: use the external number as a plausible upper bound, not as your direct estimate — and adjust it downward according to how different the context is (lesson 6 gives you the language to express that uncertainty with a range).

Working backward from the result you want. What happens: the team has already decided it wants to build the bet, and adjusts expectedLift until the extraGMV "looks big enough" to justify the quarter invested — instead of estimating the lift first and accepting whatever result comes out. Why it happens: it's easier to adjust an input than to question a decision already made. How to spot it: if you changed the expectedLift more than once and each change was "upward", after seeing the result "wasn't enough", that's the exact pattern. How to fix it: write the expectedLift and its source before calculating the extraGMV, not after — and if the result is smaller than expected, that's real information about the bet, not a mistake to fix by adjusting the input.

Exercises

Exercise 1 — Rank the sources. For the "add seller reviews" bet, the team has these four possible sources for expectedLift. Rank them from most to least reliable, per this lesson's hierarchy:

• (a) A product manager's feeling that "reviews always help"
• (b) An A/B test Mercado ran a year ago testing trust badges at checkout (not reviews exactly, but related)
• (c) A public study from a comparable e-commerce company that raised its conversion 12% by adding seller reviews
• (d) A small experiment: showing hand-curated reviews to 200 buyers this week and measuring whether they buy more than a control group

See solution

Order from most to least reliable: (d) → (b) → (c) → (a). (d) is your own test, current, specific to this bet —the strongest source, even if small—. (b) is your own test but of a related change, not identical —strong, but needs adjustment—. (c) is an external benchmark, from another company —useful as a reference bound, but the context can differ a lot—. (a) is pure intuition, with no data behind it —the weakest source, the last resort—.

Exercise 2 — Spot working backward. A coworker says: "I proposed expectedLift: 0.10 for 'recommendations at checkout', but the extraGMV came out much smaller than the team expected, so I raised it to 0.25 to make the number look better in the presentation." Which of this lesson's mistakes is this, and what should you say back?

See solution

It's the third mistake: working backward from the result you want. expectedLift got adjusted after seeing the result, not before, and the reason for the adjustment was "to make it look better", not new evidence. The right response: ask them to go back to the original 0.10 (or whatever source justified it) and present that number, even if the extraGMV is smaller than the team expected — an honest sizing that gives a modest number is still more useful than an inflated one that overpromises and doesn't deliver.

Exercise 3 — Justify your own expectedLift. For the bet "improve Mercado's internal search" (adding spell correction and synonyms to searches), write which source from this lesson's hierarchy you'd use to estimate expectedLift, and why that one and not another, given that Mercado never changed the search engine before.

See solution

A reasonable answer: since Mercado never touched the search engine before, source 1 (a similar test of your own) doesn't exist. Source 2 (industry benchmark) is plausible —there are published articles about how much search conversion improves when typo tolerance is added—, but Mercado's context may differ. The most honest option would be combining source 2 as a starting point (adjusted downward, out of caution) with source 3: running a small experiment —turning on spell correction for just 5% of search traffic for two weeks— before committing to the quarter's full sizing. Using source 4 (intuition) directly would be skipping both stronger options unnecessarily.

Summary and next step

expectedLift is the only one of the three assumptions that can't be measured today, because the change doesn't exist yet — and that's why it's the one most easily inflated, whether by genuine optimism or by working backward from a desired result. The source hierarchy —your own test, industry benchmark, small experiment, expert judgment, in that order— is the discipline that replaces "it feels like a big improvement" with "this comes from X, and here's the adjustment I made". In the worked example, the difference between estimating by eye and estimating with a benchmark was worth $27,000 of unsupported expectation.

Before moving on you should be able to: name the hierarchy's four sources, in order of reliability; and for any expectedLift you write, complete the sentence "this comes from..." without using "I feel" or "I think" as the only justification.

Lesson 6 takes all three complete assumptions —including an expectedLift that, even with the best possible source, remains uncertain— and gives them an honest way to express that uncertainty: a pessimistic/expected/optimistic range, instead of a single number that fakes a precision that doesn't exist.

Resources

• Itamar Gilad, "Product Discovery With ICE and The Confidence Meter" — itamargilad.com/the-tool-that-will-help-you-choose-better-product-ideas. Gilad's hierarchy of evidence types —from personal opinion to market evidence— is the same idea as this lesson's source hierarchy, applied to a whole bet's confidence, not just expectedLift. In English.
• Reforge, "Estimate business value for new features" — reforge.com/guides/estimate-business-value. On how product teams look for comparables and benchmarks before committing to an expected impact. In English, requires a free account.
• Lenny's Newsletter, "Introducing DRICE: A modern prioritization framework" — lennysnewsletter.com/p/introducing-drice-a-modern-prioritization. Explains why RICE's impact needs more rigor than a 1-to-5 scale, and how real teams look for evidence before estimating it — the same argument as this lesson, for all of RICE, not just expectedLift. In English.