Retention as the best proxy for real value

Overview

You already know how to calculate a curve (lesson 3), read its plateau (lesson 4), avoid the average's deception (lesson 5), and read a full table by row and column (lesson 6). One last piece is missing before the module can close: precisely defining what "day-N retention" means — because, surprisingly, that very common phrase has at least two different, non-interchangeable definitions, and using them without clarifying which one produces numbers that don't even describe the same phenomenon.

With that definition now precise, this lesson closes the module with the central argument: of every metric a product team can measure, retention is, probably, the one that best approximates the real value a product delivers. A high conversion rate tells you someone completed an action today; high retention tells you something much harder to fake: that same person, after having tried the product and having every reason in the world not to come back, decided to come back anyway.

How this connects to the module. This lesson closes the module's full conceptual arc — cohort (lesson 2), curve (lesson 3), plateau (lesson 4), why the aggregate misleads (lesson 5), full table (lesson 6) — with the underlying question that justifies having invested six lessons in this: why does it matter so much? Lesson 8's mini-project applies the whole framework to real Mercado data.

An analogy: the exact appointment versus the open-ended one

Imagine you ask a friend: "let me know on day 7." That sentence, with no more context, allows two completely different readings. It could mean "let me know exactly on day 7, not a day before or after" —a precise appointment, and if they let you know on day 9, they technically didn't keep it—. Or it could mean "let me know at some point starting on day 7 onward" —an open window, and if they let you know on day 9, they kept it perfectly well—. Both interpretations are reasonable, both use the same words ("let me know on day 7"), and yet they describe completely different commitments: one is much easier to keep than the other.

"D7 retention" has exactly this same ambiguity problem. It can mean "the user came back exactly on day 7 after entering" —the precise appointment—, or it can mean "the user came back on day 7 or any day after" —the open window—. A team that reports "D7 = 50%" using the first definition and another team that reports "D7 = 80%" using the second could, in reality, be describing exactly the same cohort with exactly the same behavior — the difference lies only in which question was asked, not in how well the product retains.

Worked example: the same raw data, two D7 definitions

The activity log for 10 users from the July 6 cohort, with the exact days (since their entry, day 0) each one opened Mercado. Let's calculate D7 retention two different ways over exactly the same data:

// The raw activity log for 10 users from the July 6 cohort: each array holds the
// DAYS (since their entry, day 0) that user opened the app. We observe through day 21.
const users = [
  { id: 'u01', activeDays: [0, 1, 2, 3, 7, 15] },
  { id: 'u02', activeDays: [0, 1, 9, 10] },
  { id: 'u03', activeDays: [0, 5, 6, 7] },
  { id: 'u04', activeDays: [0] },
  { id: 'u05', activeDays: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] },
  { id: 'u06', activeDays: [0, 3, 12] },
  { id: 'u07', activeDays: [0, 1, 7, 14, 21] },
  { id: 'u08', activeDays: [0, 2, 4, 6, 8, 10, 12] },
  { id: 'u09', activeDays: [0] },
  { id: 'u10', activeDays: [0, 7] },
];

// EXACT D7: active precisely on day 7, no earlier, no later.
function exactDayRetention(users, day) {
  const returned = users.filter((u) => u.activeDays.includes(day));
  return { returned: returned.map((u) => u.id), rate: (returned.length / users.length) * 100 };
}

// UNBOUNDED D7 ("on or after"): active on day 7 or ANY day after, with no ceiling.
function unboundedRetention(users, day) {
  const returned = users.filter((u) => u.activeDays.some((d) => d >= day));
  return { returned: returned.map((u) => u.id), rate: (returned.length / users.length) * 100 };
}

console.log('=== Exact D7 vs unbounded D7, same raw data ===\n');
const exact = exactDayRetention(users, 7);
const unbounded = unboundedRetention(users, 7);

console.log('EXACT D7 (active precisely on day 7):');
console.log('  users: ' + exact.returned.join(', '));
console.log('  exact D7 = ' + exact.returned.length + '/10 = ' + exact.rate.toFixed(1) + '%\n');

console.log('UNBOUNDED D7 (active on day 7 or after, no cap):');
console.log('  users: ' + unbounded.returned.join(', '));
console.log('  unbounded D7 = ' + unbounded.returned.length + '/10 = ' + unbounded.rate.toFixed(1) + '%\n');

console.log('Same raw data, same cohort, same "D7" in the name -- two different');
console.log('numbers depending on the definition. Reporting "D7 = ' + exact.rate.toFixed(0) +
  '%" without saying which one was used leaves the reader guessing.');

What to expect. When you run the file with Node, the output is exactly this:

=== Exact D7 vs unbounded D7, same raw data ===

EXACT D7 (active precisely on day 7):
  users: u01, u03, u05, u07, u10
  exact D7 = 5/10 = 50.0%

UNBOUNDED D7 (active on day 7 or after, no cap):
  users: u01, u02, u03, u05, u06, u07, u08, u10
  unbounded D7 = 8/10 = 80.0%

Same raw data, same cohort, same "D7" in the name -- two different
numbers depending on the definition. Reporting "D7 = 50%" without saying which one was used leaves the reader guessing.

A 30-percentage-point gap —50% versus 80%— over exactly the same 10 users, with neither number "calculated wrong". u02 (active on days 0, 1, 9, 10) never opened the app precisely on day 7, so they don't count for exact D7 — but they did come back later (days 9 and 10), so they do count for unbounded D7. The same thing happens with u06 and u08. Mixpanel's own documentation, in fact, prefers the "unbounded" definition (which it calls "On or After") as the default option, precisely because —as its own documentation says— "most products don't require the user to come back in every exact time unit" to have received real value. Neither definition is "the correct one" in the abstract; what's a real mistake is not saying which one is being used.

Why retention is the best proxy for real value

With the definition now precise, the underlying question closes: why give retention so much weight, compared to conversion or any other funnel metric? The reason has to do with how hard each one is to fake.

A high conversion rate can be achieved with tricks that don't reflect real value: an aggressive first-purchase discount, a design that pushes the user toward checkout, a limited-time offer that manufactures artificial urgency. All of those tricks can raise checkoutConversionRate without the product having improved at all. Retention is much harder to fake the same way: for someone to come back next week, with no discount or urgency pushing them, they have to have received some real value the first time — they found what they were looking for, they trusted the process, or it was simply useful to them. Casey Winters and Lenny Rachitsky, after synthesizing the criteria of 20 growth experts from different industries, arrived at concrete benchmarks for what counts as "good" and "excellent" retention depending on the type of business — evidence that the entire industry treats retention, not conversion, as the yardstick for whether a product truly works.

This doesn't mean retention is immune to manipulation — a flood of push notifications can raise the number without the user receiving more real value, exactly the kind of "optimizing the metric instead of the business" module 4 develops with guardrails—. But, compared to a single-session conversion rate, retention demands that value hold up over time, and that makes it, in the vast majority of cases, the most honest signal a product team has available.

Common mistakes

Poorly defined N-day retention: exactly day N, or up to day N? What happens: two reports cite "D7 = X%" with different numbers for the same cohort, and nobody in the room can explain why they don't match. Why it happens: the phrase "D7 retention" sounds self-explanatory, and rarely does anyone stop to ask which of the two definitions —exact or unbounded— was used to calculate it. How to spot it: exactly today's example's scenario — the same raw data produces 50% with one definition and 80% with the other, and both numbers "sound" reasonable on their own. How to fix it: always state the exact definition alongside the number —"D7 unbounded: 80%"—, never just "D7: 80%". It's the same discipline this guide's DESIGN demands for any statistical approximation: name the method, not just the result.

Using D1 as the main health metric when the product has a weekly or monthly cycle. What happens: a Mercado team —where buying is, typically, a weekly or even monthly event, not a daily one— obsesses over D1 retention (did they come back the next day?), a time horizon that makes sense for a messaging or social media app, but not for an occasional-purchase marketplace. Why it happens: D1 is the number that can be measured fastest —you only need to wait a day—, and that speed makes it tempting as a main metric, even though it doesn't fit the product's real usage cycle. How to spot it: the retention window reported as "the" health metric doesn't match the natural frequency with which a typical user would use the product. How to fix it: choose the retention window (D1, D7, D30) based on the product's natural cycle — for Mercado, something like D30 or even D90 makes far more sense than D1, because nobody buys on a marketplace every single day.

Treating retention as an end in itself, without connecting it to real value. What happens: a team aggressively optimizes to raise the retention number —more notifications, more reminders, more re-engagement hooks— without asking whether those tactics are generating real value or just interrupting the user until they open the app one more time. Why it happens: retention, like any metric, can be "gamed" (artificially inflated) if optimized carelessly, exactly the same risk any metric in this module runs. How to spot it: retention goes up, but other signals —complaints, uninstalls, real time spent using the product once opened— worsen at the same time. How to fix it: remember why retention matters —it's hard to fake when it comes from real value—, and watch that the tactics used to improve it don't turn it, itself, into an easy metric to fake. Module 4 formalizes this protection with guardrails.

Exercises

Exercise 1 — Calculate D14 with both definitions. Using the same 10 users from today's example, calculate by hand exact D14 retention (active precisely on day 14) and unbounded D14 (active on day 14 or after).

See solution

Exact D14: only u07 has exactly day 14 in their list ([0, 1, 7, 14, 21]) — 1/10 = 10.0%. Unbounded D14: users with any day ≥ 14 are u07 (14, 21) — checking the rest: u01 reaches day 15 (which is ≥ 14) so it also counts. Each user's activeDays maximum: u01 up to 15 (counts), u02 up to 10 (no), u03 up to 7 (no), u04 up to 0 (no), u05 up to 10 (no), u06 up to 12 (no), u07 up to 21 (counts), u08 up to 12 (no), u09 up to 0 (no), u10 up to 7 (no). Unbounded D14 = 2/10 = 20.0%. The gap between the two definitions (10% vs 20%) holds, though smaller than at D7, because by day 14 fewer users overall are still active.

Exercise 2 — Choose the right window. For each product, decide whether D1, D7, or D30 is the most sensible retention window as the main metric, and justify in one sentence: (a) a daily meditation app, (b) Mercado (occasional-purchase marketplace), (c) annual tax-filing software.

See solution

(a) D1 (or even continuous daily retention) — a daily meditation app expects daily use; if someone doesn't come back the next day, that's already an early warning sign. (b) D30 — Mercado is an occasional-purchase marketplace, not something used daily; measuring D1 would unfairly penalize perfectly healthy users who simply don't need to buy every day. (c) None of the three makes sense as a main metric — annual tax-filing software has a twelve-month usage cycle; D30 would still be too short, and the relevant retention metric would be something like "D365" or, more precisely, "did they come back next tax season?". The general principle: the retention window should reflect the product's natural usage cycle, not a standard industry number.

Exercise 3 — Defend retention against a skeptic. A colleague says: "I'd rather optimize for conversion — it's easier to measure, and it goes up faster with less effort than retention." In 3-4 sentences, using this lesson's argument, explain why that can be a medium-term trap.

See solution

A reasonable answer: "It's true conversion rises faster and with less effort —an aggressive discount or a more insistent design can move that number in a week—, but that same ease is the warning sign: if it's easy to move with tricks that don't change the product, it isn't measuring whether the product generates real value. Retention is slower and harder to move precisely because it requires the user, with no artificial incentive pushing them, to decide to come back on their own. Optimizing only for conversion can produce a business that attracts a lot of people once and loses them forever —lesson 4's leaky bucket—, while a real retention improvement builds a user base that sustains itself without needing constant reinvestment in acquisition."

Summary and next step

In this lesson you resolved a source of confusion that seemed trivial and wasn't: "D7 retention" can mean "exactly day 7" or "day 7 or after", and both definitions, over the same Mercado data, gave 50% and 80% respectively. With that precision resolved, you closed the module's central argument: retention is hard to fake with single-session tricks, and that's why it's, of every metric in this module, the one that best approximates the real value Mercado delivers — with the caveat of not turning it, itself, into an easy metric to game.

Before moving on you should be able to: tell exact N-day retention apart from unbounded and calculate both over raw data, choose the right retention window based on a product's usage cycle, and explain why retention is harder to fake than conversion.

With the six content lessons complete, lesson 8 —the mini-project— puts you in front of the real cohort table from Mercado's recommendations launch: you're going to calculate the curve, find the plateau, and descriptively compare the cohort that saw recommendations against the one that didn't.

Resources

• Casey Winters and Lenny Rachitsky, "What Is Good Retention: An Exhaustive Benchmark Study" — lennysnewsletter.com/p/what-is-good-retention-issue-29. The synthesis of 20 growth experts into concrete benchmarks for "good" and "excellent" retention by business type — the evidence behind this lesson's central argument. In English.
• Andrew Chen, "The Power User Curve" — andrewchen.com/power-user-curve. Extends the idea of retention as real value toward the most engaged users: the "smile" shape that distinguishes a product with a solid core of genuinely hooked users. In English.
• Mixpanel, "Retention: Measure engagement over time" (official documentation) — docs.mixpanel.com/docs/reports/retention. The technical source for the "On" (exact) vs. "On or After" (unbounded) distinction this lesson develops with the Mercado example. In English.