Final project: validate `recommendations`, end to end

Overview

This is the close of this module's eight lessons, and the close of the eight modules you've been building since this guide's lesson 1. The assignment is the same one that's accompanied the whole guide: Mercado's product team has recommendations's risky assumption —"users are going to buy more if they see personalized recommendations"— with risk: 0.6, impact: 5, and RICE's confidence capped at 0.3 since product-thinking-for-engineers-guide left it untested. Your job is to deliver, with executed logic and not an opinion, the complete discovery pipeline: the one-page plan, the findings running it produced, and a final decision —persevere, pivot, or kill— you can defend, step by step, to anyone who asks "and with what evidence?".

How this connects to the module. The previous seven lessons assembled, one layer at a time, this project's pieces: the falsifiable hypothesis (L2), the test chosen with pickTest() (L3), the verified interview script (L4), the fake door's signal (L5), the bias-audited evidence (L6), and the synthesis with the final decision (L7). This project adds no new tool: it brings the six steps together in a single Node run, start to finish, over the complete case — exactly what a real discovery team does every time it closes a round, not once per lesson.

Closing the circle: from vague diagnosis to evidence-backed verdict

In this module's introduction, the doctor who had ordered several separate tests finally sat down, with all the results on the desk, to give the diagnosis. This project is that final consultation. There's no more test left to order —all six have already run—; what remains is putting them all together, in the same file, and signing the complete verdict: this is the evidence, this is the direction it points in, and this is what we do now with recommendations.

The reference solution, verified

We're going to run the complete pipeline once more, end to end, and verify it step by step. (The transfer exercises at the end ask you to apply the same method to a case this guide never saw.)

Part 1 — The discovery plan, on one page

Before running anything, this is the document Mercado's team would bring to the week's planning meeting — the summary of the decisions this module's lessons 2 and 3 already verified:

DISCOVERY PLAN -- recommendations (Mercado)
════════════════════════════════════════════════════════════════════
Risky assumption:      "Users will buy more if they see
                        personalized recommendations"
Source:                 product-thinking-for-engineers-guide, M6
                        (risk=0.6, impact=5, score=3.0)

Falsifiable hypothesis (isFalsifiable -> true):
  we believe:           Users will buy more if they see
                        personalized recommendations
  we will know it if:    >= 8% of users who see a recommendation
                        add it to their cart
  we will be wrong if:   < 8%, even after 2 weeks of exposure

Main test (pickTest, byValue):
  fake_door_checkout (cost=3, strength=6, value=2.00)
  -- discarded: behavioral_interview (1.50), clickable_prototype
     (1.40), build_the_engine (0.23)

Complement:             7-question script (classifyQuestion,
                        7/7 good), run BEFORE the fake door

Planned sample:         10 people -- 3 power_user / 4 occasional_
                        buyer / 3 new_user (sampleBias.biased=false)

Fake door threshold:    5% click-through, agreed on BEFORE measuring
decide() thresholds:    persevere >= 0.6 | pivot >= 0.3 | kill if
                        neither is reached -- agreed on BEFORE
                        calculating confidence

Timeline:    days 1-3   interviews (8 buyers)
             days 4-10  fake door runs for 1 full week
             day 11     synthesis, confidence update,
                        decision
════════════════════════════════════════════════════════════════════

Notice something this document makes explicit, that an improvised plan almost never does: every decision threshold is written before the timeline row that produces the data that threshold is going to evaluate. The hypothesis's 8%, the fake door's 5%, and decide()'s 0.6/0.3 — all three got fixed on this page, before day 1, not adjusted afterward with the result already in hand.

🟡 A gap worth reconciling before moving on. The hypothesis's wrongIf talks about adding to cart (< 8%); the fake door measures click-through (5%) on the "See my recommendations" button. They're not the same metric, and the plan shouldn't treat them as if they were: the fake door's click is an early proxy for that condition —it measures whether someone is interested enough to discover the recommendation, not whether they truly add it to their cart—. It's the right metric for a cheap, fast signal like this one, but it doesn't close the full wrongIf condition on its own; measuring the real add-to-cart, with the rigor that condition demands, is product-metrics-and-experimentation-guide's job once the team ships the real solution.

Part 2 — The end-to-end run: the 7 steps, chained together

Now we run, in a single file, the six models reused unchanged from modules 2, 4, 5, and 6, plus module 7's three models:

// PROJECT: the complete discovery pipeline over "recommendations," end to
// end. Chains isFalsifiable + pickTest (M4), classifyQuestion (M2),
// fakeDoorSignal (M5), sampleBias + rankByStrength (M6), and synthesize +
// updateConfidence + decide (M7, CANONICAL functions, with no changes at
// all).

// ===== M4 =====
function isFalsifiable(hypothesis) {
  const hasBelief = typeof hypothesis.believe === 'string' && hypothesis.believe.trim().length > 0;
  const hasFailureCondition = typeof hypothesis.wrongIf === 'string' && hypothesis.wrongIf.trim().length > 0;
  if (!hasBelief) return { ...hypothesis, falsifiable: false, reason: 'does not declare a clear belief (believe) -- there is nothing to test' };
  if (!hasFailureCondition) return { ...hypothesis, falsifiable: false, reason: 'does not declare a failure condition (wrongIf) -- cannot be refuted' };
  return { ...hypothesis, falsifiable: true, reason: 'has both a belief and a failure condition, both observable' };
}
function pickTest(assumption, candidateTests, options = {}) {
  const byValue = options.byValue || false;
  const falsifiable = candidateTests.filter((t) => t.canFalsify);
  if (falsifiable.length === 0) return { assumption, chosen: null, discarded: candidateTests, reason: 'no candidate can refute the assumption' };
  const scored = falsifiable.map((t) => ({ ...t, value: t.strength / t.cost }));
  const chosen = byValue ? scored.reduce((best, t) => (t.value > best.value ? t : best)) : scored.reduce((best, t) => (t.cost < best.cost ? t : best));
  const discarded = candidateTests.filter((t) => t.type !== chosen.type);
  return { assumption, chosen, discarded, mode: byValue ? 'best strength/cost' : 'lowest cost' };
}
// ===== M2 =====
function classifyQuestion(question) {
  const q = question.toLowerCase();
  const leadingSignals = ["wouldn't it be true that", "don't you think", "don't you find", "isn't it true that", "isn't it correct that"];
  const hypotheticalSignals = ['would you use', 'would you buy', 'would you like', 'would you pay', 'would you install', 'would you prefer'];
  const goodSignals = ['what did you do', 'when was the last time', 'what was the last time', 'how did you solve', 'what did you use', 'do you remember the last time', 'how did you do'];
  if (leadingSignals.some((s) => q.includes(s))) return { question, label: 'leading', reason: 'puts the expected answer inside the question' };
  if (hypotheticalSignals.some((s) => q.includes(s))) return { question, label: 'hypothetical', reason: 'asks for a prediction about the future, not a fact' };
  if (goodSignals.some((s) => q.includes(s))) return { question, label: 'good', reason: 'asks for a specific fact about past behavior' };
  return { question, label: 'hypothetical', reason: 'no evidence of past behavior' };
}
// ===== M5 =====
function fakeDoorSignal({ impressions, clicks, threshold }) {
  const clickRate = clicks / impressions;
  return { impressions, clicks, clickRatePercent: Math.round(clickRate * 1000) / 10, thresholdPercent: Math.round(threshold * 1000) / 10, passesThreshold: clickRate >= threshold };
}
// ===== M6 =====
function sampleBias({ segments }) {
  const total = segments.reduce((sum, s) => sum + s.count, 0);
  const withShare = segments.map((s) => ({ ...s, share: +((s.count / total) * 100).toFixed(1) }));
  const dominant = withShare.reduce((max, s) => (s.share > max.share ? s : max));
  const missing = withShare.filter((s) => s.count === 0);
  return { total, segments: withShare, dominant, missing, biased: dominant.share >= 70 || missing.length > 0 };
}
const STRENGTH_BY_TYPE = { observed_behavior: 4, reported_behavior: 3, stated_opinion: 2, hypothetical: 1 };
function rankByStrength(evidences) {
  return evidences.map((e) => ({ ...e, strength: STRENGTH_BY_TYPE[e.type] })).sort((a, b) => b.strength - a.strength);
}
// ===== M7 (CANONICAL functions, identical to module 7 -- no changes at all) =====
function synthesize(findings, options = {}) {
  const signalMinUsers = options.signalMinUsers || 2;
  const byProblem = {};
  findings.forEach((f) => {
    if (!byProblem[f.problem]) byProblem[f.problem] = [];
    byProblem[f.problem].push(f);
  });
  return Object.keys(byProblem)
    .map((problem) => {
      const items = byProblem[problem];
      const distinctUsersUnprompted = new Set(
        items.filter((f) => f.unprompted).map((f) => f.user)
      ).size;
      return {
        problem,
        mentions: items.length,
        distinctUsersUnprompted,
        classification: distinctUsersUnprompted >= signalMinUsers ? 'signal' : 'noise',
      };
    })
    .sort((a, b) => b.distinctUsersUnprompted - a.distinctUsersUnprompted);
}
function updateConfidence(prior, evidence) {
  const posteriorRaw = evidence.validates
    ? prior + (1 - prior) * evidence.strength
    : prior - prior * evidence.strength;
  const posterior = Math.round(posteriorRaw * 100) / 100;
  return {
    prior,
    posterior,
    delta: Math.round((posterior - prior) * 100) / 100,
    direction: evidence.validates ? 'up' : 'down',
  };
}
function decide(signalStrength, threshold) {
  if (signalStrength >= threshold.persevere) return { signalStrength, action: 'persevere', reason: 'the accumulated evidence clears the threshold to keep investing as-is' };
  if (signalStrength >= threshold.pivot) return { signalStrength, action: 'pivot', reason: 'there is real signal but not enough to continue unchanged' };
  return { signalStrength, action: 'kill', reason: 'the accumulated evidence is not even enough to pivot' };
}

// =========================================================
console.log('=== recommendationsBet, as it arrives from product-thinking-for-engineers ===\n');
const recommendationsBet = { assumption: 'Users will buy more if they see personalized recommendations', risk: 0.6, impact: 5, tested: false, confidenceCeiling: 0.3 };
console.log('assumption: "' + recommendationsBet.assumption + '"');
console.log('risk=' + recommendationsBet.risk + '  impact=' + recommendationsBet.impact + '  score=' + (recommendationsBet.risk * recommendationsBet.impact));
console.log('tested=' + recommendationsBet.tested + '  confidenceCeiling=' + recommendationsBet.confidenceCeiling);

console.log('\n=== Step 1/7: the falsifiable hypothesis (M4 isFalsifiable) ===\n');
const recommendationsHypothesis = {
  believe: 'Users will buy more if they see personalized recommendations',
  weWillKnowIf: 'at least 8% of users who see a recommendation add it to their cart',
  wrongIf: 'fewer than 8% of users who see a recommendation add it to their cart, even after two weeks of exposure',
};
console.log('falsifiable: ' + isFalsifiable(recommendationsHypothesis).falsifiable);

console.log('\n=== Step 2/7: pickTest() chooses the cheapest test that can refute it (M4) ===\n');
const candidateTests = [
  { type: 'behavioral_interview', cost: 2, canFalsify: true, strength: 3 },
  { type: 'fake_door_checkout', cost: 3, canFalsify: true, strength: 6 },
  { type: 'clickable_prototype', cost: 5, canFalsify: true, strength: 7 },
  { type: 'build_the_engine', cost: 40, canFalsify: true, strength: 9 },
];
const testDecision = pickTest(recommendationsHypothesis.believe, candidateTests, { byValue: true });
console.log('chosen: ' + testDecision.chosen.type + ' (cost=' + testDecision.chosen.cost + ', value=' + testDecision.chosen.value.toFixed(2) + ')');

console.log('\n=== Step 3/7: the interview script, verified (M2 classifyQuestion) ===\n');
const interviewScript = [
  'When was the last time you bought something on Mercado without searching for it directly, because you saw it somewhere else?',
  'When was the last time you bought something based on an automatic suggestion from another store or app (like Amazon, Netflix, or Spotify)?',
  "What did you do the last time Mercado didn't show you something you were looking for and you ended up buying it somewhere else?",
  'What did you do the last time you were torn between two similar products on Mercado?',
  'What did you use the last time you needed help deciding between similar products on Mercado?',
  "Do you remember the last time a seller or a review made you buy something you hadn't planned to? Tell me what happened.",
  'What was the last time a recommendation from a friend, a review, or a store saved you time looking for something?',
];
const scriptCheck = interviewScript.map(classifyQuestion);
const goodCount = scriptCheck.filter((r) => r.label === 'good').length;
console.log(goodCount + ' of ' + scriptCheck.length + ' questions ready (good). Script run with 8 buyers.');

console.log('\n=== Step 4/7: the checkout fake door signal (M5 fakeDoorSignal) ===\n');
const weekResult = fakeDoorSignal({ impressions: 4000, clicks: 260, threshold: 0.05 });
console.log(weekResult.clicks + ' clicks / ' + weekResult.impressions + ' impressions = ' + weekResult.clickRatePercent + '% (threshold: ' + weekResult.thresholdPercent + '%) -> clears: ' + weekResult.passesThreshold);

console.log('\n=== Step 5/7: weigh the evidence without bias (M6 sampleBias + rankByStrength) ===\n');
const correctedSample = { segments: [{ segment: 'power_user', count: 3 }, { segment: 'occasional_buyer', count: 4 }, { segment: 'new_user', count: 3 }] };
const sampleCheck = sampleBias(correctedSample);
console.log('sample: ' + sampleCheck.total + ' people, dominant ' + sampleCheck.dominant.segment + ' (' + sampleCheck.dominant.share + '%), biased: ' + sampleCheck.biased);
const correctedEvidence = [
  { claim: 'Clicked "recommended for you" on the checkout fake door', type: 'observed_behavior' },
  { claim: 'Ignored the recommendations section, zero clicks across two visits', type: 'observed_behavior' },
  { claim: 'Added a recommended product to the cart on the fake door', type: 'observed_behavior' },
  { claim: 'Reported that the last time they bought from another store, they followed a recommendation', type: 'reported_behavior' },
  { claim: 'Said they would love to see personalized recommendations', type: 'hypothetical' },
];
const rankedEvidence = rankByStrength(correctedEvidence);
rankedEvidence.forEach((e) => console.log('  strength=' + e.strength + ' [' + e.type.padEnd(18) + '] ' + e.claim));

console.log('\n=== Step 6/7: synthesize the 8 interviews (M7 synthesize) ===\n');
const findings = [
  { user: 'buyer_01', problem: "I don't discover products I'd like without searching for the exact name", unprompted: true },
  { user: 'buyer_01', problem: "I don't discover products I'd like without searching for the exact name", unprompted: true },
  { user: 'buyer_02', problem: "I don't discover products I'd like without searching for the exact name", unprompted: true },
  { user: 'buyer_02', problem: 'I forget about my cart and never go back to finish it', unprompted: true },
  { user: 'buyer_03', problem: "I don't trust new sellers without reviews", unprompted: true },
  { user: 'buyer_04', problem: "I don't discover products I'd like without searching for the exact name", unprompted: false },
  { user: 'buyer_05', problem: 'I forget about my cart and never go back to finish it', unprompted: true },
  { user: 'buyer_06', problem: "I don't discover products I'd like without searching for the exact name", unprompted: true },
  { user: 'buyer_07', problem: 'I forget about my cart and never go back to finish it', unprompted: true },
  { user: 'buyer_08', problem: 'the app crashes on its own when the phone has low memory', unprompted: true },
];
const synthesis = synthesize(findings);
synthesis.forEach((r) => {
  console.log('[' + r.classification.toUpperCase().padEnd(6) + '] "' + r.problem + '"  (distinct unprompted users: ' + r.distinctUsersUnprompted + ')');
});

console.log('\n=== Step 7/7: updateConfidence() with the fake door + decide() (M7) ===\n');
const confidenceUpdate = updateConfidence(recommendationsBet.confidenceCeiling, { validates: weekResult.passesThreshold, strength: 0.7 });
console.log('prior=' + confidenceUpdate.prior + ' (capped by RICE in product-thinking)');
console.log('posterior=' + confidenceUpdate.posterior + '  (' + confidenceUpdate.direction + ', delta=' + confidenceUpdate.delta + ')');
const THRESHOLD = { persevere: 0.6, pivot: 0.3 };
const decision = decide(confidenceUpdate.posterior, THRESHOLD);
console.log('thresholds agreed on ahead of time: persevere >= ' + THRESHOLD.persevere + ', pivot >= ' + THRESHOLD.pivot);
console.log('signalStrength=' + decision.signalStrength + '  ->  ' + decision.action.toUpperCase());
console.log(decision.reason);

console.log('\n=== recommendationsBet, from this guide\'s module 1 to today ===\n');
console.log('BEFORE:  assumption="' + recommendationsBet.assumption + '", risk=' + recommendationsBet.risk + ', impact=' + recommendationsBet.impact + ', tested=' + recommendationsBet.tested + ', confidenceCeiling=' + recommendationsBet.confidenceCeiling);
console.log('AFTER: tested=true, confidence=' + confidenceUpdate.posterior + ', decision="' + decision.action + '"');

What to expect. When you run the complete file with Node, the output is exactly this:

=== recommendationsBet, as it arrives from product-thinking-for-engineers ===

assumption: "Users will buy more if they see personalized recommendations"
risk=0.6  impact=5  score=3
tested=false  confidenceCeiling=0.3

=== Step 1/7: the falsifiable hypothesis (M4 isFalsifiable) ===

falsifiable: true

=== Step 2/7: pickTest() chooses the cheapest test that can refute it (M4) ===

chosen: fake_door_checkout (cost=3, value=2.00)

=== Step 3/7: the interview script, verified (M2 classifyQuestion) ===

7 of 7 questions ready (good). Script run with 8 buyers.

=== Step 4/7: the checkout fake door signal (M5 fakeDoorSignal) ===

260 clicks / 4000 impressions = 6.5% (threshold: 5%) -> clears: true

=== Step 5/7: weigh the evidence without bias (M6 sampleBias + rankByStrength) ===

sample: 10 people, dominant occasional_buyer (40%), biased: false
  strength=4 [observed_behavior ] Clicked "recommended for you" on the checkout fake door
  strength=4 [observed_behavior ] Ignored the recommendations section, zero clicks across two visits
  strength=4 [observed_behavior ] Added a recommended product to the cart on the fake door
  strength=3 [reported_behavior ] Reported that the last time they bought from another store, they followed a recommendation
  strength=1 [hypothetical      ] Said they would love to see personalized recommendations

=== Step 6/7: synthesize the 8 interviews (M7 synthesize) ===

[SIGNAL] "I don't discover products I'd like without searching for the exact name"  (distinct unprompted users: 3)
[SIGNAL] "I forget about my cart and never go back to finish it"  (distinct unprompted users: 3)
[NOISE ] "I don't trust new sellers without reviews"  (distinct unprompted users: 1)
[NOISE ] "the app crashes on its own when the phone has low memory"  (distinct unprompted users: 1)

=== Step 7/7: updateConfidence() with the fake door + decide() (M7) ===

prior=0.3 (capped by RICE in product-thinking)
posterior=0.79  (up, delta=0.49)
thresholds agreed on ahead of time: persevere >= 0.6, pivot >= 0.3
signalStrength=0.79  ->  PERSEVERE
the accumulated evidence clears the threshold to keep investing as-is

=== recommendationsBet, from this guide's module 1 to today ===

BEFORE:  assumption="Users will buy more if they see personalized recommendations", risk=0.6, impact=5, tested=false, confidenceCeiling=0.3
AFTER: tested=true, confidence=0.79, decision="persevere"

This last line is the moment the entire guide has been building toward since its very first lesson: tested goes from false to true for the first time, and confidence stops being capped at 0.3 — not by decree, but because two independent sources of evidence, the synthesis of eight interviews and the fake door's result, moved it up to 0.79. The number, the decision, and the vocabulary match exactly module 7's closing project — the same evidence, processed with the same functions, can't reach a different verdict.

The findings, laid out cleanly

The decision, written out: PERSEVERE, with justification

Decision: PERSEVERE. With confidence: 0.79, well above the persevere threshold (0.6) agreed on in module 7's lesson 6, the evidence supports continuing to invest in recommendations as it's currently framed — not a pivot to the alternative solution of curated categories, not a kill. Two completely independent sources of evidence point in the same direction: the synthesis of the eight interviews confirms the underlying problem —buyers don't discover products they'd like without searching for the exact name— is real and frequent, mentioned by 3 distinct buyers with nobody suggesting it, and it's exactly the same highest-impact (5) opportunity from module 3's tree; and the checkout fake door, the pipeline's strongest piece of evidence, cleared its threshold agreed on ahead of time. The bias-audited evidence (lesson 6) doesn't cancel this reading: even though one of the three direct behavior observations was unfavorable —someone ignored the section across two visits—, that signal is already included, statistically, in the fake door's aggregate 6.5%, which summarizes thousands of real impressions, not just the few cases the team could write down by hand.

What comes next isn't more discovery — it's shipping and measuring. Unlike a pivot decision, which would require designing a different solution before continuing, persevere means the team can move from validation to construction with reasonable confidence:

1. Building the real recommendations engine —production code, backend, frontend— is the Fullstack ecosystem's job, which takes exactly this project's result as its starting point: recommendations is validated, with confidence: 0.79 and the evidence that backs it up.

2. Measure the real lift, with statistical rigor, once it's in production. This guide's fake door measured click interest on a small sample, for one week — a cheap, early proxy, not a significance measurement. Before declaring victory over the 8% add-to-cart wrongIf demands, the team needs product-metrics-and-experimentation-guide: calculated sample size, cohorts, and a real comparison against the control group. That guide is what finally closes the gap between "the click suggests interest" and "the recommendation truly moves GMV."

3. Ship with care, not all at once. shipping-and-iterating-products-guide covers progressive rollout, feature flags, and the rollback plan if the real engine, once in production, doesn't confirm what this early evidence suggests — persevere is a reasonable bet with today's evidence, not a guarantee about tomorrow's result.

What doesn't change with this decision: RICE's risk: 0.6 and impact: 5 remain intact — this decision didn't reprioritize the backlog, it only confirmed, with real evidence, that the bet that was already the most urgent deserves to keep receiving the investment it had planned.

Rubric

Before considering your own discovery pipeline complete —or your own backlog's, in the transfer exercises—, check each point:

• The hypothesis has separate believe and wrongIf, with a concrete threshold and deadline — not a version where the failure condition is a vague mirror of the success one.
• The chosen test uses pickTest() in byValue mode, not the cheapest or the strongest alone — with the discarded candidates explicitly named and their value calculated.
• The interview script passes classifyQuestion() at 100% before running with real users — a script with unrewritten leading or hypothetical questions doesn't get run.
• The main test's threshold gets fixed before seeing any data, and the result gets reported with the full number, not just the boolean.
• The sample passes sampleBias() and the complete evidence —favorable and unfavorable— gets weighed with rankByStrength(), without silently discarding uncomfortable evidence.
• synthesize() distinguishes distinct unprompted users from total mentions, and the signal threshold (signalMinUsers) is declared explicitly.
• updateConfidence() receives a single formal piece of evidence, not an inflated count — the main test's aggregate result, not that same result added again to the individual observations that already make it up.
• decide()'s thresholds get fixed before calculating the final confidence, not adjusted afterward to make the result "look good."
• The final decision comes with a concrete plan, not just the word persevere/pivot/kill — if it's persevere, the plan says what comes next (build, measure, ship); if it were pivot, which direction to change solutions toward.

Transfer exercises

Unlike the previous lessons, these three exercises don't ask you to recalculate with numbers already given — they ask you to apply the complete method to something this guide never saw. It's the real test of whether you learned the pipeline or just memorized recommendations's result.

Exercise 1 — Apply the complete pipeline to a new assumption. Mercado's team has another pending risky assumption, from the "I don't trust new sellers without reviews" opportunity (the second one in module 3's tree): "a verified-seller badge increases the probability that a buyer completes the purchase." Apply the complete method: write the falsifiable hypothesis (with believe and wrongIf), propose at least three candidate tests with estimated cost, canFalsify, and strength, and choose one with pickTest() in byValue mode, design two interview questions that would pass classifyQuestion() as good, and propose a reasonable threshold for the chosen test. Don't run updateConfidence() or decide() yet —that would require real data this guide doesn't have for this case—; the goal is to complete the design, not simulate a made-up result.

const trustTests = [
  { type: 'behavioral_interview', cost: 2, canFalsify: true, strength: 3 },
  { type: 'clickable_badge_prototype', cost: 3, canFalsify: true, strength: 5 },
  { type: 'ab_test_real_badge', cost: 8, canFalsify: true, strength: 8 },
];

Exercise 2 — Apply the pipeline to your own work. Choose a real assumption from your own context —a product, a feature, even an internal process on your team— that today gets treated as an assumed fact, without having been validated with anyone. Write its falsifiable hypothesis, propose at least two candidate tests and choose one with pickTest()'s criterion (even if you estimate cost and strength by eye), and define the decision threshold before running anything. There's no single solution —the goal is for you to complete the exercise with your own context—, but before considering it done, check every box in this lesson's rubric against your own work.

Exercise 3 — Defend PERSEVERE in writing to a skeptical director. Imagine you have to send this project's result to a director who didn't see any of this module's eight lessons, and who, reading "persevere," asks directly: "is persevere just saying everything went perfectly, with no nuance at all?" Write the complete answer —between 120 and 200 words—, including the real confidence, the two independent sources of evidence backing it, and what's left to do before considering the bet fully closed.

Summary and next step

In this final project you ran, end to end, recommendations's complete discovery pipeline: seven chained steps —falsifiable hypothesis, chosen test, verified script, fake door signal, bias-audited evidence, synthesis of the eight interviews, and confidence update— that carried the bet's confidence from RICE's inherited 0.3 ceiling up to 0.79, and a final decision, defensible step by step: PERSEVERE — the same number, the same vocabulary, and the same decision module 7's closing project already reached over this identical evidence.

With this, you close out the complete guide. You now have the entire method: why discovering before building is cheaper than building to discover (module 1), how to talk to users without the conversation lying to you (module 2), how to map the complete problem before jumping to a solution (module 3), how to turn a hunch into a hypothesis that can be knocked down and choose the cheapest test that would knock it down (module 4), how to prototype at the right level of fidelity (module 5), how to distrust your own reading of the evidence (module 6), how to synthesize real signals and move a belief with discipline (module 7) — all applied, end to end, to a real assumption in this module 8.

Where you go from here

This module delivers a user-validated decision: persevere, with confidence: 0.79. It doesn't deliver, and never promised to deliver, three things you need next:

• Measure the real lift, with statistical rigor, once the engine is in production. This guide's fake door measured click interest on a sample of 10 people, in one week — a cheap, early proxy for the add-to-cart condition wrongIf demands (8%), not a direct measurement. The recommendations engine, once built, needs to be measured with real significance, calculated sample size, and cohorts — that's product-metrics-and-experimentation-guide, the sibling guide that picks up exactly where this one ends.
• Ship with care, not all at once. Rolling out recommendations to production with feature flags, progressive rollout, and a rollback plan if the real result doesn't confirm this early evidence is shipping-and-iterating-products-guide.
• Reprioritize the complete backlog, if this result changes the quarter's outlook. RICE's risk and impact didn't change with this project —but a real team, with tested: true and confidence: 0.79 in hand, probably wants to revisit how recommendations now compares against the rest of the backlog. That work, again, is product-thinking-for-engineers-guide, the guide that left this assumption at this one's door.

And there's one final boundary, the same one that's held up this entire guide since its first lesson: actually building —the recommendations engine— is the Fullstack ecosystem's job. This guide taught you to decide whether it's worth building, with what evidence, and with what level of confidence — an argument you can defend, step by step, column by column. The rest of the road —measuring with rigor, shipping with care, actually building— is what comes next.

Resources

• Teresa Torres, Continuous Discovery Habits — producttalk.org/continuous-discovery-habits. The book that supports this entire guide's argument, start to finish. Worth reading in full now that you have the complete method. In English.
• Rob Fitzpatrick, The Mom Test — momtestbook.com. The source behind the interview script this project ran unchanged since module 2. In English.
• David J. Bland and Alexander Osterwalder, Testing Business Ideas — strategyzer.com/library/testing-business-ideas-book. The complete catalog of experiments, from which this project chose, with an explicit criterion, just one. In English.
• Eric Ries, The Lean Startup — theleanstartup.com/book. The original source of the "pivot" vocabulary — worth rereading now that this project's final decision was persevere, not pivot, to know clearly what the alternative would have meant. In English.
• Marty Cagan (Silicon Valley Product Group), Inspired — svpg.com/inspired-how-to-create-products-customers-love. The direct bridge to product-metrics-and-experimentation-guide and shipping-and-iterating-products-guide — how to carefully measure and ship what this module decided is worth building. In English.