Fidelity vs. cost: why more expensive isn't always better

Overview

You already have all four levels: paper (lesson 3), clickable (lesson 3), wizardOfOz (lesson 4), fakeDoor (lesson 5), each with its relative cost and its list of questions it can answer. This lesson brings them together for the first time into a single table and asks something you've only hinted at so far: how does cost, exactly, rise as fidelity rises — and is that rise proportional to how much more each level can answer, or not?

Today you build relativeToBaseline(), which compares each level's cost against the cheapest of the four (paper), and you're going to see something not intuitive at first glance: cost doesn't rise in a straight line with fidelity, and the most expensive of the four levels isn't, not by a long shot, the one that can answer the most questions.

How this connects to the module. The four objects you reuse today —paper, clickable, wizardOfOz, fakeDoor— are exactly the same ones from lessons 3, 4, and 5, with no changes at all. This is the complete catalog (FIDELITY_LEVELS) lesson 7 is going to use, also unchanged, inside pickFidelity — the module's final decision algorithm.

An analogy: back to the car, now looking at the bill

You already saw, in the module overview, the sketch-model-real-car analogy. Take it one step further and look at what each step costs, not just how it looks. A pencil sketch of a new car costs almost nothing — one designer, one afternoon. A full-scale model, hand-sculpted in clay by specialists, costs orders of magnitude more and takes weeks — but it's still a tiny fraction of what the next step costs. A full-size functional prototype, with a real engine, real brake systems, able to drive on a test track, costs orders of magnitude more still, and is measured in months of work by dozens of engineers.

Notice something a car design studio knows by heart, even though it never says it out loud in a meeting: the clay model doesn't answer "twice" as many questions as the sketch for costing much more — it answers different questions, about how light falls on curved surfaces, something a flat sketch could never answer. And the functional prototype doesn't beat the model on "more design questions" — it beats it on a completely separate type of question: whether the car is safe at high speed. Cost doesn't buy more of the same — it buys access to a type of question the previous level couldn't touch. That's exactly the idea this lesson puts into numbers for a product prototype's four levels.

Worked example: relativeToBaseline() over the 4 levels together

We bring the catalog's four levels together —exactly the same objects from lessons 3, 4, and 5, with no changes at all— and calculate, with paper as the reference, how many times more each one costs.

// L6: the 4 fidelity levels, brought together -- exactly the same objects
// from lessons 3, 4, and 5, with no changes at all. relativeToBaseline()
// compares each level's cost against the cheapest one (paper), to see how
// fast cost rises as fidelity rises.
const paper = {
  type: 'paper',
  cost: 0.5,
  canAnswer: [
    'the flow makes sense from start to end',
    'the screen order makes sense',
  ],
};

const clickable = {
  type: 'clickable',
  cost: 3,
  canAnswer: [
    'the flow makes sense from start to end',
    'the screen order makes sense',
    'the interaction feels natural',
    'users complete a multi-step task without help',
  ],
};

const wizardOfOz = {
  type: 'wizardOfOz',
  cost: 5,
  canAnswer: [
    'users trust a response that looks automatic',
    'they would use this in a real task, with real data, more than once',
  ],
};

const fakeDoor = {
  type: 'fakeDoor',
  cost: 1.5,
  canAnswer: [
    'there is real, measurable demand (clicks) for this idea, without building anything',
  ],
};

const FIDELITY_LEVELS = [paper, clickable, wizardOfOz, fakeDoor];

function relativeToBaseline(levels, baselineType) {
  const baseline = levels.find((l) => l.type === baselineType);
  return levels.map((l) => ({
    type: l.type,
    cost: l.cost,
    multiplier: Math.round((l.cost / baseline.cost) * 10) / 10,
    questionsItCanAnswer: l.canAnswer.length,
  }));
}

console.log('=== cost of each level, relative to paper ===\n');
const comparison = relativeToBaseline(FIDELITY_LEVELS, 'paper');
comparison
  .sort((a, b) => a.cost - b.cost)
  .forEach((l) => {
    console.log(l.type.padEnd(12) + 'cost=' + String(l.cost).padEnd(5) + (l.multiplier + 'x paper').padEnd(10) + 'answers ' + l.questionsItCanAnswer + ' question type(s)');
  });

What to expect. When you run the file with Node, the output is exactly this:

=== cost of each level, relative to paper ===

paper       cost=0.5  1x paper  answers 2 question type(s)
fakeDoor    cost=1.5  3x paper  answers 1 question type(s)
clickable   cost=3    6x paper  answers 4 question type(s)
wizardOfOz  cost=5    10x paper answers 2 question type(s)

Look carefully at the table's two extremes. wizardOfOz is the most expensive of the four —ten times what paper costs— and yet it only answers two types of question, exactly the same as paper, the cheapest. fakeDoor, for its part, costs barely three times what paper costs —a fraction of clickable's or wizardOfOz's cost— and answers only one question, but it's a question no other level in the table can touch: whether there's real, measurable demand, without building anything. Cost doesn't rank the four levels from "worst" to "best" — each one buys access to a different type of question, and the number of questions it answers has no direct relationship to how much it costs.

Why clickable "wins" on quantity, without winning on everything

clickable answers more question types than any other level in the table —four—, and costs less than wizardOfOz. Looking only at these two columns, it's tempting to conclude clickable is "the best" of the four overall. But go back to the specific recommendations question that's accompanied you since module 4: "is there real demand for this idea?" — that question isn't on clickable's list, no matter how many other questions it does answer. No level is superior to the others in the abstract; each one is superior for a specific question. Lesson 7 turns this observation into an explicit algorithm.

Common mistakes

Assuming the most expensive level always gives more signal, for any question. What happens: facing any given question, someone automatically proposes the highest available fidelity level —"let's do a wizard-of-oz, that way we get the strongest possible signal"— without checking whether that specific question is on its canAnswer list. Why it happens: "more fidelity = more expensive = more serious" feels like a logical progression, and it's easy to generalize it to "more fidelity = better answer," which today's table shows is false. How to spot it: today's table is the direct proof — wizardOfOz costs ten times more than paper and answers exactly the same number of questions. How to fix it: before choosing a level for its fidelity, verify the specific question is on its canAnswer — a higher cost doesn't automatically buy an answer to questions that level, by design, can't answer.

Rationalizing "since we're going to spend anyway, let's just build the real version". What happens: someone argues that, if the team has to invest time in a high-fidelity prototype anyway —like a wizard-of-oz, at cost=5—, the gap to building the full real feature isn't that big anymore, so it's better to skip the prototype entirely. Why it happens: compared to the very high cost of actually building (weeks or months, as you saw in module 1 with buildAndSee), any prototype's cost looks small — and that mistaken comparison makes the gap between cost=5 and "build everything" feel smaller than it really is. How to spot it: the proposal to skip the prototype compares its cost against the real product's build cost, instead of comparing it against a lower-fidelity prototype's cost that could also answer the question. How to fix it: the right comparison is always between prototype levels, not between "any prototype" and "the real product" — go back to today's table and ask whether a cheaper level already suffices, not the full build.

Treating cost as if it captured everything a prototype costs. What happens: the team chooses a level based only on the cost number (build person-days), without considering other real costs today's model doesn't include — the time of the users recruited to test it, the risk of showing something unpolished to an important customer, or the fatigue of repeatedly asking the same group of users to test things. Why it happens: cost is a concrete, easy-to-compare number, and it's natural to treat a concrete number as if it were the complete measure of "how much" something costs. How to spot it: the decision of which level to use never mentions who the prototype is going to be shown to, or how many times that same group of users has already been asked for something similar. How to fix it: use cost as a useful starting point, not as the complete answer — before running any prototype, also ask what it costs the user to participate, and whether the risk of showing something early justifies the savings over building the full version.

Exercises

Exercise 1 — Change the comparison baseline. Without running Node, if relativeToBaseline() ran with fakeDoor as the reference instead of paper, what would wizardOfOz's multiplier be? (Hint: divide wizardOfOz's cost by fakeDoor's cost.)

See solution

5 / 1.5 = 3.333..., rounded according to the lesson's formula (Math.round(x × 10) / 10) to 3.3x. Compared to the 10x that came out when the reference was paper, the same wizardOfOz looks "only" 3.3 times more expensive than fakeDoor — the multiplier changes completely depending on which level you use as the reference point. The exercise shows that "how many times more expensive" isn't a fixed property of a level — it depends on what you're comparing it against, so it's always worth asking "more expensive than which alternative, specifically?", not just "is it expensive?".

Exercise 2 — Find the question nobody can answer cheaply. Looking at all four canAnswer lists together, is there any question no fidelity level, not even the most expensive one (wizardOfOz), can answer? Name a real question about recommendations that falls outside all four lists, and explain which sibling guide it would belong to.

See solution

A real question that falls outside all four lists: "how much does real GMV go up if we launch recommendations to all users?" — no prototype, no matter its fidelity, can answer that question, because it needs to measure real purchase behavior, at real scale, with statistical rigor (significance, sample size). That question belongs to product-metrics-and-experimentation-guide, the sibling guide mentioned several times in this module — prototyping answers whether it's worth continuing to invest; measuring the real result, once built, is a completely separate territory.

Exercise 3 — Argue for fakeDoor using only today's table. Without mentioning the word "cheap," use today's lesson's table numbers to explain, in 2-3 sentences, why fakeDoor was a good choice for recommendations's demand question, compared to the other three levels.

See solution

A reasonable answer: "Of the four levels, fakeDoor is the only one that has the demand question on its canAnswer list — neither paper, nor clickable, nor wizardOfOz, no matter how much they cost, can answer it. On top of that, its relative cost (3x over the baseline) is lower than clickable's (6x) and far below wizardOfOz's (10x) — so it isn't just the only candidate that answers the right question, it also turns out to be one of the most economical of the four." The point of the exercise is to show that choosing fakeDoor in module 4 wasn't a coincidence or an arbitrary preference — today's table backs it up with concrete numbers.

Summary and next step

In this lesson you brought the catalog's four levels together into a single table and calculated, with relativeToBaseline(), how much each one costs relative to paper: fakeDoor at 3x, clickable at 6x, wizardOfOz at 10x. And you saw the module's central finding: the most expensive level (wizardOfOz) isn't the one that answers the most questions, and the level that answers the most questions (clickable) doesn't include the specific demand question you needed for recommendations. Cost and answering power aren't aligned on a single ladder — each level buys access to a different type of question.

Before moving on you should be able to: explain why the most expensive of the four levels isn't automatically "the best"; calculate the relative cost between any pair of levels; and defend, with numbers, why fakeDoor was the right choice for recommendations's demand question.

Lesson 7 turns everything you saw today into a complete algorithm: pickFidelity(question, options), which, given any question and the full list of candidates, automatically chooses the cheapest one that can actually answer it — the same reasoning you just did by hand, now formalized.

Resources

• Nielsen Norman Group, "UX Prototypes: Low Fidelity vs. High Fidelity" — nngroup.com/articles/ux-prototype-hi-lo-fidelity. You already saw it in previous lessons; it's worth rereading with today's cost table in mind — it describes why "tearing up finished code is far more expensive than discarding a simple prototype." In English.
• IDEO, "7 principles to guide your prototyping" — ideo.com/journal/7-principles-to-guide-your-prototyping. On why refining a prototype beyond what's needed to answer your specific question is, literally, spending with nothing bought in return — this lesson's same central idea, from design practice. In English.
• David J. Bland and Alexander Osterwalder, Testing Business Ideas — strategyzer.com/library/testing-business-ideas-book. The book organizes its 43 experiments, precisely, by cost and by evidence strength — the same comparison-table logic you built today, applied to a much broader catalog. In English.