7. Testing and replaying executions with the debugging engine

Description

By the end of this lesson you'll be able to load a real, past execution of your agent inside the editor, freeze its input data, and compare two variants — of model or of prompt — against exactly the same message, without messaging the bot again or triggering the real channel a second time.

This matters because you already glimpsed it from a distance in this module's introduction: comparing variants by messaging the real agent on WhatsApp or on the live chat has a double cost. You pay again for every call to the model, and on top of that you lose certainty that the test message is identical to the previous one — because you're rewriting it by hand, with slightly different wording each time. A freelancer or an agency iterating on an agent for a client needs to be able to answer "yes, I already confirmed the change works with real data" without that meaning spending another batch of tokens or bothering the client again asking them to repeat their test message.

Connection to the module: in lesson 2 you already saw the debugging engine in passing — as the tool for confirming, on a real sample, which model candidate performs better before pinning it down. In lesson 4 you saw it mentioned again, comparing Llama 3.2 against Mistral. Today is its turn: you're going to learn the mechanism itself. You won't write a new System Message (that was lesson 5) or adjust temperature or structured output (lesson 6) — you're going to assume those pieces already exist, and you're going to learn to test them with no friction before lesson 8's mini-project.

Comparing without repeating the experiment

Imagine a sound engineer testing two different guitar pedals to see which one sounds better on a song. If they play the song live once with pedal A and, minutes later, play it live again with pedal B, any difference they hear could come from the pedal — or it could come from the second take being a bit faster, or from a chord they got wrong the first time. There's no way to isolate the cause. That's why no serious engineer tests pedals that way: they record a single take of the song, keep it fixed, and run it through pedal A and then through pedal B. The performance is identical both times; the only thing that changes is the pedal. Only then does the difference they hear mean something.

n8n keeps something similar to that recorded take. Every time your agent runs, the workflow's Executions tab keeps a complete record of that run: what went into each node, what came out. Two buttons let you bring that record back to the canvas — Debug in editor if the execution failed, Copy to editor if it finished successfully — and both do the same thing: they copy that exact execution's data and pin it to the first node in your flow, usually the Chat Trigger. From that point on, that node stops waiting for a new message from the real channel — it delivers, every time you run it, the same saved data. It's your recorded take.

With the input frozen, you can now change a single thing — the model sub-node, one line of the System Message — and run just that part of the flow again. Nothing triggers toward the real channel: no new message arrives via WhatsApp, and you don't spend an extra call on the trigger. The only thing that runs again is whatever you decided to touch, against the same message as always.

One practical detail before continuing: if you're running n8n self-hosted — like the Self-Hosted AI Starter Kit v2 you spun up in lesson 4 — Debug in editor and Copy to editor only show up on a registered Community instance. Registration is free: you give your email, get a license key, and once activated it doesn't expire. If you don't see those buttons on your local instance, that's the first thing to check, not an n8n bug.

Worked example

Your TuTienda agent already has lesson 5's System Message and lesson 6's structured JSON output: every response has to arrive as an object with a text field for the customer and a boolean field flagging whether the case needs to be escalated to a human. Yesterday, a real customer wrote this on the live chat:

Real input (execution #1842, logged yesterday at 14:32):

Hi! I have two orders, 4521 and 4530 (they're from the same month)
can you tell me the status of BOTH? and a separate question:
what's the minimum for free shipping?

The execution failed. The node that processes the agent's response expecting lesson 6's JSON threw a format error: the model, faced with a message with two orders and a separate question, put a courtesy phrase before the JSON ("Sure, happy to help with that!"), and the parser didn't know what to do with text before the opening brace.

Step 1 — locate and load the failed execution. In the Executions tab, you filter by "Failed" and find #1842. You open it: the node marked in red is the one trying to interpret the agent's response as JSON. You select Debug in editor.

What to expect: n8n copies the customer's real message to the canvas and pins it on the Chat Trigger — you see the "This data is pinned" banner in that node's output panel. The rest of the flow stays built exactly as it was yesterday.

Step 2 — isolate one variable at a time. You have two hypotheses about why it failed: (a) maybe a model with more reasoning capability handles a message with two stacked questions better, without touching the prompt; (b) maybe the System Message needs an explicit line like "respond only with the JSON object, with no text before or after." You test them one at a time, never both together — if you change the model and the prompt at the same time and the error goes away, you won't know which of the two fixed it.

Variant A — same System Message, change the model. You replace the current Chat Model sub-node (Claude Sonnet 5) with one connected to GPT-5.6 Terra, without touching the System Message. You run just the AI Agent node against the pinned data.

What to expect — Variant A: the new model also puts a courtesy phrase before the JSON. The error persists. Conclusion: it wasn't a model reasoning-capability problem — both models, given the same ambiguous prompt about the format, make the same slip.

Variant B — you go back to Claude Sonnet 5, add a line to the System Message. You add: "Your entire response must be only the JSON object — no greeting, no text before or after." You run the AI Agent node again against the same pinned data.

What to expect — Variant B: the response arrives as clean JSON, with no preamble, and the node that parses it no longer fails. The case confirms what you saw in the module's introduction: model and prompt are independent axes, and this symptom — a broken output format — lived in the prompt, not the model. Changing models was never going to fix it.

Step 3 — confirm the fix without reopening the canvas. You already saved the new System Message line to the real workflow. To confirm, with one click, that this change really does resolve yesterday's real case, you go back to the Executions list, open #1842, and from the refresh icon you choose Retry with currently saved workflow — this re-runs that execution's original data, but against the workflow version you have saved right now, new line included.

What to expect: the execution is marked successful. You have, in one click and without rewriting anything to the customer, confirmation that the fix works on the real case that broke it — not on a test message you made up.

Two ways to replay an execution, and what each one is for

The refresh button on an execution in the Executions list gives you two options, and confusing them leads you to the wrong conclusion:

• Retry with currently saved workflow — runs that past execution's input data, but against the configuration you have saved right now in the workflow. This is the one you used in step 3: it's for confirming that a change you already applied resolves a specific real case.
• Retry with original workflow — runs exactly what ran that time: same data, same model, same prompt, with none of the changes you saved afterward applied. This is for a different question: does yesterday's failure repeat every time you give it that same input, or was it a one-time fluke?

That second question matters more than it seems. As you saw in lesson 6, the temperature parameter introduces variation into the model's response — and Anthropic's documentation is explicit that, even at its lowest value, the result isn't completely deterministic. That means the same model, with the same prompt and the exact same input, can respond differently across two separate runs. If you run Retry with original workflow on execution #1842 several times and the format error always shows up, you have a real prompt problem (the one you fixed in Variant B). If it only shows up sometimes, you were looking at a sampling fluke, not a pattern — and there the criterion changes: maybe the System Message didn't need touching at all, but the temperature needed lowering so that kind of slip happens less often.

Common mistakes

Thinking loading a past execution "freezes" the whole flow, not just the input data (conceptual). What happens: after using Debug in editor, you change the model or the System Message, run it again, and expect to see yesterday's same response because "that execution is already recorded." When the result is different, some people assume the debugging engine isn't useful for testing real changes. Why it happens: the name "pin data" and the idea of "loading the past" suggest everything stayed fixed, when actually only the node where you put the pin — typically the trigger — stops asking for new data. Everything downstream of that node actually executes, with your current configuration, every time you run it. How to spot it: if you changed something and the result doesn't move at all, first suspect the change wasn't saved or that you're looking at an old execution's output panel, not that "the debugging engine won't let you test anything new." How to fix it: think of the pin as "freezing only the input," never as "freezing the entire experiment" — the rest of the flow stays alive and responds to whatever you edit.

Changing two variables at once when comparing (conceptual). What happens: you touch the model and the System Message in the same run — as would have happened in the worked example if you'd tried GPT-5.6 Terra with the new format line from the start — the error goes away, and you call the case closed without knowing which of the two changes fixed it. Weeks later, someone wants to revert the model for cost reasons and the format bug comes back, because the model was never actually what fixed it. Why it happens: once you already have two hypotheses in your head, it's tempting to test them together to "save a round" — it seems more efficient, even though it destroys your ability to isolate the cause. How to spot it: if you can't answer in one concrete sentence which change caused the improvement, you probably changed more than one thing at once. How to fix it: use the pinned data to test one variable per run, exactly like Variants A and B in the example — it's slower comparison by comparison, but it's the only way to know, with certainty, what actually worked.

Confusing "Retry with original workflow" with "Retry with currently saved workflow" (practical). What happens: you apply a fix to the System Message, save it, and to confirm it you mistakenly pick Retry with original workflow on the execution that failed. The execution fails again in exactly the same way — because that button ignores your saved change and repeats the original configuration, not the new one — and you wrongly conclude the fix didn't work. Why it happens: the two names are almost identical and sit right next to each other in the same menu; the difference between "currently saved" and "original" is subtle if you don't read carefully. How to spot it: if you "confirmed" a fix with a retry and the error persisted identically, check which of the two buttons you used before assuming the fix failed. How to fix it: use "Retry with currently saved workflow" to put a change you already saved to the test, and reserve "Retry with original workflow" strictly for confirming whether a past failure is reproducible or was a one-off fluke.

Exercises

Exercise 1. Going back to lesson 2's example — comparing Claude Sonnet 5, GPT-5.6 Terra, and Gemini 2.5 Flash for the agent that classifies support tickets —: you have a real execution where Sonnet 5 misclassified a ticket's urgency. Describe, in order, the steps you'd use with the debugging engine to test whether Gemini 2.5 Flash classifies that same ticket better, without asking a real customer for that ticket again.

See solution

1. In the ticket agent's workflow Executions tab, you locate the execution where Sonnet 5 misclassified the urgency and open it.
2. You select Debug in editor (it was an execution with an incorrect result, but technically it didn't "fail" — if it finished with no error, the button would be Copy to editor; if the output parser did throw an error, it would be Debug in editor). Either one copies the real ticket and pins it to the first node.
3. You replace the current model sub-node with one connected to Gemini 2.5 Flash, without touching the System Message or any other node.
4. You run just the agent node against the pinned data and compare the new urgency classification against the one Sonnet 5 gave on that exact same ticket.

Why it works: the real ticket stays frozen as the comparison's only input, so any difference in classification comes solely from the model — the only variable you changed — not from the second ticket being different or worded differently.

Exercise 2. A teammate changes, in the same run, the agent's model and a line of the System Message. The error they had goes away. They ask you: "was it the model or the prompt that fixed it?" What do you tell them, and what should they have done differently?

See solution

There's no way to know from the evidence they generated, because they changed two variables at once — exactly this lesson's common mistake. They should have tested each change separately against the same pinned data: first just the new model (with the old System Message), then just the new System Message line (with the old model), and compared each result against the original. Only with those two isolated runs would they know which of the two changes — or whether both were needed — resolved the case.

Why it works: isolating one variable per run is the only thing that lets you attribute an improvement to a specific cause; changing several things at once can fix the symptom, but leaves everyone guessing why, which matters the day someone wants to revert just one of the two changes.

Exercise 3. You already applied and saved a fix to your agent's System Message. You want to confirm, without opening the canvas or touching any node, that fix resolves the real case that failed yesterday (execution #1842). Separately, you also want to know whether the original failure was always reproducible or was a one-time fluke. Which button do you use for each question?

See solution

To confirm the already-saved fix resolves the real case: you open execution #1842 in the Executions list and choose Retry with currently saved workflow — it runs that execution's original data against the configuration you have saved now, fix included. To find out whether the original failure was reproducible: you choose Retry with original workflow once or several times on that same execution — this repeats exactly yesterday's configuration and data, without your fix, and if the error shows up every time, it confirms it was a real prompt problem and not a model sampling fluke.

Why it works: each button answers a different question — one tests your change against the past, the other tests whether the past repeats as-is — and picking the wrong one gives you an answer that doesn't match the question you're asking.

Exercise 4 (challenge). Design, step by step, how you'd use the debugging engine to compare three model candidates (Claude Sonnet 5, GPT-5.6 Terra, Gemini 2.5 Flash) on the same batch of 5 real support tickets — not just one — while controlling for the only variable that changes between runs being the model.

See solution

For each of the 5 real tickets: you locate its execution in the Executions list and load it with Debug in editor or Copy to editor depending on whether it failed or not, pinning that ticket to the first node. With the ticket pinned, you run the agent node three times in a row — once per model — replacing only the Chat Model sub-node between runs, without touching the System Message or any other node. You record the classification and the draft reply each model gave for that ticket. You repeat the same process for the other four tickets, and at the end you compare, ticket by ticket, which of the three models got the classification right more often and produced the better draft.

Why it works: pinning each ticket separately ensures all three models compete on exactly the same input in every round — you never compare one model's response on ticket 1 against another model's response on ticket 2 — and changing only the model sub-node between runs keeps isolated the single variable you're interested in measuring.

Summary and next step

You now know how to load a real execution — a failed one with Debug in editor, a successful one with Copy to editor — and leave its input data pinned to the first node in your flow. You know how to change a single variable at a time — model or prompt — and run just that part against the frozen data, without triggering the real channel or spending an extra call. And you know when to use Retry with currently saved workflow to confirm a fix, versus when to use Retry with original workflow to check whether a past failure is reproducible.

This is the foundation for what's next: in lesson 8's mini-project you're going to assemble the complete agent — a model chosen with judgment, a System Message with its own boundaries — and the final step before calling it done is exactly this: running it against saved real cases and confirming it behaves as you expect, instead of assuming it from how it looks over a couple of loose messages.

Before moving on you should be able to explain, without looking back at this lesson: what the difference is between Debug in editor and Copy to editor; what it exactly means for data to be "pinned" on a node, and what part of the flow keeps genuinely running after that; and why changing two variables in the same run keeps you from knowing which of the two caused an improvement.

Resources

• Debug executions — n8n Docs — how Debug in editor and Copy to editor work, and exactly what they copy to the canvas.
• Pin and mock data — n8n Docs — what pinning a node's data means and how it combines with execution history.
• View executions for a single workflow — n8n Docs — the exact difference between Retry with currently saved workflow and Retry with original workflow.
• Community edition features — n8n Docs — why you need a registered (free) Community instance to see these buttons if you run n8n self-hosted, like lesson 4's stack.
• Messages API — Claude Docs — the temperature parameter and the official note that, even at its lowest value, the result isn't completely deterministic; the basis for why it's sometimes worth re-running an execution without changing anything.