5. Error workflows and dead-letter queues

Description

By the end of this lesson you will be able to build two pieces that work together: a central error workflow — with the Error Trigger node as its first node — that catches any failure from any workflow in the system in one single place, and a dead-letter queue — a dead_letter table in the same ledger Postgres — where an item that failed after exhausting all its retries gets set aside with its full context, so it doesn't get lost and can be reprocessed by hand. You will see how to configure the error workflow in a workflow's settings, exactly what payload the Error Trigger receives when something fails, how that payload turns into the alert-or-log-only decision you designed in lesson 4, and how you reprocess a set-aside item without duplicating anything.

This matters because it's where the previous lessons' decisions turn into machinery that runs on its own. You defined that a stuck refund should alert finance (lesson 4); here's the node that catches that failure and sends the alert. You defined that a malformed order shouldn't get lost; here's the table that stores it. Without this piece, a terminal failure ends up as an execution marked red that someone has to discover by chance, and the item that caused it evaporates. With this piece, every failure in the system passes through a single funnel that logs it, routes it by severity, and — if it couldn't be processed — sets it aside intact. It's the net under the trapeze: you might not use it often, but it's the difference between a stumble and a fall.

Connection to the module: this lesson builds the infrastructure that executes lesson 4's decisions and catches what lessons 2 and 3 couldn't resolve — the retry that ran out, the compensation that also failed. It leans on Module 4's ledger, because the dead-letter queue is another table in the same Postgres and follows the same dedup and idempotency ideas. Lesson 6 will use what this queue stores: the context of a set-aside failure is precisely what you're going to load into the replay engine to reproduce a bug. Remember a restriction that still stands: writing to Postgres is done with the Postgres node, not from a Code node, and calling an alert API is done with the HTTP Request node.

The Error Trigger node: a single funnel for every failure

Let's start with the piece n8n gives you ready-made: the Error Trigger node.

An Error Trigger is a special kind of trigger node. Instead of firing off a webhook or a schedule, it fires when another workflow fails. It's the first node of a separate workflow — the error workflow — whose only job is to react to everyone else's failures. When any workflow in the system ends in an error, n8n starts the error workflow and hands it, through the Error Trigger, every detail of what went wrong.

The analogy is a building's emergency switchboard. Every unit has its own sensors, but they're all wired to a single central panel at the front desk. It doesn't matter which floor an alarm goes off on: the signal arrives at the same place, with information about which floor and what type of alarm, and from there someone decides who to call. The Error Trigger is that front desk: a single place where failures from every workflow arrive, with information about which one failed and why.

The advantage of having one central error workflow instead of handling errors inside each workflow is the same as any funnel's: one single place to maintain, one single "how do I react to a failure" logic, instead of repeating that logic — and its inevitable inconsistencies — across each of Cumbre's four workflows. You change the policy in one place and it applies to the whole system.

It's worth knowing that the two ways of handling errors don't exclude each other, and each has its place. Handling inside the workflow — with the node's error output you saw in lesson 2, which routes the failed item down its own branch without stopping the flow — serves failures you want to resolve on the spot, like triggering a compensation (lesson 3). The central error workflow serves failures that already stopped the execution and need to be logged, classified, and set aside afterward. A mature system uses both: the error output to react on the spot to what can be resolved, and the central Error Trigger as the net that catches everything that reached the end unresolved. It isn't one or the other; it's each at its own level.

How to set up an error workflow

It's two steps, and it's worth having each one clear because the order matters.

Step 1 — Create the error workflow. You create a new workflow whose first node is an Error Trigger. You give it a recognizable name — for Cumbre, cumbre-error-handler — and save it. That Error Trigger needs no configuration: its job is to receive the failure. Everything that comes after — logging, deciding severity, alerting, setting aside in the queue — you design.

Step 2 — Tell each workflow to use that error workflow. In every workflow in the system you open its options — Options > Settings — and in the Error workflow field you pick cumbre-error-handler. With that, when that workflow fails, n8n will start the error workflow and pass it the failure.

Three details from the official documentation worth knowing, because they save confusion:

• A workflow that contains an Error Trigger node uses itself as its own error workflow by default. That is, if cumbre-error-handler itself failed, you don't need to set up another one for it; it already knows how to handle itself. And you don't need to activate or publish it for that.
• You can't test an error workflow by running it manually. The Error Trigger only fires when a workflow fails during an automatic execution. If you run it manually to test it, nothing happens, because there was no real failure to trigger it. This confuses a lot of people the first time: it looks like "it doesn't work," when it's actually waiting for a real failure.
• It's worth assigning the error workflow to every workflow in the system, so none of them fails silently. It's one checkbox per workflow, and it's one of the ones that give the most peace of mind for how little it costs.

The payload: what the Error Trigger receives when something fails

Here's the concrete part. When a workflow fails, the Error Trigger receives an item with a precise structure, and you need to know it because that's where you pull everything you're going to log and alert on.

According to the official documentation, when any node in the workflow fails (not the trigger), the payload has this shape:

{
  "execution": {
    "id": "231",
    "url": "https://your-instance/workflow/abc/executions/231",
    "retryOf": "230",
    "error": {
      "message": "error description",
      "stack": "technical error trace"
    },
    "lastNodeExecuted": "Place credit hold",
    "mode": "trigger"
  },
  "workflow": {
    "id": "abc",
    "name": "check-credit"
  }
}

Let's break down the fields you're actually going to use:

• workflow.name and workflow.id: which workflow failed. It's the first thing you need to decide the policy — remember, in lesson 4 the policy was per workflow.
• execution.lastNodeExecuted: the name of the node where the execution stopped. It tells you where it failed: Place credit hold, issue-refund, the validation node. It's gold for diagnosing.
• execution.error.message: the error description, in readable text. It's what goes in the alert and the log.
• execution.error.stack: the technical trace, more detailed. Useful for debugging, but usually not for the alert — it's noise for a rushed human.
• execution.id and execution.url: the identifier and the direct link to the execution that failed. This is what turns an alert into a useful one (lesson 4): the owner clicks and lands directly on the detail. Watch out for a nuance from the documentation: these two fields require the instance to be saving executions to the database; if the failure happens in the trigger itself, they might not be present.
• execution.retryOf: if this execution was a retry of another one, here's the id of the original. It only appears on retries.

There's a second form of the payload, for when the failure happens in the trigger node itself — before the execution really starts. In that case, instead of execution you get a trigger object with the error, and workflow with its id and name. It's an edge case, but it's worth knowing it exists so your error workflow doesn't break when it receives a payload shaped differently than it expected. In practice, you read the fields carefully and tolerate some being missing.

The dead-letter queue: not losing what couldn't be processed

Now the second piece, the one that gives the lesson its name.

A dead-letter queue is a place where messages or items get set aside that, after every attempt, couldn't be processed — so they don't get lost and can be reviewed later. The name comes from postal mail: a "dead letter" is a letter that couldn't be delivered or returned to the sender, and that the post office sets aside in a special drawer instead of throwing away.

That image is exactly the concept. When a Cumbre order fails terminally — it exhausted its retries, its compensation also failed, or it arrived so malformed it couldn't even be evaluated — you don't discard it and you don't leave it spinning around breaking things. You put it in the dead-letters drawer: the dead_letter table, in the same ledger Postgres. There it waits, intact and with its context, until a human reviews it and decides what to do — reprocess it, fix it, or knowingly discard it.

The difference between a system with a dead-letter queue and one without is brutal, and it shows at the worst moment. Without the queue, an order that failed terminally simply disappears: the execution was left red, and if no one saw it, Luna Coffee's order never got processed and no one knows until Luna calls angry. With the queue, that same order is stored, tagged with why and when it failed, waiting for someone to attend to it. Nothing gets lost silently: that's the module's entire promise, made into a table.

The dead_letter table

Let's look at what columns it needs, and why each one. Remember that creating and writing this table is done with SQL through the Postgres node, on the Postgres the Starter Kit ships with.

-- Cumbre's dead-letter queue.
-- Lives in the same Postgres as Module 4's run_ledger.
CREATE TABLE dead_letter (
  id              BIGSERIAL PRIMARY KEY,
  order_id        TEXT,          -- to find the order again and correlate
  source_workflow TEXT,          -- which workflow failed: 'issue-refund', etc.
  failed_node     TEXT,          -- lastNodeExecuted: where it stopped
  error_message   TEXT,          -- error.message: why it failed, readable
  execution_id    TEXT,          -- execution.id: link to the execution for replay
  payload         JSONB,         -- the full item that was being processed
  severity        TEXT,          -- 'critical' | 'warning' (from lesson 4)
  status          TEXT DEFAULT 'pending',  -- 'pending' | 'reprocessed' | 'discarded'
  created_at      TIMESTAMPTZ DEFAULT now()
);

Notice two columns that do all the heavy lifting:

payload (the full item). This is the queue's whole reason for being. You store the entire order, exactly as it was being processed when it failed, not just its order_id. That way, when someone reprocesses it, they have every original piece of data without depending on the original event still being available anywhere. It's the whole letter inside the drawer, not a note saying "a letter arrived."

status. Marks the state of every set-aside item: pending (waiting for review), reprocessed (already reprocessed successfully), discarded (someone knowingly decided not to process it). Without this column, you wouldn't know which items in the queue have already been handled and which are still waiting, and you'd end up reprocessing things already resolved.

Worked example: Cumbre's error workflow end to end

Let's build the complete cumbre-error-handler, combining the Error Trigger, lesson 4's decision, and the queue. This is the workflow all four workflows in the system point to in their settings.

Error Trigger  ──►  Code: "Classify failure"  ──►  Switch (by severity)
                                                      ├─ critical ─► HTTP Request: alert
                                                      │                        │
                                                      │                        ▼
                                                      └─ warning ──────► Postgres: INSERT dead_letter

Node 1 — Error Trigger. Receives the failure. It needs no configuration; it just delivers the payload.

Node 2 — Code: "Classify failure". Reads the payload and decides the severity based on lesson 4's policy. It's a pure Code node — only logic, no HTTP or Postgres:

// ============================================================
// Node: Code — "Classify failure" (in cumbre-error-handler)
// Mode: Run Once for Each Item
//
// INPUT:   the Error Trigger's payload
// OUTPUT:  a flattened, classified item, ready to log/alert on
// WHY:      translate the technical failure into lesson 4's
//           business decision (which severity, which workflow, which context)
// ============================================================

const payload = $json;

// Fields might be missing if the failure happened in the trigger: read
// them carefully, tolerating absences, so the handler doesn't crash.
const workflowName = payload.workflow?.name ?? 'unknown';
const failedNode   = payload.execution?.lastNodeExecuted ?? 'unknown';
const errorMessage = payload.execution?.error?.message ?? 'no message';
const executionId  = payload.execution?.id ?? null;

// The order_id travels inside the data the workflow was processing.
// Depending on how each workflow exposes it, it can be in different
// places; here we try the expected location and leave null if absent.
const orderId = payload.execution?.error?.context?.order_id ?? null;

// Lesson 4's policy, turned into code: issue-refund is always critical;
// everything else is a warning unless the system is down.
const isRefund = workflowName === 'issue-refund';
const severity = isRefund ? 'critical' : 'warning';

return {
  json: {
    order_id: orderId,
    source_workflow: workflowName,
    failed_node: failedNode,
    error_message: errorMessage,
    execution_id: executionId,
    severity,                       // decides which way the Switch routes it
  },
};

Node 3 — Switch by severity. A Switch node routes based on the severity field: critical items go to the alert branch, and all of them — critical and warnings — end up in the queue. (I simplified in the diagram; in practice a critical item gets alerted on and also saved to the queue, because alerting doesn't excuse you from not losing the item.)

Node 4a — HTTP Request: alert (critical branch only). Calls Cumbre's alert channel API — remember, the call is an HTTP Request, not a Code node — with a useful message per lesson 4: what failed, the order_id, the link to the execution, what was already attempted. This node carries its own Retry On Fail: it would be ironic for the alert about a failure to fail silently.

Node 4b — Postgres: INSERT into dead_letter. Stores the item in the queue with its full context. An INSERT with the fields the Code node prepared.

An order detail worth thinking about: it's a good idea for the INSERT into dead_letter to happen before — or at least with the same priority as — the attempt to alert. The reason is the hierarchy of guarantees. Saving to Postgres is a local, reliable write that almost never fails; sending an alert depends on an external service that might really be down. If you alerted first and the save came afterward, a save failure would leave you with an alert sent but no item in the queue — you know something failed, but you lost the order. Saving first, even if the alert fails, the item is safe and you can always alert later. The general rule: first make sure nothing is lost, then notify. Losing the notice is annoying; losing the order is the failure this entire module exists to prevent.

What to expect end to end. When issue-refund exhausts its retries trying to issue a refund, the execution fails, n8n starts cumbre-error-handler, the Code classifies the failure as critical, the Switch sends it to the alert branch and the queue: finance gets an alert with the order_id and the link to the execution, and the order stays saved in dead_letter with status = 'pending'. When instead order-triage rejects a malformed order, the same handler classifies it as warning, alerts no one, and saves it to the queue for same-day review. One error workflow, two different behaviors, governed by the policy you designed in lesson 4.

Reprocessing from the queue without duplicating

Storing the item is half the story; the other half is taking it back out. And here, for the last time in the module, the golden rule comes back: reprocessing an item from the dead-letter queue re-executes its effects, so it has to be idempotent.

Think about it. A refund ended up in the queue because the payments API was down. Hours later, the API comes back, and someone reprocesses the item. If the API had actually already received the original request before going down — lesson 2's world B — reprocessing without idempotency issues a second refund. The dead-letter queue isn't an exception to the module's rules: reprocessing is, exactly, a manual, late retry, and plays by the same rules as any retry.

That's why reprocessing leans on the same two protections as always: the idempotency key in the request (so the API recognizes the duplicate) and the state — Module 4's ledger and the queue's status column — to avoid reprocessing the same thing twice. The reprocessing flow is:

1. A human (or a maintenance workflow) reads the pending items from the queue.
2. For each one, relaunches processing with the same idempotency key it originally had — that's why the full payload was stored: the key is right there.
3. If it succeeds, marks the item as reprocessed. If the human decides not to process it, marks it as discarded.

A detail that closes the loop with Module 2: the idempotency key has to be derived from stable data of the order, not from the moment of the reprocess. If the key depended on the time, reprocessing would generate a new key and the protection would collapse. That's why, since Module 2, keys are derived from the order_id and the operation type: they survive a reprocess that happens hours or days after the original failure. The whole module rests on that decision.

Common mistakes

Trying to test the error workflow by running it manually (practical). What happens: cumbre-error-handler gets finished, someone hits run to test it, and nothing useful happens — the Error Trigger has no real failure to process. It gets concluded "it doesn't work" and an afternoon gets wasted reviewing something that's fine. Why it happens: it's the natural reflex, and the documentation warns exactly about this: the Error Trigger only fires from a real failure in an automatic execution. How to detect it: if you're running the error workflow directly and expecting to see the error handling, this is the case. How to fix it: test by causing a real failure in a workflow that has this error workflow assigned — for example, a node pointed at an invalid URL on purpose — let it fail during an execution, and watch the handler fire. That's the only way to actually see it work.

Storing only the order_id in the queue, not the full order (conceptual). What happens: to "save space," only the order's identifier gets stored in dead_letter, not its whole payload. When someone goes to reprocess, it turns out the original event isn't available anymore — the webhook doesn't repeat — and there's no way to reconstruct the data that was being processed with. The stored item is useless. Why it happens: storing everything feels redundant if the order "already exists somewhere." How to detect it: ask yourself "if I had to reprocess this three days from now, do I have here everything I need, with no external dependency?" If the answer is no, you're missing context. How to fix it: store the item's full payload. The dead-letter queue exists precisely so you don't depend on the source still being available; storing only a reference breaks exactly that guarantee.

Reprocessing the queue with no idempotency (conceptual). What happens: a batch of refunds that failed because of a payments API outage gets gathered, the API comes back, and everything gets reprocessed "to catch up," with no idempotency key check. Some of those refunds had actually been issued before the outage — only the confirmation had been lost — and now they get issued again: money going out twice. Why it happens: the queue feels like "what was left pending," and pending suggests "didn't happen." But a terminal failure doesn't guarantee the effect didn't happen; it guarantees it wasn't confirmed. How to detect it: before reprocessing in bulk, ask yourself whether every item carries its original idempotency key and whether the API respects it. How to fix it: reprocess with the original key stored in the payload, and check the ledger to skip whatever's already marked as done. Reprocessing is a late retry, with all the risks of a retry.

Exercises

Exercise 1 — Read the payload. This payload arrives at Cumbre's Error Trigger. Say which workflow failed, on which node, with what message, and what severity you'd assign per lesson 4's policy:

{
  "execution": {
    "id": "1187",
    "url": "https://cumbre.n8n/workflow/refund/executions/1187",
    "error": { "message": "Payments API timeout after 3 retries", "stack": "…" },
    "lastNodeExecuted": "Emit refund",
    "mode": "trigger"
  },
  "workflow": { "id": "refund", "name": "issue-refund" }
}

See solution

• Which workflow failed: issue-refund (from workflow.name).
• On which node: Emit refund (from execution.lastNodeExecuted).
• With what message: "Payments API timeout after 3 retries" (from execution.error.message). Note the message already says "after 3 retries": lesson 2's retries got exhausted, so this is terminal, not transient.
• Severity: critical. It's issue-refund, which in lesson 4's policy is always critical because it involves money. It goes to the finance alert branch and to the dead-letter queue.

Additionally, you'd store execution_id = "1187", which is what will let you, in lesson 6, load exactly this execution into the replay engine to see what happened.

Why this works: the exercise trains you to read the Error Trigger's real payload and translate it, field by field, into the queue's columns and lesson 4's decision. That payload is the raw material of the entire error workflow.

Exercise 2 — Design the columns for a new case. Cumbre wants to be able to answer, by looking at the dead-letter queue, the question: "how many orders have been in pending for more than 24 hours without being reprocessed?" What columns of the dead_letter table do you need to answer it, and what query would you run (in words, not necessarily exact SQL)?

See solution

You need two columns the table already has: created_at (when the item fell into the queue) and status (whether it's still pending).

The query, in words: count the records where status = 'pending' and created_at is older than 24 hours ago. In SQL it would be something like filtering by those two conditions and counting.

What's interesting is why this question matters: an item that's been in pending for more than 24 hours is a signal the queue's manual review isn't happening, or that something is systematically stuck. It's, in itself, an alert candidate — a "the queue is piling up" alert, which is different from the alert for each individual failure. This ties to lesson 4's idea of alerting by rate and not just by event: you don't alert for one item in the queue, but you do for "there are ten items that have gone more than a day unattended."

Why this works: the exercise shows the dead-letter queue isn't just a drawer you throw things into; it's a table you can query to understand the system's health. Designing its columns well — with timestamps and status — is what turns it from a trash can into a tool.

Exercise 3 — The dangerous reprocess. A batch of five refunds fell into the dead-letter queue because of a two-hour payments API outage. The API is back now. Describe the correct flow for reprocessing them without risking double refunds, naming the two protections that make it safe and what column or data each one contributes.

See solution

The correct flow:

1. Read the five items with status = 'pending' and source_workflow = 'issue-refund'.
2. For each one, relaunch the refund issuance using the same idempotency key the original attempt had — stored inside the full payload, which is why it got stored whole.
3. Before or during that, check the ledger to see whether that refund already shows as issued; if so, skip it and mark it reprocessed without calling the API again.
4. When each one finishes successfully, update its status to reprocessed.

The two protections:

• The idempotency key (provided by the payload stored in the queue): protects on the API's side. If any of those five refunds had actually been issued before the outage — world B: effect done, confirmation lost — the API recognizes the repeated key and doesn't issue a second refund.
• The state (provided by Module 4's ledger and the queue's status column): protects on the system's side. It avoids reprocessing something already resolved, and keeps a record of what got reprocessed.

The risk being avoided: without the key, reprocessing "to catch up" would issue a second refund for any of the five that had already gone out, because a terminal failure doesn't guarantee the effect didn't happen — only that it wasn't confirmed.

Why this works: reprocessing from the dead-letter queue is lesson 2's retry taken to its latest form, done by hand hours later. Whether it can be done safely depends entirely on the keys having been derived from stable data since Module 2. The whole module rests on that one early decision.

Summary and next step

In this lesson you built the two pieces that catch what everything else couldn't resolve. You saw the Error Trigger node as the first node of a central error workflow — cumbre-error-handler — that works like a switchboard: a single funnel where failures from all four workflows arrive, configured in each one's Options > Settings > Error workflow, with the documentation's details in mind (a workflow with an Error Trigger handles itself, doesn't get tested manually, and it's worth assigning it to every workflow). You broke down the real payload it receives — workflow.name, execution.lastNodeExecuted, execution.error.message, execution.id and its link — and turned it, with a pure-logic Code node, into lesson 4's severity decision. And you set up the dead-letter queue: the dead_letter table in the ledger's Postgres, with the full payload and the status, so no item gets lost silently. You closed with the usual rule: reprocessing from the queue is a late retry, and it's only safe with the stable idempotency key and the state in the ledger.

Before moving on you should be able to: configure an error workflow and explain why it doesn't get tested by running it manually; name the Error Trigger's payload fields you'd use for logging and alerting; and explain why the dead-letter queue stores the full order and why reprocessing it needs idempotency.

What you haven't seen yet is how you investigate a failure when the error message isn't enough. You saved each failure's execution.id in the queue — and that identifier is a key. Lesson 6 uses it: with n8n 2.0's debugging engine you're going to load the data of a real execution that already happened, run it again step by step, and follow the idempotency key through every node until you see exactly where a second refund got created. It's the tool that turns the worst kind of bug — the one that shows up one time in a hundred and vanishes when you go looking for it — into a reproducible, fixable one.

Resources

• Error Trigger — n8n Docs — the node's official page, with the exact structure of the payload it receives when a node fails and the variant for when the trigger fails.
• Error handling — n8n Docs — how to create an error workflow, assign it in the workflow's settings, and the notes that it handles itself and doesn't get tested manually.
• Handle errors gracefully — n8n Docs — error-handling design guide that the central error workflow pattern comes from.
• Postgres node — n8n Docs — the node used to create and write the dead_letter table on the Starter Kit's Postgres.
• Switch node — n8n Docs — the node that routes the failure by severity toward the alert or the queue, inside the error workflow.