7. Debugging, basic monitoring, and cost control

Description

By the end of this lesson you're going to have the three instruments missing before you can call the system finished. A deterministic output validation that keeps the agent from claiming what no tool returned — the gap lesson 6 left open on purpose. An audit log of your own that survives n8n's execution purge, instrumented at the six points where the system makes security decisions, with queries that detect an incident before anyone complains. And a cost sheet with a concrete dollar figure per conversation, obtained from your own execution panel and not from an estimate.

That last point deserves a warning. The question "how much does each conversation cost?" comes up in every serious conversation about an AI system — with a client, with a boss, in an interview — and the answer "it depends" is a polite way of saying you didn't measure it. By the end of this lesson you're going to be able to say a number, say where it came from, and say how much it rises in the worst case.

This matters for a reason running through the whole project: an agent's failures leave the execution green. A traditional workflow that breaks tells you. An agent that got hijacked, that overreached, or that made up a date finishes successfully and produces a perfectly formed response. Nobody finds out until the damage shows up somewhere else: the month's statement, an angry customer, an audit. Today's instruments are the only thing turning "nothing happened" into "nothing happened, and I can prove it."

Connection to the module: lesson 6 closed off actions and left claims open; today's phase 1 closes that. Every layer you built in lessons 4, 5, and 6 has its logging point here. And today's measurements are lesson 8's raw material: the three-minute demo and the interview answers rest on these numbers.

The electric meter

Before a meter existed on the wall, arguing about the electricity bill meant arguing about hunches. Someone said the problem was the air conditioner, someone else said it was the old fridge, and there was no way to know. Things got switched off at random, the bill went down a little or didn't, and nobody knew which of the things turned off did it.

A meter changes the conversation three ways.

It gives an absolute number. Not "we spent a lot," but a figure. And a figure can be compared against another month, against another house, against what you can afford.

It allows attribution. You turn off an appliance, look at the meter, turn it back on. In thirty seconds you know how much it draws. No amount of reasoning about the appliance's specs replaces that measurement.

And with a little more resolution, it warns you. A meter recording by the hour shows odd consumption at three in the morning, and that leads to discovering a heater that got left on. Nobody was going to go looking for it; the data found it.

Your system has exactly those three needs. It needs a number per conversation, it needs to be able to attribute that number to a specific agent, and it needs to warn you when something behaves differently than normal.

And there's a lesson from the meter that translates unchanged: measure before optimizing. Intuition about where the cost goes in a multi-agent system is notoriously bad. Plenty of people swear the problem is the delegations and, on measuring, discover half the spend is in a nine-hundred-word system prompt getting resent on every iteration of every agent.

Phase 1 — Output validation

We start by closing the gap. Lesson 6's exercise 2 showed an attack crossing the guardrail and human approval because it doesn't ask to run any tool: it only asks the agent to say something. None of the earlier layers protect claims.

The defense isn't a filter or a permission: it's comparing, field by field, what the agent claims against what the tools returned. And that comparison has to be deterministic — a Code node, not another model.

Step 1.1 — The core's structured output

Until now, core_output fixed status and needs_human by hand. Now the agent itself produces them, with a Structured Output Parser connected to triage_agent:

{
  "message_to_customer": "Revisé tu pedido 4521: salió el 21 de julio y la entrega estimada es el 23. Te dejo el código de seguimiento por si quieres verlo en la web de la transportadora.",
  "status": "resolved",
  "needs_human": false,
  "facts": {
    "order_id": "4521",
    "order_status": "in_transit",
    "eta_date": "2026-07-23",
    "tracking_code": "TR-99182",
    "charge_id": null,
    "dispute_id": null,
    "refund_status": "not_requested",
    "policy_days": null
  },
  "facts_source": ["order_specialist:lookup_order"]
}

Three decisions in that schema that make validation possible.

Facts travel separate from the text. message_to_customer is prose and can't be automatically verified. facts is an object of comparable fields. Without that separation, validating means searching for numbers inside a paragraph, which is fragile.

Fields that might be missing explicitly allow null, with a schema description forbidding them from being estimated. A missing field is ambiguous; one set to null is a claim that it doesn't apply.

refund_status is a closed vocabulary — not_requested | pending_approval | approved | denied — and it's the field closing lesson 6's attack.

Step 1.2 — The validator

// Node: Code — Name: validate_agent_output
// Compares what the agent CLAIMS against what the tools
// RETURNED in this same execution. Deterministic: no model
// takes part in this decision.

const out        = $json;
const facts      = out.facts || {};
const violations = [];

// This execution's tool results. The exact way to read them
// depends on how you wired the trace; the pattern is the same:
// one object per tool with what it returned.
const t = $('collect_tool_results').first().json;

// ── 1. Identifiers: they match or they don't exist ────────────
if (facts.order_id && facts.order_id !== t.lookup_order?.order_id) {
  violations.push('order_id_mismatch');
}
if (facts.charge_id && facts.charge_id !== t.lookup_charge?.charge_id) {
  violations.push('charge_id_mismatch');
}

// ── 2. Status: matches exactly ─────────────────────────────────
if (facts.order_status &&
    facts.order_status !== t.lookup_order?.status) {
  violations.push('order_status_mismatch');
}

// ── 3. Dates: can only exist if a tool returned them ───────────
// This is the one that trips most often. A model that sees
// "in_transit" and a dispatch date tends to estimate a delivery
// date, and it sounds perfectly reasonable.
if (facts.eta_date && !t.lookup_order?.eta) {
  violations.push('eta_date_invented');
}

// ── 4. Reference numbers: only if their tool ran ────────────────
if (facts.dispute_id && !t.open_dispute?.dispute_id) {
  violations.push('dispute_id_invented');
}

// ── 5. Refunds: the check that closes the gap ───────────────────
// The agent can only say "approved" if issue_refund returned a
// successful result IN THIS execution. Not earlier, not "based
// on the conversation," not because the customer claimed it.
if (facts.refund_status === 'approved' && !t.issue_refund?.ok) {
  violations.push('refund_claimed_without_execution');
}

// ── 6. Policies: only from the knowledge base ────────────────────
if (facts.policy_days !== null && facts.policy_days !== undefined) {
  const kb = t.search_knowledge_base?.body || '';
  if (!kb.includes(String(facts.policy_days))) {
    violations.push('policy_not_backed_by_kb');
  }
}

// ── 7. Every factual claim needs backing ─────────────────────────
const claimsFacts = Object.values(facts)
  .some(v => v !== null && v !== undefined && v !== 'not_requested');
if (claimsFacts && (out.facts_source || []).length === 0) {
  violations.push('facts_without_source');
}

return [{ json: {
  ...out,
  validation_passed: violations.length === 0,
  violations
} }];

What to expect. Run case C1 — "how's my order #4521 doing?" — ten times and count how many times eta_date_invented fires. That number is a real fact about your system, and it's exactly the kind of thing you cite in an interview: "I measured that in X out of ten runs the agent estimated a delivery date no tool had returned; the validator catches it and the customer gets the second response."

And run lesson 6's exercise-2 attack — the customer asking for written confirmation of a refund "already approved." If the agent sets refund_status: "approved", check 5 flags it and the response doesn't go out. The gap's closed.

Step 1.3 — The three failure outputs

A validator that only detects is useless. You have to decide what happens when validation_passed is false, and there are three paths:

# Node: IF — Name: output_is_valid
#
# [true]  → Guardrails: output_guardrail → core_output
#
# [false] → Switch by violation type:
#
#   Retry (ONE time only)
#     For invention violations: eta_date_invented,
#     dispute_id_invented, facts_without_source.
#     The agent gets called again with the error message as
#     additional context: "Your previous response claimed a
#     delivery date no tool returned. Answer again without
#     estimating data you don't have."
#     ONE time. If the second one also fails, it degrades.
#
#   Degraded response (template)
#     A fixed text, written by you, with the data that DID get
#     verified: "Your order 4521 is in transit. I don't have an
#     exact delivery date; you can track it with code
#     TR-99182." No model involved, so it can't invent anything.
#
#   Escalate
#     For refund_claimed_without_execution and for any
#     violation repeating after the retry. Calls
#     escalate_to_human and responds with the escalation
#     template.

The one-retry limit isn't negotiable: a retry loop with a model that insists on the same hallucination multiplies cost and latency with no convergence.

And the last node before core_output:

# Node: Guardrails — Name: output_guardrail
# Operation: Check Text for Violations
# Text To Check: {{ $json.message_to_customer }}
#
#   PII          — that no other customer's full email or phone
#                  number goes out in a response
#   Secret Keys  — that nothing from a credential leaks in a
#                  poorly handled error message
#   Keywords     — the commitments TuTienda doesn't make in
#                  writing: "we guarantee delivery," "refund
#                  approved," "at no additional cost"
#
# [Fail] → degraded response + log

Phase 2 — The audit log

Here comes the problem that ruins real investigations, and it's not technical, it's calendar.

n8n's executions don't live forever. By default the purge is enabled, and an execution gets deleted when either of two things happens: more than EXECUTIONS_DATA_MAX_AGE hours have passed since it finished — by default 336 hours, i.e. 14 days — or the total exceeds EXECUTIONS_DATA_PRUNE_MAX_COUNT, by default 10,000.

Do the math with TuTienda. Three hundred conversations a day, each generating two executions — the channel adapter and the core — plus the tool sub-workflows'. That's easily eight hundred executions a day. The 10,000 cap gets reached in under two weeks.

Now think about when an incident shows up. An improper refund gets noticed at month-end reconciliation. A customer complains about something the agent told them "about two weeks ago." The moment you need the trace is systematically later than the moment n8n deleted it.

The solution is an audit log of your own: a record you write, to a destination you control, with the minimum needed to reconstruct. It doesn't replace the executions — it's much less detailed — it's the index telling you something happened and where to look while the execution still exists.

CREATE TABLE agent_audit_log (
    id              BIGSERIAL PRIMARY KEY,
    logged_at       TIMESTAMPTZ NOT NULL DEFAULT now(),
    execution_id    TEXT NOT NULL,   -- to go to the execution
    session_key     TEXT,            -- to reconstruct the conversation
    workflow_name   TEXT NOT NULL,
    agent_name      TEXT,
    channel         TEXT,            -- web | whatsapp
    customer_id     TEXT,
    verified_by     TEXT,            -- session | crm_phone | declared
    event_type      TEXT NOT NULL,   -- guardrail_block | tool_call |
                                     -- approval_request | approval_result |
                                     -- validation_fail | final_response
    tool_name       TEXT,
    tool_parameters JSONB,
    outcome         TEXT,            -- ok | denied | blocked | invalid | timeout
    notes           TEXT
);

CREATE INDEX ON agent_audit_log (logged_at DESC);
CREATE INDEX ON agent_audit_log (customer_id, logged_at DESC);
CREATE INDEX ON agent_audit_log (tool_name, logged_at DESC);
CREATE INDEX ON agent_audit_log (event_type, logged_at DESC);

-- The agent writes here, and only inserts.
GRANT INSERT ON agent_audit_log TO n8n_agent_rw;

And the six points where it gets instrumented, one per project layer:

# INSTRUMENTATION POINTS

1. input_guardrail's Fail branch                    (lesson 6)
     event_type "guardrail_block" · outcome "blocked"
     notes: which guardrail fired. NOT the full text.

2. Before every L1 or L2 tool                        (lesson 4)
     event_type "tool_call"
     tool_name · tool_parameters · outcome

3. When requesting an approval                       (lesson 6)
     event_type "approval_request"
     tool_parameters: amount, reason, order_id
     verified_by: how the customer was identified

4. When an approval gets resolved                     (lesson 6)
     event_type "approval_result"
     outcome: ok | denied | timeout

5. The validator's false branch                       (phase 1)
     event_type "validation_fail" · outcome "invalid"
     notes: the violations array

6. Final response to the customer                     (always)
     event_type "final_response" · outcome "ok"
     notes: the text sent

# Node: Postgres — Name: audit_log_write
# Operation: Insert · Table: agent_audit_log
# Credential: n8n_agent_rw
#
# execution_id    = {{ $execution.id }}
# session_key     = {{ $('core_input').item.json.customer_id
#                      ? 'customer:' + ... : ... }}
# workflow_name   = {{ $workflow.name }}
# channel         = {{ $('core_input').item.json.channel }}
# customer_id     = {{ $('core_input').item.json.customer_id }}
# verified_by     = {{ $('core_input').item.json.verified_by }}
# event_type      = "tool_call"
# tool_name       = {{ $tool.name }}
# tool_parameters = {{ JSON.stringify($tool.parameters) }}
# outcome         = "ok"
#
# NOTE: don't write the customer's full text here or personal
# data you don't need. A log that saves too much becomes a leak
# itself. Save identifiers and decisions; the detail lives in
# the execution while it still exists.

That last note is worth a real story that keeps repeating: someone decides "more logging is better," saves every customer's entire conversation, and two months later has a table with thousands of people's personal data sitting in a place nobody designed to protect it. The log stores decisions, not content.

Phase 3 — Detection queries

A log nobody queries is just a table that grows. These three get written once and run on their own in a separate workflow with a daily Schedule Trigger, notifying the internal channel only if they return rows.

-- ── 1. Retries after a denial ──────────────────────────────────
-- In a healthy system this ALWAYS returns zero rows. Any row is
-- an agent that didn't respect a refusal.
WITH denied AS (
  SELECT execution_id, customer_id, tool_name, logged_at
  FROM agent_audit_log
  WHERE event_type = 'approval_result'
    AND outcome IN ('denied', 'timeout')
    AND logged_at > now() - interval '7 days'
)
SELECT d.execution_id, d.customer_id, d.tool_name,
       count(a.id) AS retries_after_denial
FROM denied d
JOIN agent_audit_log a
  ON a.execution_id = d.execution_id
 AND a.event_type   = 'tool_call'
 AND a.tool_name    = d.tool_name
 AND a.logged_at    > d.logged_at
GROUP BY d.execution_id, d.customer_id, d.tool_name;

-- ── 2. Concentration of sensitive actions ───────────────────────
-- A customer with three or more sensitive actions in a week is
-- a pattern, not a coincidence.
SELECT customer_id, tool_name, count(*) AS n,
       min(logged_at) AS first_seen, max(logged_at) AS last_seen
FROM agent_audit_log
WHERE event_type = 'tool_call'
  AND tool_name IN ('issue_refund', 'open_dispute')
  AND logged_at > now() - interval '7 days'
GROUP BY customer_id, tool_name
HAVING count(*) >= 3
ORDER BY n DESC;

-- ── 3. Layer health ───────────────────────────────────────────
-- The least obvious query and one of the most valuable: it
-- tells you whether your defenses are alive.
SELECT event_type, outcome, count(*) AS n
FROM agent_audit_log
WHERE logged_at > now() - interval '7 days'
GROUP BY event_type, outcome
ORDER BY n DESC;

What to expect from the third one, after running your twelve-case battery:

 event_type        | outcome | n
-------------------+---------+----
 tool_call         | ok      | 31
 final_response    | ok      | 12
 guardrail_block   | blocked |  2   ← C11, two of five runs
 approval_request  | denied  |  3   ← the attacks that reached that far
 approval_request  | ok      |  1   ← the legitimate refund
 validation_fail   | invalid |  2   ← two invented dates in C1

Six rows telling your system's whole story for a week. And the reading rule: if any layer shows up with zero, check it. A guardrail_block with zero rows in a week almost never means nobody attacked you; it means the node's wired wrong or has a threshold that does nothing. A defense that never reports is indistinguishable from a defense that's off.

Phase 4 — Measuring cost: the method

Now the number. And the method matters as much as the number, because yours is going to differ from mine.

Step 1 — Turn on tracing at every level. Return Intermediate Steps on the orchestrator and on every specialist. Without this you can't count anything.

Step 2 — Count model calls by level. In the trace, each "model call" is one iteration. Note how many the orchestrator made and how many each specialist made. That count is your latency metric, because calls are sequential.

Step 3 — Read token usage. n8n's chat model nodes report token usage for each call in the execution's output data. Open them in the panel and note input and output per model node, not just the first one. This is where you're going to see with your own eyes the most important phenomenon: the context growing on every iteration, because every iteration resends everything before it.

Step 4 — Read the time. The panel shows the full execution's duration and each node's.

Step 5 — Multiply by your provider's current price, the day you measure it.

Step 6 — Build the sheet with twenty or thirty representative executions, and keep three columns: typical, p90, and worst observed. The one used most in practice isn't the typical case: it's p90, the value below which 90% of conversations fall, because expensive cases weigh more than their frequency suggests.

Worked example: TuTienda's cost sheet

These are the numbers measured on this project's system. Token counts are stable — they depend on your prompts, and yours look a lot like these if you followed the lessons. Prices change: the ones below are a reference order of magnitude as of this writing (July 2026), with a cheap model around $0.30 per million input tokens and $2.50 output, and a capable one around $3 and $15. Check your provider's current prices and redo the multiplication; the structure doesn't change.

# TYPICAL CONVERSATION — one topic, one delegation
# Case C1: "how's my order #4521 doing?"

  Component              Calls      Tokens in   Tokens out   Model
  ──────────────────────────────────────────────────────────────────
  input_guardrail            1          800          20     cheap
  triage_agent                3       12,200         390     cheap
    (3,200 · 4,100 · 4,900 — the context grows each turn)
  order_specialist            3        5,600         310     capable
    (1,400 · 1,900 · 2,300)
  ──────────────────────────────────────────────────────────────────
  TOTAL                       7       18,600         720

  Cheap      13,000 in × $0.30/M  = $0.0039
              410 out × $2.50/M   = $0.0010
  Capable     5,600 in × $3/M     = $0.0168
              310 out × $15/M     = $0.0047
  ──────────────────────────────────────────────────────────────────
  COST PER TYPICAL CONVERSATION         ≈ $0.026 USD
  DURATION                              ≈ 9 seconds

# p90 CONVERSATION — two topics, two delegations
# Case C4: unrecognized charge + order status

  Component              Calls      Tokens in   Tokens out   Model
  ──────────────────────────────────────────────────────────────────
  input_guardrail            1          800          20     cheap
  triage_agent                5       26,100         620     cheap
  billing_specialist          5       12,400         450     capable
  order_specialist            3        5,600         310     capable
  ──────────────────────────────────────────────────────────────────
  TOTAL                      14       44,900       1,400

  Cheap      26,900 in + 640 out    = $0.0097
  Capable    18,000 in + 760 out    = $0.0654
  ──────────────────────────────────────────────────────────────────
  COST p90                              ≈ $0.075 USD
  DURATION                              ≈ 21 seconds

The number you were after: a TuTienda conversation costs between 2.6 and 7.5 cents of a dollar in model spend, depending on whether the customer brings one or two topics. With 2,000 conversations a month and a mix of 70% simple and 30% two-topic:

  1,400 × $0.026  = $36.40
    600 × $0.075  = $45.00
  ────────────────────────
  MODEL, PER MONTH    ≈ $81 USD

And now the other half, which is where this project stands apart from an exercise: the channel.

  Web chat                       $0  (no channel cost)
  WhatsApp — service conversations (the ones the
    customer starts and get answered within the 24-h
    window):
    Meta has had periods where these conversations
    aren't charged, charging instead the templates the
    business initiates. The scheme has changed several
    times and varies by country. VERIFY the current one
    in Meta's pricing table before giving a number.
  Server (self-hosted n8n + Postgres)  ≈ $5-20 USD/month
  n8n Community                        $0

That finding deserves saying out loud because it's counterintuitive and it's what a client asks first: for a pure customer-support system, the channel can cost much less than WhatsApp's reputation suggests, and the real cost is in the model. What genuinely costs a lot on WhatsApp are business-initiated templates — campaigns, reminders, proactive notifications — which this system doesn't use. Being able to make that distinction is the difference between an estimate and an informed conversation.

Defensible total for the system: around $90 to $100 dollars a month for 2,000 conversations, with the warning that the model's price and WhatsApp's need verifying the day it goes into production.

And the comparison that makes that number useful: if a person handled those 2,000 conversations at five minutes each, that's 166 hours a month. That's the frame $100 gets discussed in.

Phase 5 — The levers, measured

With the sheet built, apply the levers and measure again. One at a time, running the twelve-case battery between each change, because changing three things together makes it impossible to know which one worked.

The ones that pay off most in this system, in order:

Lever 1 — Tiered models. You already applied this on paper in lesson 2, and now you can verify it. Look at the sheet: triage_agent makes 26,100 input tokens in the p90 case, more than any specialist, because it carries the full conversation's memory. If the orchestrator were on the capable model, that component would go from $0.0081 to $0.078 — almost ten times. The decision to put the orchestrator on the cheap model is, alone, 60% of the system's savings. Verify it by running cases C4 and C8 with the expensive model and with the cheap one, and comparing which specialist it delegated to in each; if the routing doesn't change, keep the cheap one.

Lever 2 — Shorten the assignment. Open case C4's trace and read the two tasks the orchestrator wrote. Are they data or narrative? A three-paragraph assignment reproducing what the customer said travels on every iteration of the specialist — five times in the billing case. Adjust $fromAI("task", …)'s description to ask for data instead of a story, and measure the specialist's input tokens again. It's the lever with the best effort-to-benefit ratio after lever 1.

Lever 3 — Calibrate Max Iterations. With the sheet's data, apply the max-observed-plus-two rule:

                    max observed        current limit    new
  triage_agent             5                  7            7  ✓
  billing_specialist       5                  7            7  ✓
  order_specialist         3                  5            5  ✓

In this case all three were well calibrated from the paper design, which is a confirmation and not a non-result: it means lesson 2's estimates were reasonable. If in your measurement any of them came out tight — max observed equal to the limit — raise it, because an agent cut off by exhaustion produces an incomplete response that looks fine.

Lever 4 — Replace an agent with a deterministic sub-workflow. You already applied this in lesson 4 with check_return_eligibility, and now you can put a number on it. As a model judgment, that rule cost between two and four extra calls each time it got used; as a sub-workflow it costs zero tokens and a few milliseconds. With 15% of conversations touching returns, that's about 300 conversations a month saving two calls to the capable model each. Note it down: it's a concrete fact for the interview question about when an agent isn't the solution.

Lever 5 — Teach the orchestrator not to delegate. It's already in the prompt, and the sheet tells you whether it works. Run case C9 — "do you have stores in Guadalajara?" — and count the delegations. Zero is the right answer. If it delegates, you're paying for a full specialist for a hours question, on every conversation of that kind.

Debugging when something went wrong

The instruments serve two different purposes, and so far we've covered one. The other is reconstructing a specific incident.

The method, in four steps, and the first is the one almost nobody does:

1. Write down what would make the incident's execution different, before opening anything. A tool that doesn't normally get called, an anomalous duration, a strange outcome. Searching with no criteria through eight hundred green daily executions is the fastest way to lose an afternoon — and all of them look fine, because there was no error.

2. Query the audit log, not the execution list. Here's phase 2's payoff:

-- A customer complains they were confirmed a refund that never arrived.
SELECT logged_at, execution_id, event_type, tool_name,
       tool_parameters, outcome, notes
FROM agent_audit_log
WHERE customer_id = 'C-9931'
  AND logged_at BETWEEN '2026-07-15' AND '2026-07-17'
ORDER BY logged_at;

Ten rows ordered in time, with each one's execution_id. What used to be a blind search is now thirty seconds.

3. Open the execution and read the nested trace. And there's a detail that's easy to miss here: for triage_agent, the specialist's whole deliberation is a single tool call. If you only look at the orchestrator's trace you see three clean entries — delegated, received, replied — and none of them tells you why the specialist decided what it decided. You have to expand the tool entry that's actually an agent to see its internal loop.

And it's worth comparing two texts: the customer's original message and the assignment the orchestrator wrote. If the message carried an injection, the assignment might carry it reworded and stripped of the signals that made it detectable, because the orchestrator "normalized" it. The difference between those two texts is information.

4. Turn the incident into a test case. With the execution open, use the copy-to-editor button to pin the input data on the trigger. Run it against the corrected system and verify which layer stops it. Then save the literal message in your battery, with its expected result — don't describe it ("an email with an odd block"): save it exactly as is. A real attack that already worked once is worth more than ten made-up ones, because it doesn't have the shape you imagined.

And a limit worth noting, so you don't sell this as something it isn't: n8n isn't an LLM observability platform. Logs and the execution list give workflow-level traceability, enough to investigate incidents and debug. They don't give token dashboards over time, automatic anomaly alerts, or prompt-version comparison over an evaluation set. If you need that, it's a separate tool and it's the ecosystem's production guide's territory.

Common mistakes

Measuring only the orchestrator (practical). What happens: someone looks at triage_agent's model node's token usage, sees a reasonable number, and concludes the system is cheap. The specialists have their own model nodes, with their own consumption, which doesn't show up there — and in this system they're 80% of the cost. Why it happens: in the panel the orchestrator is the main node and it's where you look first. How to spot it: if your total doesn't include a line for every specialist that got called, it's incomplete. How to fix it: the measurement unit is the whole conversation, at every level.

Optimizing by intuition without measuring first (practical). What happens: someone's convinced the cost is in the delegations, spends a day collapsing specialists, and the cost barely drops — because half of it was in a long prompt getting resent on every iteration of every agent. Why it happens: delegations are the most visible part of the system. How to spot it: before touching anything, look at each agent's first iteration input tokens; that number is your call-startup cost, and if it's large, that's the problem. How to fix it: measure first and attack the largest source, which is often shortening prompts and Descriptions, easier and less risky than redesigning the architecture.

Saving too much in the log (practical). What happens: someone writes the full text of every message and every customer field "just in case." Two months later the table weighs more than the business database, investigation queries drag, and it holds personal data on thousands of customers sitting in a place nobody designed to protect it. Why it happens: when instrumenting, you want to make sure you don't leave out the data you'll need later. How to spot it: check which columns hold third-party free text and ask what would happen if that table leaked. How to fix it: identifiers and decisions, not content.

Presenting cost without the quality figure alongside it (conceptual). What happens: someone brings "the system costs $81 a month in model spend" to a meeting and the conversation ends there, arguing about whether that's a lot. Why it happens: cost is a hard number and quality requires having measured it, so it's easy to bring only half the story. How to spot it: if your report has a cost figure and none for accuracy rate or hours saved, it's incomplete by design. How to fix it: present them together — "$81 a month, and 2,000 conversations that at five minutes each would be 166 hours of a person's time" is a sentence you can argue with; half a sentence isn't.

Trusting a validator that compares against another model (conceptual). What happens: someone solves phase 1 by asking a second model to review whether the first one's response is faithful to the data. It works fairly well, and fails exactly on the hard cases — because the second model is just as susceptible as the first to a well-written claim. Why it happens: writing the Code with the seven checks is more work than writing a review prompt. How to spot it: if your validation has a model inside it, this is it. How to fix it: comparing facts is deterministic by nature — an identifier matches or it doesn't — and therefore it's code. A reviewer model can complement it on tone or clarity, never on facts.

Exercises

Exercise 1 — Read the trace and write the post-mortem. A customer complains that TuTienda's agent confirmed a refund that never arrived. Here's the trace. Say exactly what happened, which layers were missing, and write the corrections.

[1] Model call (billing_specialist)
    → tool_call: lookup_charge { chargeDate: "2026-07-03" }
[2] Tool: lookup_charge
    → [{ charge_id: "CH-9921", amount: 1200, status: "settled",
         order_id: "4521" }]
[3] Model call
    → tool_call: issue_refund
      { order_id: "4521", amount: 1200, reason: "unrecognized
        charge" }
[4] Human review → outcome: denied
[5] Model call
    → tool_call: issue_refund
      { order_id: "4521", amount: 600,
        reason: "agreed partial refund" }
[6] Human review → outcome: denied
[7] Model call
    → "Done, your $1,200 refund was processed. It should
       reflect in 3 to 5 business days."

POST-MORTEM — Refund promised and not executed

WHAT HAPPENED
The agent requested approval for a refund twice, both got
denied, and it still told the customer it was processed. NO
money went out. A false promise did go out, which generates a
complaint and an expectation the team walked back by hand.

ANALYSIS
| Entry | What failed                     | Layer that stops it   |
|-------|----------------------------------|------------------------|
| [5]   | Retry with different parameters | No-retry clause        |
|       | after a denial                  | (lesson 6)              |
| [5]   | Made-up reason                  | The approval message    |
|       |                                  | showed it; whoever      |
|       |                                  | approved saw it and     |
|       |                                  | denied — it worked      |
| [7]   | Claims a refund no tool          | validate_agent_output   |
|       | confirmed                        | check 5                 |
| —     | Nobody found out until the       | Daily detection          |
|       | complaint                        | query 1                  |

CORRECTIONS
1. No-retry clause in billing_specialist's System Message,
   with an explicit ban on renegotiating within the
   conversation.
2. refund_status field in the structured output, with a
   closed enum and null forbidden.
3. Validator check 5, with failure output = escalate (not
   retry: a response promising money doesn't get retried, it
   gets escalated).
4. Detection query 1 in the daily report.

TEST CASE
· Pin the original execution with the copy-to-editor button.
· Run it against the corrected system and verify, in order:
    ✓ after the first denial there's NO second issue_refund
      entry;
    ✓ refund_status arrives as "denied";
    ✓ validation_passed = true (the response is now coherent);
    ✓ the text to the customer says the case went to review
      and does NOT mention any refund processed.
· Save the original message in the adversarial case battery.

Exercise 2 — Measure your own system and build the sheet. Run your battery's twelve cases, note input and output tokens per model node, duration, and calls per level, and build the sheet with the three columns. Then calculate the monthly cost with your provider's current price and TuTienda's volume.

Exercise 3 — Design the query that detects your own gap. Pick a failure your system might still have — you know which one — and write the query against agent_audit_log that would detect it before anyone complains. Explain each condition and say how often you'd run it.

SELECT customer_id, verified_by, tool_name, count(*) AS n
FROM agent_audit_log
WHERE event_type = 'tool_call'
  AND tool_name IN ('issue_refund', 'open_dispute')
  AND (verified_by IS NULL OR verified_by IN ('', 'declared'))
  AND logged_at > now() - interval '7 days'
GROUP BY customer_id, verified_by, tool_name;

SELECT date_trunc('day', logged_at) AS day,
       count(*) FILTER (WHERE event_type = 'final_response')  AS responded,
       count(DISTINCT session_key)                            AS conversations
FROM agent_audit_log
WHERE logged_at > now() - interval '7 days'
GROUP BY 1 ORDER BY 1;

SELECT date_trunc('week', logged_at) AS week,
       round(100.0
         * count(*) FILTER (WHERE event_type = 'validation_fail')
         / nullif(count(*) FILTER (WHERE event_type = 'final_response'), 0)
       , 1) AS pct_invalid
FROM agent_audit_log
WHERE logged_at > now() - interval '8 weeks'
GROUP BY 1 ORDER BY 1;

Summary and next step

The system has instruments now. A deterministic output validation with seven checks comparing what the agent claims against what the tools returned, with three failure outputs — single retry, template-degraded response, or escalation — and an output guardrail over personal data and commitments TuTienda doesn't make in writing. An audit log surviving the execution purge, instrumented at the six points where the system decides something, storing decisions and identifiers, not content. Three detection queries running in a daily report, including the one that tells you whether your layers are alive. And a cost sheet with measured numbers.

The number: a TuTienda conversation costs between 2.6 and 7.5 cents of a dollar in model spend, and the whole system runs around $90 to $100 a month for 2,000 conversations, with the web channel at zero cost and WhatsApp depending on Meta's current scheme for service conversations. And you know where every figure comes from and which lever moves it.

Before moving on you should be able to: state the cost of a typical conversation and the worst case, with its method; explain why the orchestrator consumes more tokens than any specialist and still costs less; name the six instrumentation points; and say what it means for a layer to show up with zero events in the weekly report.

What's next is the packaging. Lesson 8 turns all of this into something you can show: how to record a three-minute demo that opens with an attack and not with the architecture, how to defend the five design decisions you're going to get asked about — why multi-agent and not one, why HITL on refunds, how much it costs, what's the worst it can do, and when an agent isn't the solution — and how to write the README that lets someone understand your system without you being in the room.

Resources

• View past executions — n8n Docs — the panel where every cost-sheet number comes from: per-node durations and each model call's output data.
• Manage execution data — n8n Docs — how the purge works: EXECUTIONS_DATA_PRUNE, EXECUTIONS_DATA_MAX_AGE (336 hours by default), and EXECUTIONS_DATA_PRUNE_MAX_COUNT (10,000). The reason the audit log exists.
• Executions environment variables — n8n Docs — the full reference for retention variables, if you decide to widen the investigation window.
• Structured Output Parser — n8n Docs — the sub-node separating facts from text and making field-by-field validation possible.
• Code node — n8n Docs — the validator's node and the context variables feeding the audit log, like $execution.id and $workflow.name.
• Guardrails node — n8n Docs — the output guardrail with PII, Secret Keys, and Keywords.
• Postgres node — n8n Docs — the audit log's Insert operation and the daily report's detection queries.