6. Testing AI Agent workflows at zero cost

Description

By the end of this lesson you will be able to test order-triage's AI Agent node as many times as you want without spending a cent, pointing it at a local Ollama language model (like Llama 3.2 or Mistral) instead of the paid model production uses. You're going to understand why testing an agent is inherently expensive if you don't take precautions, how Module 4's Self-Hosted AI Starter Kit already gives you a local model ready to go, when it's actually worth validating against a cloud model —and why never against a retired one— and how to combine this with lesson 5's pinned data for deterministic agent tests.

This matters because the AI Agent node is the most expensive piece of all of order-triage to test. Every test run calls a language model, and in production that model charges per call. Since testing well means running many times —you adjust the prompt and run, you add a case and run, you check an edge case and run— the cost of testing an agent against a paid model spikes exactly when you need to iterate the most. It's lesson 1's "cost in money" in its sharpest form. The local model brings it to zero.

Connection to the module: this is the checklist's fifth point, and it closes the cost front lesson 4 left open. There you saw a dry run avoiding the CRM write but calling a paid model is only half a dry run; this lesson covers the other half. It leans heavily on Module 4: the Starter Kit you brought up already includes Ollama with a local model, so the infrastructure is ready and here we just use it for testing. And it connects with lesson 5 (pinning the agent's output so you don't call it on every test) and with 7 (once you can run the agent for free, you write assertions on its output without evaluation costing anything).

The cook who rehearses with pantry ingredients

Think of a cook developing a new dish that uses truffle, an extremely expensive ingredient. If they tested the recipe with real truffle every time —and a recipe gets tested twenty, thirty times before it's right— they'd spend a fortune just on failed trials. So they don't. They rehearse the technique —the timing, the proportions, the order of the steps— with a cheap pantry ingredient that behaves similarly: a common mushroom instead of the truffle. They do their twenty trials with mushroom, for free. And only once the recipe is dialed in do they cook one final version with the real truffle, to confirm it also comes out well with the expensive ingredient. Rehearse cheap, validate expensive.

Testing an AI agent is the same thing. The paid cloud model is the truffle: powerful, the one going to production, but expensive on every use. The local Ollama model is the pantry mushroom: not identical, but similar enough to rehearse the logic —is the prompt worded well? does the agent receive the order correctly? does the output have the shape the rest of the workflow expects?— for free and without limit. You do your twenty, hundred, or thousand test runs against the local model, and only once the logic is dialed in do you do a final validation against the real cloud model, in staging. Rehearse cheap, validate expensive.

Let's define the pieces. A language model (LLM, for Large Language Model) is the "brain" the AI Agent node uses to classify the order: you send it the order and a prompt, and it gives you back a decision. A cloud model lives on a provider's server, which charges per call —you call it over the internet, with an API key. A local model lives on your machine: you download it once and it runs right there, no internet and no charge, because your computer provides the compute, not anyone's server. Ollama is the program that makes it easy to download and run local models; it's one of the pieces Module 4's Self-Hosted AI Starter Kit already left installed for you.

Why the local model is free

It's worth understanding why it doesn't cost anything, because it isn't magic. A cloud model charges because every call consumes compute on the provider's server —their GPUs, their electricity, their infrastructure— and they pass that bill to you. A local model runs on your own machine: you provide the compute, with the processor or GPU you already have and already paid for. There's no per-call bill because there's no third party providing the compute. You download the model once —it's a few gigabytes— and from then on you run it as many times as you want with no per-use cost.

There's an honest trade-off, to be clear: a local model is usually smaller and less capable than the best cloud model, and runs slower if your machine is modest. But for testing the workflow's logic —which is what we do in this module— that almost never matters: we're not evaluating how brilliant the agent is, but whether the workflow surrounds it correctly. For that, the mushroom is more than enough.

How much you save, in orders of magnitude

Let's put numbers on it, with the warning that they're hypothetical —model prices change and depend on the provider and each call's size; take them as an illustration of the order of magnitude, not a quote.

Imagine a typical work cycle tuning order-triage's classifier. You have six synthetic cases. Tuning the prompt until it's good takes, say, twenty iterations, and each iteration runs the six cases. That's 20 × 6 = 120 calls to the model, just to tune it. Now let's add regression tests: every time you touch something in the workflow and want to confirm you didn't break the classification, you run the six cases again; in a week of active work, that's easily another 100 runs. You're over 200 calls in a few days, and that's a single developer tuning a single agent.

Approach	Cost of those ~200+ calls	Effect on how you work
Against the cloud model	A charge per call; adds up fast and arrives depersonalized at month's end	You ration testing: "do I really need to run this again?"
Against the local model (Ollama)	Zero per call; only your machine's compute, already paid for	You test without a second thought: run as many times as you want

The point isn't the exact amount —which can be small or large depending on the model— but the behavior change it produces. When every test costs something, even a little, you start rationing them, and testing too little is exactly what this module fights. When testing is free, you test more than enough, which is exactly what you want. The local model doesn't just save money: it removes the mental brake making you test less than you should.

The Starter Kit already gives you the local model

Good news from Module 4: you don't have to install anything new. The Self-Hosted AI Starter Kit you brought up includes, in its Docker Compose, Ollama alongside n8n, and pre-configures a local model —llama3.2— that downloads on its own on the first startup. So on your test environment you already have a model running, free, waiting for you to use it.

If you want another model —for example, mistral, which many prefer for classification tasks— it gets downloaded with an Ollama command:

# Downloads the mistral model to your local Ollama. Done once.
# 'pull' means "bring/download"; the model stays saved on your machine.
ollama pull mistral

What to expect: Ollama downloads the model (a few gigabytes, takes a while depending on your connection) and leaves it available locally. From then on, your n8n instance can use it without downloading it again and with no per-call cost. Whichever models you have downloaded are the ones you'll be able to choose between in the node.

Version note: the Starter Kit pre-configures llama3.2 as of this guide's writing (2026). The default model and the exact names change with the kit's versions; check your Docker Compose and your Ollama for which ones you have downloaded. What's stable is the pattern —Ollama runs local models for free— the model name of the day, confirm it yourself.

About speed, and to set expectations up front: every machine is different. If you have a GPU, the local model responds fast; if you're only running with a processor (the Starter Kit's cpu profile), each classification can take several seconds instead of one. That's fine. For testing the logic —our thing here— that slowness doesn't matter: you're not serving live orders, you're iterating on your machine, and a few seconds per run is a tiny price compared to paying per call. If the default model runs too slow on your machine, download a smaller one (Ollama offers different-sized variants) and use it for iterating; save the big model, or the cloud one, for final validation. The usual rule: if your machine behaves differently from what the guide says, adjust the model to your hardware; the pattern doesn't change.

Pointing the AI Agent at the local model

Now the concrete part: making order-triage's AI Agent node use the local model instead of the cloud one.

In n8n, the AI Agent node doesn't have the model built in; it connects to a chat model sub-node telling it which "brain" to use. A sub-node is a helper node that plugs in underneath another to give it a capability —here, the model. In production, that sub-node is a cloud model. To test for free, you connect the Ollama Chat Model sub-node instead, which points at your local Ollama.

The Ollama Chat Model sub-node has, among others, these fields (check the exact names on your version):

• Model — the local model to use; you choose between the ones you have downloaded in Ollama (for example llama3.2 or mistral).
• Sampling Temperature — how much randomness the model injects. It's key for deterministic testing, and I come back to it in a moment.
• A credential pointing at your Ollama server (its local address). Since Ollama runs on the same Docker Compose as n8n, that address is internal to the kit; check the exact value in the Starter Kit's docs.

Worked example: iterating order-triage's prompt for free

Let's see it. You want to tune the prompt the agent uses to classify orders, and for that you're going to run it many times against your six synthetic cases.

Step 1 — Connect Ollama. On your dev instance, you connect the Ollama Chat Model sub-node to order-triage's AI Agent node, with the Model field set to llama3.2 (or mistral).

Step 2 — Pin the input. You pin the test order at the Webhook (lesson 5), so every run uses the same order.

Step 3 — Iterate. You run. The agent classifies using the local model. You read the output, adjust the prompt, run again. Adjust, run. Ten, twenty, fifty times.

What to expect: every run classifies the order in a couple of seconds (or a few more, depending on your machine), and —the important part— the bill doesn't move. You can run the agent fifty times tuning the prompt and at month's end there isn't a single charge for those runs, because your machine provided the compute. Compare that with the cloud model: fifty test runs would be fifty billed calls, and that's for just one case; with six cases and several iterations, the bill grows fast. The local model turns "testing the agent" from something you ration because of its cost into something you do without a second thought.

The two-versions problem (and how to handle it honestly)

Here's a real tension I'm not going to hide from you, because it's this lesson's trickiest point. Throughout the guide we've insisted the same workflow, byte for byte, runs across every environment, and that it's the environment —not the workflow— that changes the behavior (the CRM credential in lesson 2, the environment gate in lesson 4). But the chat model is a sub-node inside the workflow, and the Ollama sub-node and a cloud model's sub-node are different node types. Switching from one to the other isn't changing a credential: it's changing a piece of the workflow. So, how do I test with Ollama without my test workflow diverging from the one going to production?

There are two honest ways to handle it, and it's worth knowing both:

Way 1 — Ollama as a temporary development action, with final validation against the real model. You connect Ollama in dev only while you iterate —it's a work action, not a change you commit. The workflow you version and promote keeps the production model's sub-node. Before promoting, you do one validation pass in staging against the real cloud model, to confirm the logic you tuned with the mushroom also works with the truffle. It's exactly the cook's pattern: rehearse with Ollama (cheap, many times), validate with the real model (expensive, once). The key discipline: what gets promoted uses the production model, and it passed at least one validation against it.

Way 2 — A single node pointing at both, via configuration. Ollama exposes an endpoint compatible with the OpenAI API. That opens the possibility of using a single model sub-node whose base URL points, per environment, at your local Ollama (in dev) or the cloud provider (in prod) —the same "one credential per environment" trick from Module 4, applied to the model. With this the workflow stays identical and only the credential changes. Check on your version whether your model sub-node allows overriding the base URL and whether Ollama's compatibility covers what your agent needs; not every node or every function behaves the same way through this path, so confirm it before trusting it.

My practical recommendation, until you confirm Way 2 on your version: use Way 1. It's simpler to reason about and doesn't depend on compatibilities that might change. And in both cases, the golden rule is the same: the artifact reaching production uses the production model, and it was validated against it at least once. Ollama is for cheap iteration, not for replacing the final validation.

When the cloud model is actually worth it (and never a retired one)

The local model doesn't replace the cloud one; it complements it. There are two moments where you do want the cloud one, and it's worth being clear on them.

For the final validation before production. A local model behaves differently from the cloud one: it might classify a borderline order differently, phrase things differently, understand a subtle instruction worse. If you tuned everything with the mushroom, you have to confirm with the truffle before serving. That final validation —one pass of your cases against production's current model, in staging— is what catches the differences the local model hid. Skipping it is like approving a recipe you never cooked with the real ingredient.

When the real task exceeds the local model. If your agent does something a small model doesn't handle well —complex reasoning, a language the local one handles poorly— you might not even be able to use the local one for iterating, and have to test with the cloud one from the start (watching the cost with pinned data and few runs). It's the less common case, but it exists.

And a hard rule about which cloud model to use when you do use one: never a retired one. Providers retire (deprecate and then shut down) old models over time. Validating your workflow against a model the provider is going to shut down —or already marked obsolete— is building on sand: the day they shut it down, order-triage in production suddenly stops classifying, because it calls a model that no longer exists. When you validate against the cloud, do it against the current model your production is going to use, not an old one "because it's cheaper" or "because I already knew it." A retired model doesn't save money: it's a scheduled failure with a date.

I'm not going to name specific cloud models in this guide, and it's on purpose: current names and versions change fast, and any list I write today would be outdated in months. The rule that is stable: use the current model running your production, check the provider's docs that it isn't marked as retired or on its way to being retired, and treat "which model" as a volatile fact you confirm per cohort, not something fixed in the guide.

Deterministic agent tests: temperature and pinned data

An agent is, by nature, something unpredictable: on the same input it can respond differently across two runs (you saw this at the end of lesson 5). That fights against reproducibility. You have two levers to tame it, and they combine.

Lower the temperature. The sub-node's Sampling Temperature field controls how much randomness the model injects when responding. At high values, the model "improvises" more and varies between runs; at low values —near zero— it tends to give the most stable response to the same input. For testing, lower the temperature: you want the agent to respond as consistently as possible, so that if the result changes, it's from your change and not from its mood. It doesn't make it perfectly deterministic —a language model rarely is entirely— but it reduces the variation a lot.

Pin the agent's output. When you're testing the logic after the agent —the environment gate, the CRM write— you don't need the agent to think again on every run; you need a stable output from it. That's where you pin the AI Agent node's output (lesson 5) with a good response you captured. Double win: the test becomes perfectly reproducible downstream, and along the way you don't even call the model, so the cost is zero even if the sub-node pointed at the cloud. Pinning the agent's output is the most decisive way to test everything coming after it without depending on it.

The rule that ties it all together: pin what you're not testing, to isolate what you are. Testing the agent itself? Don't pin its output (it's what you're evaluating), but run it against Ollama with low temperature so it's cheap and stable. Testing what comes after the agent? Pin its output and forget about it. Each question calls for freezing a different part.

And a privacy consequence that usually goes unnoticed: since the local model runs on your machine, the data you send the agent never leaves your computer. When you test against a cloud model, every test order travels to the provider's server; with synthetic data that's harmless, but if you ever tested with real data (which you shouldn't, lesson 3), you'd be sending it to a third party. The local model closes that door by design: the order enters the agent and stays on your machine. For testing, it's one more peace of mind; for certain environments with strict data rules, it's outright a requirement.

An extra benefit of the local model worth naming: it works without internet. Since Ollama runs on your machine, you can iterate the agent on a plane, at a café with bad wifi, or on a machine with no internet access by security policy. The cloud model demands a connection —and a stable one, because every run is a round-trip call; the local one depends on nothing external. For this module's testing work, that independence is worth more than it looks: testing well means many runs, and you don't want every one of them depending on the network being good. One more reason cheap rehearsal is also robust rehearsal.

Common mistakes

Iterating the prompt against the paid model without thinking about cost (practical and expensive). What happens: someone tunes their agent's prompt by running it over and over against the cloud model, and at month's end is surprised by the bill. Tuning a prompt is dozens of runs; against a paid model, dozens of charges. Why it happens: in dev every run feels "free" because you don't see the charge in the moment; the cost arrives depersonalized, a month later. How to spot it: if you're iterating an agent against a paid model, every run costs, even if you don't feel it. How to fix it: iterate against local Ollama. The Starter Kit already gave it to you; use it. Save the paid model for final validation, not for rehearsal.

Tuning everything with the local model and promoting without validating against the real one (conceptual and dangerous). What happens: someone tunes order-triage perfectly against llama3.2, sees it classifies its six cases correctly, and promotes it to production —where a different cloud model runs, behaving differently. In production, the agent classifies some edge cases differently, and nobody tested it. Why it happens: it's easy to confuse "it works with the mushroom" with "it works." How to spot it: if your only test of the agent was against the local model, and production uses another, you didn't validate what you're promoting. How to fix it: always do a final validation pass against production's current model in staging, before promoting. Rehearse cheap, but validate expensive; don't skip the expensive part.

Validating against a retired model (practical, fails with a due date). What happens: someone picks an old cloud model for validation, "because it's cheaper" or "because I already had it configured," without noticing the provider marked it obsolete. Months later the provider shuts it down, and order-triage in production suddenly stops working. Why it happens: an old model seems equivalent and sometimes costs less; its shutdown date isn't visible in the moment. How to spot it: check the provider's docs for whether the model you use is marked retired, deprecated, or with an end-of-life date. How to fix it: validate and run in production against the current model. "Current" is a fact you check per cohort, not something you fix once and forget.

Exercises

Exercise 1 — Mushroom or truffle? For each situation, say whether the local model (Ollama) or the cloud one is right, and why: (a) tuning the classification prompt by running it 40 times; (b) the final validation before promoting to production; (c) testing the environment gate that comes after the agent; (d) demonstrating to your boss how the agent is going to behave in production.

See solution

(a) Local — 40 tuning runs against a paid model are 40 charges; the local one does them for free, and for tuning the prompt's wording the mushroom is enough. (b) Cloud — the final validation has to be against production's current model, to catch the differences the local one hides; it's the confirmation truffle. (c) Neither, actually: pin the agent's output — if you're testing what comes after the agent, you don't need the agent to think; pin its output (lesson 5) and don't even call the model. If you had to choose one, local; but the correct answer is not calling it. (d) Cloud — if the point is showing production's real behavior, it has to be the real model; the local one would behave differently and the demo would mislead.

Why it works: the decision depends on what you're testing. Tuning logic: local (cheap). Confirming real behavior: cloud (faithful). Testing what's after the agent: pin its output (don't even call it). The question isn't "which model is better?" but "what am I testing and what do I need to freeze?"

Exercise 2 — Design the agent test flow without spending. Write the step-by-step plan for tuning and validating order-triage's classifier while spending as little as possible. Include which model you use at each step and what you freeze.

See solution

A near-zero-cost plan:

1. Pin the inputs. Pin the six synthetic cases (or run them from a Code node) so every iteration uses the same orders (lesson 5).
2. Connect Ollama. Point the AI Agent at local llama3.2 or mistral, with low temperature for stability.
3. Iterate for free. Tune the prompt by running the six cases as many times as needed, against the local model. Cost: zero.
4. Once the logic is tuned, validate expensive once. In staging, switch to production's current model and run the six cases one pass, to confirm everything holds up with the real model. Cost: six calls, once.
5. To test what's after the agent (gate, CRM), pin the agent's output and test without calling any model again. Cost: zero.

Total: near zero, with a single paid validation pass. Compare with tuning everything against the cloud: dozens of paid runs.

Why it works: the spending concentrates at the single moment where fidelity matters —the final validation— and everything else (tuning, testing what's downstream) happens for free with a local model or a pinned output. That's the module's zero-cost pattern applied to the agent.

Exercise 3 — Find the hidden risk. A teammate says: "I tuned the agent with Ollama, lowered the temperature to zero, pinned the inputs, my six cases pass perfectly. Ready for production." What are they missing before promoting?

See solution

They're missing the final validation against the production model. Everything they did is fine —iterating with Ollama, low temperature, pinned inputs, six passing cases— but everything was against the local model. Production is going to use a different cloud model, which might classify the edge cases differently. "Passes with the mushroom" doesn't guarantee "passes with the truffle."

Before promoting they should: in staging, switch to production's current model and run the six cases once. If all six still pass with the real model, then it's ready. If any changes, they found exactly the difference the local model was hiding, and it's better to discover it in staging than in production.

A second, minor detail: they should confirm the cloud model they're going to use is current, not retired, so production doesn't shut down in a few months.

Why it works: this lesson's classic mistake is confusing "I tested it exhaustively" with "I tested it exhaustively against what's actually going to run." Exhaustiveness doesn't help if it was against the wrong ingredient.

Summary and next step

In this lesson you saw why the AI Agent node is the most expensive piece to test —every run calls a model, and testing well means running many times— and how to bring that cost to zero: pointing the agent at a local Ollama model (llama3.2, mistral), which runs on your machine with no per-call charge because you provide the compute. Module 4's Self-Hosted AI Starter Kit already gives it to you ready. It's the cook's pattern: rehearse cheap with the local model (the mushroom), validate expensive once with the cloud's current model (the truffle) before promoting. You faced the two-versions problem honestly —the model is a sub-node, not a credential— and its two ways of handling it: Ollama as a temporary development action with final validation against the real model (the simple, recommended one), or a single node with a configurable base URL (check it on your version). You locked in the hard rule: validate against the current model, never a retired one, which would be a failure with a due date. And you saw how to tame the agent's unpredictability for deterministic tests: lowering the temperature and pinning its output when testing what comes after it.

With this you check off the checklist's fifth point: the AI agent gets tested at zero cost. It's the point that saves the most budget, because the LLM's cost grows with every run, and testing well means running many times; bringing it to zero is what lets you test without rationing.

Before moving on you should be able to: explain why a local model is free and a cloud one charges; describe the "rehearse cheap, validate expensive" pattern; say why you never validate against a retired model; and name the two levers that make agent tests deterministic (lowering the temperature and pinning its output).

You can already run all of order-triage —agent included— at zero cost and reproducibly. But so far "testing" has meant running and looking at the output. Lesson 7 takes the final leap: stop eyeballing it and automatically verify the output is correct. You're going to meet n8n's Evaluation node for writing assertions about the result —including the agent's output— building a set of cases with their expected result, and defining a passing threshold. It's the difference between "it ran" and "it got it right," which we set up in lesson 1 and that here finally becomes automatic.

Resources

• Self-hosted AI Starter Kit — n8n Docs — the kit that brings Ollama and a local model ready to go; this lesson's foundation, set up in Module 4.
• Ollama Chat Model — n8n Docs — the sub-node connecting the AI Agent to your local Ollama; its Model and Sampling Temperature fields.
• AI Agent — n8n Docs — order-triage's agent node and how a chat model sub-node gets connected to it.
• Ollama — official site — how to download models (ollama pull) and which local models are available (Llama 3.2, Mistral, and more).
• Ollama credentials — n8n Docs — how n8n points at your local Ollama server; check the address inside the Starter Kit there.