Module 2: Idempotency: Making Repeats Not Duplicate

2. What idempotency is, with analogies

Next →

Description

By the end of this lesson you'll be able to define idempotency without hesitating, recognize at a glance whether an operation is idempotent or not, and explain why that distinction is what decides which steps of your workflow are dangerous to repeat. You're going to have two analogies you won't forget —an elevator's floor button and a light switch— and you're going to understand why half the HTTP methods you already use every day were giving you idempotency for free without you knowing it, while the other half is exactly where the duplicate is born.

This matters because idempotency isn't a technique you apply at the end, like a coat of paint. It's a property an operation either has or doesn't have, and knowing how to read it changes how you look at a workflow. When you can walk through order-triage's nodes and say "this one is idempotent by nature, this one isn't and needs protecting, this one doesn't matter either way," you already have half the job done. The rest of the module is the tools for protecting the ones that aren't; but first you need to know which ones they are.

Connection to the module: lesson 1 gave you the one-sentence definition and the map. This one calmly takes it apart and trains your eye to classify operations. It's the foundation for everything that follows: lesson 3 chooses the key that's going to make an effect idempotent, but for that you first need to know what an effect is and why it isn't one already. Lessons 4 and 5 are the two ways of turning a non-idempotent effect into an idempotent one —the upsert and the header—; neither makes sense if you can't first tell the two categories apart. Hold on to this lesson's central heuristic, "setting a value" versus "modifying what was there," because you're going to use it in the remaining six lessons.

The definition, and why the word sounds strange

Let's start with the basics, including the word, which is scarier than it needs to be.

Idempotency comes from Latin: idem ("the same") and potentia (here, in the sense of "raised to a power"). The original idea is mathematical: an operation is idempotent if applying it several times gives the same result as applying it once. The classic example is multiplying by 1. Take a number, multiply it by 1, and you get the same number. Multiply it by 1 again: still the same. You can multiply by 1 a thousand times and nothing ever changes after the first —and really, not even after the first—. Compare it with multiplying by 2: every application doubles, so two applications don't give the same as one. Multiplying by 1 is idempotent; multiplying by 2 isn't.

We're not going to do math. We'll stick with the definition translated into the world of systems, which is the one that governs this module:

An operation is idempotent if running it many times leaves the system in the same state as running it once.

Notice the word state, because it's the key. Idempotency doesn't talk about whether the operation "does something" every time —in fact it can do work every time—; it talks about what the world is left as. If, after one execution, the world ended up a certain way, and after five executions, the world ended up exactly that same way, the operation is idempotent. It doesn't matter that the five executions burned electricity, CPU time, or five taps of your finger. What matters is the final state.

The light switch and the elevator button

Let's use two objects you have at home, because together they capture almost everything you need to know.

The light switch is the perfect example of idempotency. I don't mean the up/down toggle, but one with two separate buttons: one that says "on" and one that says "off." Press "off." The light turns off. Press "off" again. The light... stays off. And again. And ten more times. The system's state —"the light is off"— is identical after one press or ten. The "off" button sets the state to a value: off. Setting a state to an absolute value is the purest form of idempotency, because the value doesn't depend on what was there before. "Leave the light off" gives the same result whether it was on, off, or you already turned it off three times.

The floor button in an elevator is the same idea, and it's the analogy we're going to repeat throughout the module because it describes exactly what's going to happen to the second arrival of Cumbre's webhook. You get into the elevator, you want floor 5, you press "5." It lights up. Impatient, you press it four more times. Do you go to floor 25? Do five elevators get called? No: the button registers "destination = floor 5" and the following presses change nothing, because the destination is already 5 and setting it to 5 again leaves it at 5. The button sets a state —the destination— to an absolute value. It's idempotent.

Now the counterexample, so the distinction sticks. The "add to cart" button on a poorly programmed store. You click it and it adds one unit. You click it again and it adds another. Five clicks, five units. This button doesn't set the state to a value; it modifies it relative to what was there —"take what's there and add one"—. And there's the root of all non-idempotency: when an operation depends on what was there before to decide the result, repeating it accumulates.

Hold on to this heuristic, because it's the most useful thing in the lesson and you're going to apply it dozens of times:

Setting a state to an absolute value ("leave it at 5," "leave it off," "let order ORD-2041 exist with this data") is idempotent. Modifying the state relative to what was there ("add one," "add a row," "create a new charge") isn't.

The word that gives away a non-idempotent operation is almost always a creation or increment verb: add, create, insert, sum, increment, send (another). The word that gives away an idempotent one is an assignment verb: set to, fix, establish, leave at, make it.

Worked example: classify order-triage's nodes

Let's walk through Cumbre's workflow node by node and classify each operation. This is the mental exercise I want you to run automatically whenever you look at any workflow.

Remember the flow:

Webhook  ──►  AI Agent  ──►  HTTP Request (POST /charges + email + write to CRM)

The Webhook. Its operation is "receive an order and make it available to the workflow." Does it set a state or does it modify it by accumulating? Receiving a piece of data doesn't change anything permanent in the outside world: it's pure data coming in. Receiving the same order twice doesn't leave two harmful traces by itself. Idempotent, or rather, harmless. No need to protect it.

The AI Agent. Its operation is "read the order and produce a classification": priority, channel, urgency. This is a reasoned read —it consumes the order and returns a decision, but changes nothing outside—. Classifying ORD-2041 twice produces two decisions, and there's an honest nuance here that lesson 7 is going to develop: a language model can give slightly different classifications across two runs, because it isn't perfectly deterministic. But even if they differ, the classification itself doesn't charge or send emails; it's a computation. As long as the agent only classifies and doesn't act, repeating it wastes compute but leaves no permanent harm. Essentially idempotent, with the non-determinism caveat we'll cover in due time.

The HTTP Request. Here's the problem, and it's actually three effects hidden inside one node:

  • Creates a charge (POST /charges). Every run creates a new charge, with a new identifier. It's "add to cart": it modifies the state by adding. Not idempotent.
  • Sends a confirmation email. Every run sends a new email. Adding one more email. Not idempotent.
  • Writes the order record to the CRM. If it does it with an "insert a new row," every run adds a row. Not idempotent as it stands —and lesson 4 is going to fix it by turning it into an upsert—.

What to expect from this analysis. After classifying, your map of order-triage looks like this: two green nodes (webhook, agent) you can repeat with no fear, and one red node (the HTTP Request) with three effects that need protecting. That map is the entire module's work plan. You're not going to touch the green ones. You're going to wrap the red ones with a key (lesson 3), an upsert for the CRM (lesson 4), and a header for the charge (lesson 5).

Notice what you just did: you didn't memorize a rule, you applied the heuristic. "Does this operation set a value or add on top of what was there?" Creating a charge adds. Receiving a webhook changes nothing. Writing "let order ORD-2041 exist with this data" would set a value —and that's why the upsert, which does exactly that, is going to save us—.

HTTP methods: idempotency you were already using without knowing it

If you're coming from the ecosystem's APIs guide, you've already called APIs with different HTTP "verbs" —GET, POST, PUT, DELETE—. It turns out those verbs come with an idempotency promise written into the HTTP standard itself, and understanding it gives you precise vocabulary for the rest of the module.

The HTTP standard classifies methods along two properties worth not confusing: safe (don't change anything on the server) and idempotent (repeating them leaves the same state). Every safe method is idempotent, but not the other way around.

MethodSafe?Idempotent?What it means in practice
GETYesYesOnly reads. Ask for it a thousand times: nothing changes. Module 1's pure read.
HEADYesYesLike GET but only brings headers. Just as harmless.
PUTNoYes"Put this resource in this exact state." Sets a value.
DELETENoYes"Let this resource stop existing." Sets a value (the value of not-existing).
POSTNoNo"Create something new." Every call creates another thing. This is where the duplicate is born.
PATCHNoGenerally not"Partially modify." Depends on how it's written; it often adds.

Let's stop at the rows that matter.

GET is the pure read. It's the green node par excellence. Querying an order's status, reading a row from the CRM, asking for the product list: repeating a GET has no consequences, and that's why module 1 called it a read and declared it safe to repeat. Almost everything an AI Agent does to "inform itself" before deciding is GETs.

PUT is idempotent even though it changes things. This surprises a lot of people. PUT does modify the server —it isn't safe— but it is idempotent. The reason is exactly the switch heuristic: PUT means "put resource /orders/ORD-2041 in this complete state." It's setting an absolute value. Do it once or ten times: the resource ends up in that state, always the same one. That's why, when you can choose between expressing an effect as a POST (create) or as a PUT (put into a state), the PUT gives you idempotency for free. That idea is the seed of lesson 4's upsert.

DELETE is idempotent, and here's a subtlety that trips people up. Deleting order ORD-2041 once leaves it deleted. Deleting it again... leaves it deleted just the same. The state —"the order no longer exists"— is identical both times. That's why DELETE is idempotent. Now, careful: the response can change. The first time the server might respond 200 OK ("I deleted it"), and the second 404 Not Found ("that order doesn't exist"). Someone looks at that and says "the responses are different, so it isn't idempotent!" Wrong. Idempotency is measured by the system's state, not by the response code. The state stayed the same both times; the second response being a 404 is just the server telling you "it was already done." This distinction —state vs. response— is worth its weight in gold, and it comes back in lesson 5.

POST isn't idempotent, and that's why it's always the suspect. POST means "create a new resource." By definition, every call produces a new one. Cumbre's POST /charges creates a new charge every time; that's where the second charge comes from. When you see a POST producing an effect, sound the alarm: it's suspect number one for duplicating. Lesson 5 is dedicated precisely to taming a dangerous POST with an idempotency header.

The conclusion you walk away with: when you design an effect, ask yourself whether you can express it as a PUT (setting a state) instead of a POST (creating). Often you can, and that single decision gives you idempotency for free. When you can't —when the API only offers POST— you need this module's tools. And a warning about PATCH, which shows up in the table as "generally not": its idempotency depends on how it's written. A PATCH that says "set status to shipped" sets a value and is idempotent; one that says "add 100 to the balance" accumulates and isn't. Don't trust the PATCH verb by itself; look at what the operation does underneath, with the same set-vs-accumulate heuristic.

Worked example: state vs. response, with a real DELETE

The distinction between state and response sounds like philosophy until you see it in concrete requests. Let's make it concrete. Imagine Cumbre has an internal API for its orders and you want to cancel ORD-2041 by deleting it. You're going to make the same request twice, as if a retry fired it double.

First request:

DELETE /orders/ORD-2041

What to expect in the response:

HTTP/1.1 200 OK
{ "deleted": "ORD-2041" }

The server deleted the order. At this moment, the server's state is: ORD-2041 no longer exists.

Second request (identical, fired by the retry):

DELETE /orders/ORD-2041

What to expect in the response:

HTTP/1.1 404 Not Found
{ "error": "order ORD-2041 does not exist" }

This is where a lot of people get it wrong. The response changed: 200 the first time, 404 the second. The temptation is to conclude "it's not idempotent, the responses are different!"

But look at the server's state after each request:

ResponseServer state afterward
1st request200 OKORD-2041 doesn't exist
2nd request404 Not FoundORD-2041 doesn't exist

The state is identical both times: the order doesn't exist. That's what defines idempotency, and that's why DELETE is idempotent. The difference in the response is just the server informing you which of the two requests did the actual work: the first did the deletion, the second found it was already done. Neither left the order in a state different from the one you were aiming for.

Hold on to this image, because in lesson 5 you're going to see the same phenomenon with a payment gateway: the first call with an Idempotency-Key creates the charge and responds "created"; the second, with the same key, responds "already had it" and returns you the same charge without creating another. Responses that feel different, state that's the same. Learn to look at the state.

Operations that are already idempotent by nature (and you don't have to do anything)

Part of maturity with this topic is knowing when not to do anything. Not every effect needs protection; some are born idempotent, and wrapping them in idempotency machinery is wasted work and more fragile code.

These operations are already idempotent as they stand:

Setting a field to a fixed value. "Mark order ORD-2041 as status: shipped." Run it once or ten times: the state ends up shipped. No need to protect it. It's a PUT in disguise.

Deleting by identifier. "Remove the temp file tmp-2041." If it no longer exists, deleting it again does no harm. Idempotent.

Writing a whole file with the same content. "Save this report to report-2041.pdf." Overwriting with the same content leaves the same file. Idempotent (unlike appending a line to a file, which does accumulate).

Assigning someone to a role they already have. "Make CUST-118 a wholesale customer." If they already are, assigning it again doesn't create a second wholesale customer. Idempotent.

And these aren't, no matter how harmless they look:

Incrementing a counter. "Add one to this order's views." Classic non-idempotent case. Every run adds.

Adding to a list. "Add CUST-118 to the notified list." If the list allows duplicates, you add it twice. (If the list were a set that ignores duplicates, it would be idempotent —and that's exactly the upsert trick—).

Sending a message. Email, WhatsApp, notification: every send is one more message in someone's inbox. Not idempotent, and particularly annoying because the harm is seen by a human.

Creating a resource with no identifier of its own. "Create a charge." Without a key that says "this charge and the previous one are the same," every creation is new.

The pattern, again, is the same: look at whether the operation sets something or accumulates it. If it sets, relax: it's already idempotent. If it accumulates, it's a customer for the rest of this module.

There's a case worth highlighting because it's the exact bridge to lesson 4: "adding to a set." A set, in the mathematical sense, is a collection that doesn't allow duplicates. If you have a set of notified customers and you "add" CUST-118 when it was already there, the set doesn't change —it still has a single CUST-118—. Adding to a set is idempotent, precisely because the set ignores the second attempt. Compare it with adding to a list, which does allow duplicates and therefore accumulates. This difference —set vs. list— is the essence of the upsert: an upsert turns your database table into "a set by key," where trying to insert the same order_id twice doesn't create a second row. Hold on to this image; in two lessons you're going to build it.

And a note of practical humility: n8n ships with some deduplication helpers out of the box —a node for removing duplicates, options in certain triggers to ignore items already seen—. They're useful and you're going to meet them, but they belong to module 4, which studies where the state that remembers "I already saw this" lives. In this module we build idempotency with the most fundamental tools —the key, the upsert, the header— so you understand the mechanism before using the shortcut. A shortcut you don't understand is a shortcut you can't debug when it fails.

Common mistakes

Believing "idempotent" means "does nothing the second time" (conceptual). What happens: someone understands that an idempotent operation "skips" repeated executions, and expects the node to not run, or to appear grayed out, the second time. Why it happens: the intuition of "don't duplicate" translates to "don't execute," but they're not the same. An idempotent operation can absolutely run in full every time —the elevator button registers your press all five times—; what it guarantees is that the result doesn't change. How to spot it: if you expect to see the second HTTP Request "not fire" and you get alarmed when it does fire, you're confusing idempotency with non-execution. How to fix it: separate the two ideas. The operation runs; the effect doesn't accumulate. The PUT to status: shipped executes all ten times and does its job all ten times; it's just that the final state is the same. It's more robust this way, because you don't depend on a mechanism that "skips" executions.

Confusing the response with the state (conceptual). What happens: a repeated DELETE gets tested, the second time returns 404, and it's concluded "it's not idempotent because the response changed." Or the other way around: an API returns 200 OK both times and it's concluded "all good, it's idempotent," when it actually created two resources. Why it happens: it's natural to judge by what the API returns to you, which is what you see. But idempotency is measured by the server's state afterward, not by the response code. How to spot it: always ask yourself "how many resources exist now?", not "what did it respond?" A 200 doesn't prove you didn't duplicate, and a 404 doesn't prove you did. How to fix it: to verify idempotency, look at the state —count the rows, count the charges—, not the response. The DELETE that responds 404 the second time is perfectly idempotent because the order still doesn't exist, which is the only thing that matters.

Treating every effect as dangerous and over-shielding (conceptual). What happens: after learning the topic, someone wraps every node in idempotency keys, upserts, and checks, even the ones that were already idempotent by nature. The workflow fills up with unnecessary machinery, harder to read and maintain. Why it happens: the freshly discovered fear of duplicates turns into excessive caution. How to spot it: if you have an upsert protecting a node that only does PUT status: shipped, or an idempotency key wrapping a GET, you're shielding something that was already shielded out of the box. How to fix it: classify first (this lesson's heuristic), protect only the ones that accumulate. A PUT to a fixed value, a DELETE by id, a full-file write: leave them alone. The elegance of a reliable system isn't protecting everything, it's protecting exactly what needs it.

Exercises

Exercise 1 — Classify six operations. For each one, say whether it's idempotent or not, and in one sentence why (use the "sets a value" vs. "accumulates on what was there" heuristic):

(a) UPDATE orders SET status = 'shipped' WHERE order_id = 'ORD-2041' (b) INSERT INTO order_log (order_id, event) VALUES ('ORD-2041', 'received') (c) UPDATE inventory SET stock = stock - 1 WHERE sku = 'CF-ARA-500' (d) A GET to /orders/ORD-2041 to read its current status. (e) Sending ORD-2041's confirmation email to the customer. (f) DELETE FROM cart_items WHERE cart_id = 'CART-9'

See solution

(a) Idempotent. It sets status to the value 'shipped'. Run it ten times: the state ends up shipped. It doesn't depend on what was there before. It's a PUT in SQL clothing.

(b) Not idempotent. INSERT creates a new row every time. Ten executions, ten rows in order_log. It accumulates. (This is the exact pattern lesson 4 fixes with an upsert.)

(c) Not idempotent. stock = stock - 1 modifies the state relative to what was there —it subtracts one from whatever's there—. Ten executions subtract ten. It's "add to cart" in reverse: decrementing.

(d) Idempotent (and also safe). A GET only reads. Repeating it changes nothing. It's the pure read.

(e) Not idempotent. Every send is one more email in the customer's inbox. It accumulates, and on top of that, the harm is seen by a human. Needs protection.

(f) Idempotent. It sets the state to "there are no items in cart CART-9." If it's already empty, deleting again leaves it empty. The final state is the same. (The response might say "deleted 0 rows" the second time, but the state didn't change: remember state vs. response.)

Why this works: all six are solved with the same question, not with memory. SET status = 'value' sets; INSERT and stock - 1 accumulate; GET only reads; sending an email accumulates; DELETE sets (to empty). That question —does it set or accumulate?— is the one I want you to ask automatically.

Exercise 2 — The POST-to-PUT trick. Cumbre needs to mark in its CRM that order ORD-2041 has already been processed. A developer proposes two designs. Say which one is idempotent and why, and what problem the other one would have if the webhook arrives twice:

  • Design A: POST /processed_orders with body { "order_id": "ORD-2041", "processed_at": "..." } — creates a "processed order" record.
  • Design B: PUT /orders/ORD-2041 with body { "processed": true } — sets the order to "processed" state.
See solution

Design B is idempotent; A isn't.

Design B uses PUT on an identified resource (/orders/ORD-2041) to set it to a state: processed: true. If the webhook arrives twice, the second execution sets processed: true again on an order that's already processed: true. The state doesn't change. A single order, marked as processed. Perfect.

Design A uses POST to create a new record every time. If the webhook arrives twice, two records get created in /processed_orders, both with order_id: ORD-2041. Now the "processed orders" table has the same order twice, and any report that counts "how many orders did we process" gives an inflated number. On top of that, processed_at would be different in each one, sowing confusion about when it "really" got processed.

The general trick: when you can express an effect as "put this resource in this state" (a PUT on a known id) instead of "create a record of the fact" (a POST), choose the former. Setting a value on a stable identifier is idempotency for free, and the order_id that already comes in the order is exactly that stable identifier —which leads us straight into lesson 3—.

Why this works: it's the set-vs-accumulate heuristic applied to a real design decision. Design A "accumulates records of the fact"; B "sets the resource's state." Same business goal, opposite idempotency.

Exercise 3 — Audit the full order-triage. Without looking back at the worked example, write for each node of Webhook → AI Agent → HTTP Request whether it's idempotent or not, and for the one that isn't, list the concrete effects that would need protecting. Then compare with the worked example.

See solution

Webhook: idempotent (harmless). Receiving the same order twice is just data coming in; it leaves no harmful trace by itself. Nothing to protect.

AI Agent: essentially idempotent. Classifying is a reasoned read; it produces a decision but changes nothing outside. The caveat —that a model can classify slightly differently across two runs because it isn't deterministic— is real but isn't a duplicated effect; it's lesson 7's topic. Nothing to charge or send.

HTTP Request: not idempotent. It hides three effects that accumulate:

  1. Creating the charge (POST /charges) — every run, one more charge. → protected by lesson 5 (idempotency header).
  2. Sending the confirmation email — every run, one more email. → same mechanism or a guard before sending.
  3. Writing the record to the CRM with an INSERT — every run, one more row. → protected by lesson 4 (upsert by order_id).

Why this works: you rebuilt the entire module's work plan. Two nodes left untouched, one node with three effects and a tool assigned to each. If you managed to name the HTTP Request's three effects, you already know clearly what needs fixing; the rest of the module is how.

Summary and next step

In this lesson you took apart the definition: an operation is idempotent if running it many times leaves the system in the same state as running it once —and the key word is state, not response or amount of work—. You grounded it with the light switch and the elevator's floor button, which set a value and are therefore idempotent, against the "add to cart" button, which accumulates and therefore isn't. You walked away with the heuristic that governs the whole module: setting a state to an absolute value is idempotent; modifying it relative to what was there isn't. You walked through order-triage classifying each node —webhook and agent, green; HTTP Request, red with three effects—. And you gave it precise vocabulary with HTTP methods: pure GET reads, PUT and DELETE set a value and are idempotent even though they change things, POST creates and is always the suspect; with the subtlety that idempotency is measured by the server's state, not by the response code it returns.

Before moving on to lesson 3 you should be able to: classify any operation as idempotent or not with the set-vs-accumulate heuristic; explain why PUT is idempotent and POST isn't; and say why a DELETE that responds 404 the second time is still idempotent.

You now know how to recognize an effect that isn't idempotent. Lesson 3 starts repairing it, and the first thing any repair needs is a name: an idempotency key that says "this event and that one are the same." You're going to see this module's most important and most treacherous decision —using a key that already comes in the order (natural) or manufacturing one yourself (synthetic)— and the mistake that ruins the whole mechanism: choosing a key that changes on every execution, like the arrival time, and thereby making every retry look "new."

Resources

← Previous1. Introduction: repeating without causing harmNext →3. Idempotency keys: natural vs. synthetic