6. Concurrency and rate limits

Description

By the end of this lesson you'll be able to decide how many things to do at once when a workflow has to make many calls that can't be grouped, without the system on the other side blocking you. You'll know what a rate limit is and how to read the response with which an API warns you that you went over, and you'll understand the paradox that ruins a lot of people: raising concurrency can make the workflow slower, not faster, because the other side starts rejecting you and everything is retried. You'll finish with a method for finding the pace that runs fast without knocking over the erp.

This matters because concurrency is the optimization that most betrays whoever applies it without understanding it. It sounds irresistible: "if I make the calls one at a time it takes forever; if I make them all at once, they finish fast." Intuition says more parallel is faster, always. And up to a point it's true —but past that point, the curve inverts and gets worse—, and the point where it inverts isn't set by you, it's set by the system you call. Going past that point isn't "a bit less efficient": it's triggering a chain reaction of rejections and retries that can leave the workflow slower than if you'd gone one at a time, and along the way it can knock the service down for others.

Connection to the module: this lesson is the natural continuation of lesson 5. There you learned that the best play is to reduce calls —turn three hundred into three—. But lesson 5 ended with a pending case: when the API doesn't offer a bulk operation and you're left with N distinct calls you do have to make. This lesson is about that case: how to make those N calls at the right pace. The mental order is important: first reduce (lesson 5), then manage the pace (lesson 6). Concurrency is the lever for the calls you couldn't eliminate, not a substitute for eliminating them. And a connection with Module 2: when the other side rejects you, your retries (layer 1 of error handling) come into play, and here you'll see how those retries, poorly calibrated, can make the problem worse instead of fixing it.

The tollbooth with three lanes

Imagine a tollbooth that can serve three cars at once —it has three lanes with a toll collector—. Behind them come a hundred cars.

If you send the cars one at a time —one lane, the other two empty—, the line advances, but slowly: you waste two thirds of the tollbooth's capacity. You could be serving three at once and you're only serving one.

If you send the cars three at a time —you fill the three lanes—, the tollbooth works at full tilt and the line advances as fast as possible. That's the optimal point: as many cars in parallel as there are lanes.

Now, what happens if, believing "more is faster," you try to squeeze ten cars at once into a three-lane tollbooth? They don't fit. They pile up at the entrance, they get in each other's way, some have to back up and try again, a jam forms at the mouth of the tollbooth. And here's the counterintuitive part: the tollbooth now serves fewer cars per minute than when you sent it three orderly ones, because part of its time goes to managing the pileup instead of collecting tolls. Pushing beyond the capacity doesn't speed things up: it congests, and congestion is slower than orderly flow.

An external system —the erp, an API— is that tollbooth. It has a capacity: how many requests it can serve per unit of time. Concurrency is how many cars you send at once. Up to the tollbooth's capacity, more concurrency is faster. Past the capacity, more concurrency is slower, because the system starts rejecting requests —"back up and try again"— and those rejections, with their retries, congest everything. This lesson's skill is finding the number of lanes in the tollbooth you're calling, and not going over.

What a rate limit is and how the API tells you

An API's "number of lanes" has a name: rate limit. It's the rule that says how many requests the service will accept in a window of time. For example: "maximum 120 requests per minute." It's a defense of the service: without it, a single careless —or malicious— client could saturate it and leave everyone else without service. Terra Market's erp has one, as we said from Module 2: "REST API with a per-minute request limit."

The important thing is that the API warns you when you're getting close or you go over, in two ways you have to know how to read.

The headers that announce your budget. Many APIs include, in the response of every request, some headers that tell you how you're doing with your call budget. The most common names:

• X-RateLimit-Limit — your total cap in the window (e.g. 120).
• X-RateLimit-Remaining — how many calls you have left in this window (e.g. 43).
• X-RateLimit-Reset — when your budget resets (a time or a countdown of seconds).

Read them like the balance meter on a prepaid card: Limit is how much you bought, Remaining is how much you have left, Reset is when it recharges. If Remaining is dropping toward zero, you know you're about to hit the limit before hitting it. To see these headers in n8n, the HTTP Request node has an option —Include Response Headers and Status— that makes the response also bring the headers, not just the body. With that you can read your balance.

The response that says "you went over": the 429 code. When you cross the limit, the API doesn't serve your request; it responds with the status code HTTP 429 — Too Many Requests. It's the tollbooth telling you "back up, not now." And it almost always comes with a key header:

• Retry-After — how many seconds to wait before trying again (e.g. 30).

That Retry-After is a gift: the API is telling you exactly how long to wait so it serves you again. Ignoring it and retrying immediately is like pushing your car against the tollbooth's closed barrier: it's not going to open sooner from insisting, and you only congest things more.

Verify the names in your API. X-RateLimit-* and Retry-After are the most frequent names and many services use them, but they're not a universal law: some APIs use variants (RateLimit-Remaining without the X-, or proprietary names) or communicate the limit only in their documentation. The 429 code is standard and almost always means the same thing. Before building your rate-limit handling, read your API's documentation to know which headers it sends and what limit it has. The concept is universal; the exact names, not.

The paradox: more concurrency, slower

Here's the heart of the lesson, and it's what separates it from "raise the concurrency and done." Let's follow exactly what happens when you go past the tollbooth's lanes, step by step, because the chain reaction is what you have to understand.

Suppose the erp comfortably handles about 3 simultaneous requests (its tollbooth has 3 lanes). You have 300 calls to make. Let's compare two configurations.

Moderate concurrency (3 at a time): you send 3, they're served, you send 3 more, they're served. The tollbooth works at full tilt without congesting. The 300 calls flow at a good pace. Nobody rejects anything. You finish in a reasonable time.

Aggressive concurrency (30 at a time): you send 30 at once to a 3-lane tollbooth. This happens, in order:

1. The first 3 are served. The other 27 reach a system that can't serve them yet.
2. The erp, to protect itself, responds to many of those with 429 — Too Many Requests. Say 20 of the 30 are rejected.
3. Those 20 rejections are failures. And if the HTTP Request node has Retry On Fail (layer 1 from Module 2), each of those 20 is retried.
4. The 20 retries reach the erp again, which is still saturated, and are rejected again. More retries.
5. Meanwhile, the erp is spending its capacity on rejecting requests instead of serving them, so even the legitimate calls run slower.

The result is a storm: you make far more requests than the original 300 (the 300 + all the retries of the rejections), the erp spends its time saying "no" instead of working, and the workflow finishes later than with moderate concurrency —if it finishes, because it can exhaust the retries and fail—. You pushed harder and advanced less.

    Moderate concurrency (3):         Aggressive concurrency (30):

    ███ →  served                     ██████████... → 30 arrive
    ███ →  served                     ███ → served (3)
    ███ →  served                     ▓▓▓▓▓▓... → 429 (rejected)
    ...    flows                          │
    ✓ finishes at a good pace             └─► retries ─► more 429 ─► storm
                                          ✗ finishes slower (or fails)

The lesson of the paradox, so you don't forget it:

More concurrency is faster only up to the other side's capacity. Past that point, each extra unit of concurrency produces rejections, the rejections produce retries, and the retries produce more congestion. The curve doesn't keep going up: it turns over. The goal isn't "maximum concurrency," it's "the concurrency the other side serves without rejecting."

And notice the role of Retry On Fail: it's a great layer for transient failures (Module 2), but facing a 429 from excess concurrency it becomes gasoline for the fire. Retrying a saturation rejection immediately guarantees another rejection. That's why retries, when the cause is the rate limit, have to wait —ideally what Retry-After says— and not fire dry.

Finding the point without knocking over the ERP

Since the goal is "the concurrency the other side serves without rejecting," you need a method for finding it. It's similar to the one for finding the batch size from lesson 3: it's not calculated, it's searched for by measuring, carefully and with a margin.

Step 1 — Find out the declared limit, if it exists. First thing, free: read the erp documentation. If it says "120 requests per minute," you already have a reference ceiling without having made a single test. Many APIs publish it.

Step 2 — Start low, not high. Contrary to batch size, here the expensive mistake is starting high —you congest and provoke the storm—. Start with little concurrency (for example, 2 or 3 at a time) and observe. It's safer to go up from below than to come down from a storm.

Step 3 — Raise gradually and watch the rejections. Increase the concurrency gradually and, at each level, look at two things: does the total time improve? do 429 responses appear? As long as the time improves and there are no 429s, you can raise. The moment 429s start appearing, you went over: drop one step. The optimal point is just below where the rejections start.

Step 4 — Leave a margin and respect the reset. Just as with batches, don't stay right at the edge. If at 5 the 429s start and at 4 they don't, don't set 4: set 3. Production has bad days, and the erp "slow at peak hours" (its character, from Module 2) has less capacity right when you use it most. Leave air.

Step 5 — If there's a 429, honor the Retry-After. Even with a good margin, some 429 can sneak in during a spike. When it happens, your retry should wait what the API asks, not fire immediately. A retry that respects Retry-After recovers; one that insists dry feeds the storm.

The n8n tools for applying this, without leaving what you already know:

• The Batching of the HTTP Request node (which we saw at the end of lesson 5): Items per Batch controls how many requests it groups per round, and Batch Interval how long it waits between rounds. It's a direct way of saying "3 at a time, with a pause between groups."
• The Loop Over Items node with small batches, processing each round before the next, so as not to fire everything at once.
• The Wait node, to insert an explicit pause between calls or between rounds when you need to space them out more. Its After Time Interval mode (wait a certain amount of time) is the one that serves here; when a 429 arrives with Retry-After, you can wait exactly those seconds before retrying.

Waiting more and more: backoff

There's a detail about how to wait between retries that's worth knowing, because it's the difference between a retry that helps and one that makes things worse. It's called exponential backoff, and the idea is simple: if a retry fails, the next one waits more than the previous, and so on increasing.

The everyday image: you call someone and they don't answer. You don't redial every two seconds without stopping —that only saturates their phone and frustrates you—. You wait a bit, dial; if they don't answer, you wait a bit more, dial; and so on, giving more and more space. If the person was busy, the growing intervals give them real time to free up. Dialing in a burst guarantees you always find them busy.

With a rate limit it's the same. If the erp rejected you from saturation, retrying at growing intervals —1 second, then 2, then 4— gives it time to recover between one and the next, and keeps your retries from being part of the congestion that caused the rejection. Retrying dry, at a fixed and short interval, is dialing in a burst: you always arrive at the worst moment. That's why, when the cause of the failure is a 429, the growing wait beats the fixed wait, and whenever the API sends Retry-After, that value rules over any calculation of yours —it's telling you the exact moment it'll be ready—.

In n8n you can build this by combining the retry with waits: a Wait that grows between attempts, or logic that reads the Retry-After and waits exactly that. Verify in your version what the built-in retry node offers and complete it with Wait where you need fine control. The concept —wait more and more— is what I want you to take away; the exact implementation you adjust to your version's tools.

Worked example: querying 500 guides at the carrier without provoking the storm

Remember the case that was left pending in lesson 5: the carrier (andes-express) only offers GET /tracking/{guide_id}, one at a time, with no bulk operation. A workflow queries 500 guides per run. They can't be grouped; you have to make 500 calls. The question is how.

The naive attempt. Someone configures the workflow to fire the 500 with high concurrency —"so they finish fast"—. The carrier, which is also "the most unstable of the four" (Module 2), saturates, starts responding 429, the retries pile up, and the workflow finishes slower than ever —or fails leaving half the guides unqueried—. The storm.

The measured attempt. Instead of that:

1. Reduce first (lesson 5). Are there really 500? The "delivered" guides no longer change status. You filter and 90 "in transit" remain. You already dropped from 500 to 90 before touching concurrency. (This step isn't from this lesson, but it's the one that saves the most, and it goes first.)
2. Read the limit. The andes-express documentation says "60 requests per minute." That's your ceiling: 60/min ≈ 1 per second.
3. Configure to respect it. With Batching in the HTTP Request —or with Loop Over Items + Wait— you make the 90 calls at a pace of, say, 1 per second (with a margin under the 60/min limit). The 90 take around 90 seconds, smooth, without a single 429.
4. Prepare for the 429 just in case. You give the node Retry On Fail, but with a wait —and if a 429 arrives with Retry-After, a Wait that honors those seconds before retrying—.

What to expect. The measured attempt takes around a minute and a half and always finishes. The naive attempt, with luck, takes about the same on a good day, and on a bad day it enters the storm and takes much longer or fails. The difference isn't speed in the best case —they're similar—; it's stability: the measured attempt has no bad days, because it never pushes the carrier beyond its capacity. In production, "always finishes in 90 seconds" beats "sometimes 60, sometimes it fails." And notice the order: what saved the most wasn't the fine concurrency, it was reducing from 500 to 90 in step 1. Concurrency managed the 90 that remained well; the reduction eliminated the 410 that weren't needed.

A note on the concurrency of the n8n instance itself

Up to here we talked about concurrency outward: how many simultaneous calls you make to an API. There's another concurrency, a different one, worth naming so you don't confuse them: that of the n8n instance itself, that is, how many executions n8n runs at the same time.

n8n has an environment variable, N8N_CONCURRENCY_PRODUCTION_LIMIT, that limits how many production executions run at once. When the limit is reached, new executions aren't lost: they wait in a queue and start in order when capacity frees up (that is, exactly, lesson 2's "queue bucket"). Its default value and behavior depend on the edition and the mode (regular or queue), so verify the current number for your instance in the documentation instead of assuming it.

This is infrastructure, not workflow design. N8N_CONCURRENCY_PRODUCTION_LIMIT is a setting of the instance, on the server side, and its territory is the self-hosting and operations guide, not this one. I mention it for two reasons. One: so you don't confuse "how many calls I make at once to an API" (this lesson's thing, which you do control from the workflow) with "how many executions n8n runs at once" (infrastructure). Two: because that instance limit is one of the signs of lesson 7 —if your executions spend a lot of time in a queue waiting their turn, it's a sign that the instance, not the workflow, is the bottleneck—. Here, instead, we work on concurrency outward, which is a workflow-design decision and is resolved with Batching, Loop Over Items, and Wait.

Common mistakes

Raising the concurrency to the max "so it finishes fast" (conceptual). What happens: the maximum possible concurrency is configured with the logic of "more parallel, faster," and the external system saturates, responds 429, and the workflow finishes slower or fails. Why it happens: the intuition that more parallel is always faster is only true up to the other side's capacity, and that capacity is invisible until you cross it. How to spot it: if you see 429 responses in your executions, or if raising the concurrency worsened the time instead of improving it, you crossed the point. How to fix it: lower the concurrency until the 429s disappear and leave a margin. The goal isn't maximum concurrency, it's maximum concurrency that the other side serves without rejecting.

Retrying a 429 immediately (practical). What happens: the node has Retry On Fail without a wait, a 429 arrives, and the retry fires instantly against a service that's still saturated, provoking another 429, and another. Why it happens: Retry On Fail is the right solution for transient failures (Module 2), and it's easy to forget that a 429 isn't just any transient failure: it's "wait before coming back." How to spot it: if your retries facing 429 fail over and over, you're firing them too soon. How to fix it: make the retry wait —what Retry-After says if present, or a prudent time if not— before coming back. A Wait or a retry configuration with a growing wait. Retrying without waiting for a 429 is pushing the tollbooth's closed barrier.

Ignoring the headers the API gives you (practical). What happens: the API sends X-RateLimit-Remaining in each response —it tells you how much balance you have left— and the workflow doesn't read them, so it hits the limit blind when it could have seen it coming. Why it happens: by default the HTTP Request node returns only the body, not the headers, so you have to enable them on purpose and a lot of people don't know they're there. How to spot it: if you're handling an API with a rate limit and don't read its headers, you're flying without instruments. How to fix it: enable Include Response Headers and Status in the node to receive the headers, and use them to regulate the pace before hitting. It's free information the API gives you; not reading it is wasting it.

Tuning the concurrency before reducing the calls (conceptual). What happens: effort is dedicated to finding the magic concurrency number for 300 calls that, with lesson 5, would have been 3. Why it happens: concurrency is an entertaining problem to solve and the boring step of asking whether the calls were necessary gets skipped. How to spot it: if you're calibrating concurrency without having verified that the API doesn't offer batch and that you're not querying too much, you skipped the highest-impact step. How to fix it: first reduce (lesson 5); then calibrate the pace of the ones that remain. Three calls don't even need concurrency management; three hundred do, but better make them three.

Confusing concurrency outward with the instance's limit (conceptual). What happens: someone reads about N8N_CONCURRENCY_PRODUCTION_LIMIT and thinks that by adjusting it they'll control how many calls their workflow makes to the erp. They're different things: that variable limits how many executions n8n runs, not how many requests an execution makes to an API. Why it happens: both use the word "concurrency." How to spot it: ask yourself whether you want to control "how many things n8n does at once" (instance variable, infrastructure) or "how many simultaneous calls I make to an API" (workflow design, this lesson). How to fix it: for the pace of the calls to an API, use Batching, Loop Over Items, and Wait within the workflow. The instance variable is another conversation, from the self-hosting guide.

Exercises

Exercise 1 — Read the meter. A response from the erp brings these headers. Interpret each one and say what you'd do.

X-RateLimit-Limit: 120
X-RateLimit-Remaining: 8
X-RateLimit-Reset: 25

See solution

• X-RateLimit-Limit: 120 — your cap is 120 requests in the current window.
• X-RateLimit-Remaining: 8 — you have only 8 left before hitting the limit.
• X-RateLimit-Reset: 25 — your budget recharges in 25 seconds.

What I'd do: I'm at the edge. With only 8 calls of balance and more still to make, if I keep at the current pace I'll get a 429 shortly. The prudent thing is to slow down: space out the next calls, or pause until the Reset recharges the budget in 25 seconds. This is exactly the value of reading the headers: I see the crash coming with 8 to spare and can avoid it, instead of discovering it when I've already been rejected.

Why it works: Remaining is your flight instrument. Dropping toward zero is a yellow light; if you ignore it, the red one (the 429) comes on its own. Reading it turns the rate limit from a surprise into something you manage.

Exercise 2 — Explain the paradox. A colleague says: "I set the concurrency to 50 so the 400 calls to the erp would finish fast, but now it takes longer than when it was at 4 and I see a ton of 429 errors. I don't get it, wasn't more parallel supposed to be faster?". Explain to them what's happening, in terms of the tollbooth.

See solution

I'd tell them: the erp is a tollbooth with few lanes —say it serves about 4 requests at a time well—. When you send it 4, it fills its lanes and works at full tilt: it flows. When you send it 50, they don't fit: it serves 4 and tells the other 46 "I can't, back up" —that's the 429—. Those 46 rejections get retried, come back, get rejected again, and the erp spends its time saying "no" instead of serving. The tollbooth now spends its time managing the pileup, so it serves fewer requests per minute than when you sent it 4 orderly ones. That's why it took longer with 50 than with 4: you didn't speed it up, you congested it.

What to do: lower the concurrency to a number where 429s don't appear —probably near 4, with a margin— and, if some 429 sneaks in, have the retries wait before coming back. "More parallel is faster" is only true up to the tollbooth's capacity; past that, it inverts.

Why it works: the key is that the capacity is set by the erp, not by your configuration. You don't decide how many lanes the tollbooth has; you only decide whether you respect that number or ignore it. Ignoring it doesn't enlarge the tollbooth, it only creates the traffic jam.

Exercise 3 — Order the complete solution. A workflow has to update the status of 800 orders in the erp, one by one (the erp doesn't offer a bulk update for this operation), and today it does it with high concurrency and fails often with 429. Write the complete plan, in order of priority, combining what you know from lessons 5 and 6.

See solution

The plan, in order:

1. Reduce the N (lesson 5). Do all 800 really have to be updated, or only the ones that changed since the last run? If only 120 changed, I update 120. The call I don't make can't saturate anything. This step, although it's from the previous lesson, goes first because it's the highest-impact one.

2. Verify there really is no bulk operation (lesson 5). I confirm in the documentation that this specific endpoint doesn't accept several orders. Sometimes you assume there's no batch and there is one for this operation.

3. Read the rate limit (lesson 6). I find out the erp's cap (by documentation or by the X-RateLimit-* headers) to know at what pace I can go.

4. Lower the concurrency and space it out (lesson 6). I configure Batching or Loop Over Items + Wait to make the remaining calls at a pace below the limit, starting low and raising only while 429s don't appear, with a margin.

5. Prepare for the 429 (lesson 6). Retry On Fail with a wait, and if Retry-After arrives, honor it before retrying.

Result: from 800 rushed calls that fail, to ~120 orderly calls that always finish.

Why it works: the order goes from highest to lowest impact and from eliminating to managing. First you remove the calls that are superfluous (steps 1-2), then you make the ones that remain well (steps 3-5). Starting with step 4 —tuning the concurrency of the 800— would be optimizing the pace of hundreds of calls you didn't even have to make.

Summary and next step

In this lesson you saw concurrency with the image of the tollbooth: as many cars in parallel as the tollbooth has lanes make it flow at full tilt, but one car too many congests it and makes it slower. You learned what a rate limit is —how many requests an API accepts per window of time— and how the API communicates it to you: the X-RateLimit-* headers that show you your balance, and the 429 code with its Retry-After that tells you how long to wait when you went over (verify the exact names in your API; the 429 is standard). You understood the paradox that fools so many people: past the other side's capacity, more concurrency produces rejections, the rejections produce retries, and everything becomes slower —and why a Retry On Fail without a wait pours gasoline on the fire facing a 429—. And you kept the method for finding the point: read the limit, start low, raise gradually watching the 429s, leave a margin, and honor the Retry-After, with the tools you already know —Batching, Loop Over Items, Wait—. We closed by distinguishing this outward concurrency from the instance's own (N8N_CONCURRENCY_PRODUCTION_LIMIT), which is infrastructure and anticipates the next lesson's boundary.

Before moving on you should be able to: explain the concurrency paradox with the tollbooth; name the rate-limit headers and the 429 code with its Retry-After; and say why reducing calls (lesson 5) goes before managing the pace (lesson 6).

With this lesson you close the workflow's optimization techniques. You measured (L2), batched (L3), took care of memory (L4), reduced calls (L5), and found the pace (L6). The honest moment arrives: what if you did all this and the workflow —or the set of your workflows— still can't keep up? Lesson 7 is the module's boundary. It teaches you to recognize, with evidence, the point at which the bottleneck is no longer in the workflow but in the instance —when executions wait their turn in a queue, when several simultaneous workflows saturate the machine—, what to measure to prove it before spending on infrastructure, and when to cross over to the self-hosting guide to set up queue mode. It's the only moment in the module when the right answer can, truly, be "yes, now it's time for more machine."

Resources

• HTTP Request node — n8n Docs — its Batching options (Items per Batch, Batch Interval) for spacing out requests, and Include Response Headers and Status for reading the rate-limit headers.
• Wait node — n8n Docs — the node for inserting pauses between calls or retries; its After Time Interval mode is the one that serves for respecting a Retry-After.
• Control concurrency — n8n Docs — the instance's N8N_CONCURRENCY_PRODUCTION_LIMIT variable (infrastructure, self-hosting); verify its current default value and behavior here.
• HTTP status 429 Too Many Requests — MDN — the reference for the standard code with which APIs announce that you crossed their rate limit, and for the Retry-After header.