6. constant-arrival-rate: load by arrival rate

Overview

The two executors from lesson 5 have something in common: they fix how many users there are (the VUs). This executor changes the question at its root: instead of fixing the users, it fixes how many requests per second arrive —the arrival rate, the RPS— whether the server finishes them or not. That difference, which sounds technical, is actually profound, because they're two different models of the world. The by-VU model is closed: each virtual user waits for its response before sending the next, so if the server gets slow, the VUs slow down on their own —the load self-limits—. The arrival-rate model is open: the requests come in at the rate you set, without waiting for the previous ones to leave, so if the server gets slow, the requests pile up —the load doesn't self-limit—. In this lesson you understand that difference, meet the constant-arrival-rate executor (content), and —most revealing— run it against Reservo at two rates: one below its capacity and one above, to see with numbers how the open model uncovers an overload the by-VU model would have hidden.

Connection to the module: lesson 5 covered the by-VU executors (the closed model); this one covers the by-arrival-rate executor (the open model). Together they complete the executors topic. This is also the module's conceptually densest lesson, because the leap from "fixing users" to "fixing demand" reorders how you think about a load test. We reuse the notion of capacity/throughput from module 3 (the RPS the server can sustain) and that of the p95. We don't get into thresholds (module 5) or the quote→book flow (module 6): here the focus is a single executor and the model it represents.

The analogy: the bank line vs the conveyor belt

Imagine two ways to give work to a bank teller.

Closed model (by VUs): there are 20 customers and a single line. When a customer is served, they leave and then the next in line moves up. If the teller gets slow, the line simply moves slower: there are never more than 20 people in the bank, because each one waits their turn before new work comes in. The pace of work self-regulates with the teller's speed. That's the VUs: each one waits for its response before asking again.

Open model (by arrival rate): there's a conveyor belt that drops a new form on the counter every 3 seconds, no matter what, regardless of whether the teller finished the previous one. If the teller keeps the pace, perfect: they process one, another arrives, the counter is clean. But if the teller gets slow —takes 5 seconds per form when one arrives every 3—, the forms stack up: the pile grows and grows, and each new form waits behind an ever-higher mountain. The belt doesn't notice the teller is slow; it keeps dropping forms at the same rate. That's the arrival rate: the requests come in at a fixed rate, whether the server processes them or not.

Which one models real traffic better? The belt. Your API's users don't coordinate in a polite line waiting for the previous one to finish; they arrive when they arrive, at a rate that depends on how many people there are, not on how fast your server responds. If your API gets slow on a Friday night, the users don't stop arriving out of courtesy —they keep coming in at the same rate, and they pile up—. That's why the open model (arrival rate) captures a danger the closed one (VUs) hides: the possibility that demand exceeds capacity and the queue grows without a brake.

The danger the by-VU model hides

Here's the practical consequence, and it's important. With a constant-vus of 20, you can never have more than 20 requests in flight at once, because each VU waits for its response before sending another. If the server saturates, the 20 VUs simply wait longer, and your test measures a load that self-limited to 20. That is: the by-VU model can't overload the server beyond N. If your server would collapse with a demand of 500 RPS but you test it with 20 VUs that self-brake, you'll never see the collapse —your test was kind to the server without meaning to—.

With constant-arrival-rate you fix the demand (say 500 RPS) and k6 takes care of sustaining it by launching as many iterations as needed, even if the server can't keep up. If the server can only handle 337 RPS and you demand 500, the difference (163 requests per second) piles up: the queue grows, the latency spikes, and eventually errors appear. That's exactly what happens in production when traffic exceeds capacity, and only the open model reproduces it in a test.

The constant-arrival-rate executor (content)

This is how it's configured, as labeled content (k6 isn't installed):

// CONTENT (not run here). Ref: grafana.com/docs/k6 (constant-arrival-rate).
export const options = {
  scenarios: {
    fixed_demand: {
      executor: 'constant-arrival-rate',
      rate: 200,             // 200 iterations...
      timeUnit: '1s',        // ...per second  => 200 RPS target
      duration: '1m',        // hold that rate for 1 minute
      preAllocatedVUs: 50,   // VUs reserved up front to serve the rate
      maxVUs: 300,           // VU ceiling if more are needed to sustain the rate
    },
  },
};

Notice how the load is defined differently than in the by-VU executors. You don't say "how many users," you say "how many iterations per unit of time": rate: 200 + timeUnit: '1s' = 200 RPS. The VUs become a resource k6 uses to sustain that rate, not the control variable: preAllocatedVUs is how many it reserves up front (creating them costs, so they're reserved before starting) and maxVUs is the ceiling it can grow to if the server gets slow and needs more VUs to keep the 200 RPS. That inversion —the VUs in service of the rate, not the other way around— is the essence of the open model.

A detail that can slip past you: if the server saturates and k6 reaches maxVUs without being able to sustain the rate, k6 reports it as iterations that fell short (dropped iterations). That number —how many requests couldn't even be launched— is itself a signal that you exceeded capacity, something the by-VU model would never tell you.

The executed experiment: below and above capacity

Let's go to the numbers, which is where this becomes sharp. First we need to know Reservo's capacity: how many RPS it can sustain. Measuring it (24 workers full throttle against /quote):

What to expect — the maximum sustainable throughput is the capacity; above it, the demand piles up. Real output:

capacity (24 VUs): 1369 requests in 4.1s = 337 req/s

This lab Reservo's capacity is ~337 req/s. Now we apply load by arrival rate at two levels: 120 RPS (comfortable, below 337) and 450 RPS (above 337). The generator measures the latency experienced by each request: from the instant it should have gone out until it finished, including the queue wait if the server can't keep up —exactly what a real user would feel—.

What to expect — at 120 RPS the server keeps the pace: low, stable latency, everything delivered. At 450 RPS the demand exceeds capacity: the queue grows, the latency spikes to seconds, and some requests don't even complete. Real output:

$ python3.14 arrival_rate.py http://127.0.0.1:PORT
ARRIVAL RATE load against http://127.0.0.1:PORT/quote
   target RPS  delivered    p50 (ms)   p95 (ms)   max (ms)
----------------------------------------------------------
          120  480/480           6.0        7.2        7.9
          450 1470/1800        100.9     1915.2     6719.5
----------------------------------------------------------

Read the two rows, because they tell opposite stories:

• 120 RPS (below capacity): 480/480 requests delivered —all of them—, with a p50 of 6.0 ms, a p95 of 7.2 ms, and a maximum of 7.9 ms. Notice how flat those numbers are: p50, p95, and maximum almost coincide. That's a system that keeps the pace: each request arrives at a clean counter, there's no queue, the latency is the pure service one. The belt drops 120 forms per second and the teller processes 120; the counter never piles up.

• 450 RPS (above capacity): the disaster. Only 1470/1800 delivered —330 requests didn't even complete—, and of those that did, the p50 is already 100.9 ms (17 times worse than at 120 RPS), the p95 explodes to 1915.2 ms (almost two seconds!), and the maximum reaches 6719.5 ms (almost seven seconds). Here the belt drops 450 forms per second but the teller only processes ~337: the ~113 surplus per second pile up, the pile grows throughout the whole test, and the last requests wait behind a mountain that had been forming for seconds. That's why the maximum is seconds: it's not a slow request, it's a request that waited its turn behind thousands.

With load by arrival rate, crossing capacity doesn't degrade a little: it breaks. Below (120 < 337 RPS) everything is flat and perfect; above (450 > 337 RPS) the queue grows without a brake, the p95 jumps from 7 ms to ~1900 ms, and entire requests are lost. The by-VU model would never have shown this, because it would have self-limited before overloading.

Why this matters: demand doesn't wait

The difference between the two rows is the whole lesson of the open model. At 120 RPS the system is healthy; at 450 RPS it's broken —and the point where it changes is the capacity, ~337 RPS—. A constant-vus would never have let you see that wall, because the VUs slow down on their own: with 24 VUs, even if the server gets slow, you never demand more than it can process sustainably (the VUs wait). The open model, by contrast, fixes the demand and lets the server suffer the consequences —exactly like real traffic, which doesn't reduce its pace because your server is slow—.

That's why constant-arrival-rate is the right executor when your question is about capacity in terms of rate: "does my API handle 500 orders per second?", "beyond how many RPS does it fall over?". Those questions are in RPS, not users, and RPS is what this executor controls. When you think about a peak's traffic —"we expect 500 requests per second on campaign day"—, you're thinking in arrival rate, and you must test with arrival rate.

Common mistakes

Testing with VUs a question that's about RPS. What happens: someone wants to know if the API handles 500 RPS and tests it with 50 VUs, seeing it "doesn't fall over." Why it happens: concurrency (users) gets confused with rate (requests per second). How to detect it: with VUs, the load self-limits —you never demand more than the server sustains—, so "it doesn't fall over" doesn't prove it handles 500 RPS; only that it handles 50 VUs. How to fix it: if the question is in RPS, use constant-arrival-rate with that rate; it's the only model that fixes the demand without self-limiting.

Setting maxVUs too low and not noticing the dropped iterations. What happens: constant-arrival-rate is configured with a short maxVUs; the server gets slow, k6 needs more VUs to sustain the rate, runs out of them, and starts dropping iterations —but nobody looks at that number—. Why it happens: it's assumed k6 always achieves the requested rate. How to detect it: if k6 reports dropped iterations, it didn't reach the target rate; your test applied less load than you thought (and that drop is the overload signal). How to fix it: give maxVUs a generous ceiling and read the dropped iterations: they're part of the result, not a technical detail.

Believing the open model is "better" and the closed one "worse." What happens: someone concludes you should always use constant-arrival-rate. Why it happens: it's more realistic for web traffic. How to detect it: not everything is modeled by rate —a batch of 20 fixed workers processing a queue is a real closed system, and there the VUs are the correct model—. How to fix it: use the model that corresponds to your reality: user traffic that arrives without coordinating → arrival rate (open); a fixed number of agents/threads/devices → VUs (closed).

Exercises

Exercise 1 — Open or closed? For each situation, say whether it's better modeled by the open model (arrival rate) or the closed one (VUs). (a) "Thousands of web users arrive to quote without coordinating with each other." (b) "A fleet of 10 IoT devices, each sends a reading, waits for the confirmation, and sends the next." (c) "We expect 800 orders per second on launch day." (d) "20 workers from a pool consume tasks from a queue, one after another."

Exercise 2 — Read the experiment. With the executed output, answer: (a) Why are the p50, the p95, and the maximum so close together at 120 RPS (6.0 / 7.2 / 7.9 ms)? (b) At 450 RPS, 1470 of 1800 were delivered: what happened to the remaining 330 and why? (c) What number marks the boundary between "healthy" and "broken," and where did it come from?

Exercise 3 — Configure the test. You want to verify whether Reservo handles a launch's expected demand: 300 requests per second for 2 minutes. Write the scenarios with constant-arrival-rate and explain why you chose that executor and not constant-vus.

export const options = {
  scenarios: {
    launch_demand: {
      executor: 'constant-arrival-rate',
      rate: 300,
      timeUnit: '1s',        // 300 RPS
      duration: '2m',
      preAllocatedVUs: 100,
      maxVUs: 500,           // generous ceiling in case the server gets slow
    },
  },
};

Summary and next step

In this lesson you met the executor that changes the question at its root. The by-VU executors (lesson 5) fix how many users there are; constant-arrival-rate fixes how many requests per second arrive —the arrival rate—. Behind it are two models of the world: the closed one (VUs, where each user waits for its response and the load self-limits, like a line) and the open one (arrival rate, where the requests come in at a fixed rate and pile up if the server can't keep up, like a conveyor belt). The open model models real traffic better, because users don't stop arriving because your server is slow.

You saw it with numbers that leave no doubt: against a Reservo of ~337 RPS capacity, a load of 120 RPS gave flat, perfect latencies (p95 7.2 ms, everything delivered), but one of 450 RPS —above capacity— broke the system: p95 of 1915 ms, maximum of almost seven seconds, and 330 requests lost. The by-VU model would never have shown that wall, because it would have self-limited before overloading. That's why, when your question is in RPS ("does it handle 500 orders per second?"), you must test with arrival rate.

Before moving on you should be able to: explain the difference between the open model and the closed one with the belt-and-line analogy; say why the by-VU model can't overload beyond N; read the fields of constant-arrival-rate (rate, timeUnit, preAllocatedVUs, maxVUs); and interpret why crossing capacity breaks instead of degrading smoothly.

What comes next is bringing everything in the module together into a single practical decision. You now know the shapes (lesson 4) and the executors (5 and 6); in lesson 7 you learn to choose the profile according to the question: what shape and what executor to use for a load test, a stress test, a spike test, or a soak, and how they combine.

Resources

• k6 — constant-arrival-rate executor — this lesson's executor, with rate, timeUnit, preAllocatedVUs, and maxVUs.
• k6 — Open vs closed models — the official explanation of the difference between fixing VUs and fixing arrival rate; the conceptual heart of this lesson.
• k6 — Dropped iterations — what dropped iterations are and why they're a signal of over-capacity.
• concurrent.futures.ThreadPoolExecutor — Python documentation — the pool the generator sustains an arrival rate with, letting the queue grow when the server saturates.