Module 1: Why Load And Performance Testing

7. First contact: a k6 script and a Python generator

Next →

Overview

The moment has come to leave theory and launch traffic. In this lesson you make your first contact with load, and you do it twice over, so you see the two sides of the same coin. First, the industrial side: a minimal k6 script that hits POST /quote of Reservo and its output summary —presented as labeled content, because k6 isn't installed in this environment—, so you recognize what a "real" load test looks like. Second, the executable, hands-on side: a mini load generator written in Python with concurrent.futures and urllib, that hits the same /quote with N concurrent requests and measures the real latency —minimum, average, maximum, p95—, with actually executed output we cite here. Seeing the two side by side makes the idea crystal clear: k6 does conceptually the same thing as the Python generator, just industrialized and at scale. If you understand the thirty-line generator, you understand what k6 does inside.

Connection to the module: this lesson brings together everything before. It uses the target from lesson 6 (the Reservo API), answers lesson 3's questions (p95 latency) with real numbers, and contrasts lesson 5's tool (k6, as content) with its executable equivalent in Python. It's the dress rehearsal for the mini-project (lesson 8), where you'll do this whole journey with your own hands. The in-depth anatomy of the k6 script —what exactly a VU is, how options is configured, how check works— is module 2; here we see it whole but without dissecting it piece by piece.

Two ways to measure the taco stand's line

You've got the taco stand set up (the Reservo API). You want to measure how long it takes to dispatch when people arrive. There are two ways to bring in the test customers.

The first: you hire a market research firm with a hundred professional actors, walkie-talkies, and precision stopwatches. They arrive, coordinate, form realistic queues, measure everything to the millisecond, and hand you a polished report with percentiles and charts. It's powerful, scales to a thousand actors if needed, and is reusable. But it's an external firm with its own way of working, that you configure and hire. That's k6.

The second: you grab ten friends, tell them "when I say go, all order a taco at once," and you yourself, with your phone's stopwatch, note how long each one took. It's artisanal, doesn't scale to a thousand, but you set it up in five minutes, you understand exactly what it measures because you wrote it, and the numbers are just as real. That's the mini generator in Python.

Both measure the same line of the same stand. The professional firm (k6) is what you'll use in production; the ten friends (the Python generator) are so you understand hands-on what "launch load and measure latency" means, without a black box. In this guide, since we haven't hired the firm (k6 isn't installed), we see it by its brochure (content) and do the real measurement with the friends (Python).

The executable side: the mini generator in Python

Let's start with the one that does run, because touching the real numbers is what makes everything else make sense. The generator is a short script that launches N requests to /quote, using a thread pool so many are in flight at once (that concurrency is what creates the load, as we saw in lesson 2), and measures how long each one takes.

"""Mini load generator in Python — hits Reservo's /quote in concurrency.

Launches N concurrent requests with a thread pool (concurrent.futures) and
measures each one's REAL latency with time.perf_counter. Reports minimum,
maximum, average, and p95. It's the executable sibling of the k6 script (which
is content): here we see real numbers.
"""
import json
import statistics
import sys
import time
import urllib.request
from concurrent.futures import ThreadPoolExecutor

BASE_URL = sys.argv[1]            # e.g. http://127.0.0.1:51568
TOTAL_REQUESTS = int(sys.argv[2]) if len(sys.argv) > 2 else 200
CONCURRENCY = int(sys.argv[3]) if len(sys.argv) > 3 else 20


def one_quote():
    """Makes ONE POST /quote request and returns (latency_ms, price_cents)."""
    payload = json.dumps({"room": "Focus", "tier": "basic", "hours": 3}).encode()
    req = urllib.request.Request(
        f"{BASE_URL}/quote", data=payload,
        headers={"Content-Type": "application/json"}, method="POST",
    )
    start = time.perf_counter()
    with urllib.request.urlopen(req) as resp:
        body = json.loads(resp.read())
    latency_ms = (time.perf_counter() - start) * 1000  # seconds -> milliseconds
    return latency_ms, body["price_cents"]


def main():
    latencies = []
    prices = []
    wall_start = time.perf_counter()
    # ThreadPoolExecutor keeps CONCURRENCY requests in flight at once:
    # THAT concurrency is what creates the load (not the total number of requests).
    with ThreadPoolExecutor(max_workers=CONCURRENCY) as pool:
        futures = [pool.submit(one_quote) for _ in range(TOTAL_REQUESTS)]
        for fut in futures:
            latency_ms, price = fut.result()
            latencies.append(latency_ms)
            prices.append(price)
    wall_seconds = time.perf_counter() - wall_start

    latencies.sort()
    # p95 = the value below which 95% of the latencies fall.
    p95_index = int(len(latencies) * 0.95)
    p95 = latencies[min(p95_index, len(latencies) - 1)]

    print(f"requests ............ {TOTAL_REQUESTS} (concurrency {CONCURRENCY})")
    print(f"all returned ........ price_cents={prices[0]} "
          f"(correct: {all(p == 7500 for p in prices)})")
    print(f"total duration ...... {wall_seconds:.3f} s")
    print(f"throughput .......... {TOTAL_REQUESTS / wall_seconds:.1f} req/s")
    print(f"latency min ......... {min(latencies):.2f} ms")
    print(f"latency avg ......... {statistics.mean(latencies):.2f} ms")
    print(f"latency max ......... {max(latencies):.2f} ms")
    print(f"latency p95 ......... {p95:.2f} ms")


if __name__ == "__main__":
    main()

The three ideas that make this a load generator (and not a simple request loop):

  • The concurrency is created by ThreadPoolExecutor(max_workers=CONCURRENCY). The pool keeps CONCURRENCY requests in flight at the same time. That "at the same time" is the load —recall lesson 2: contention is born from simultaneous requests, not the accumulated total—. Changing CONCURRENCY from 20 to 50 is, conceptually, going from 20 to 50 virtual users.
  • Latency is measured with time.perf_counter() around each request. perf_counter is Python's high-resolution clock, meant for measuring short intervals. We mark just before sending and just after receiving; the difference (in milliseconds) is the real latency of that request, with its queue wait included.
  • The p95 is computed by sorting and cutting. All the latencies are sorted and the one at the 95% position is taken. Without averaging: the percentile sorts and cuts, as we saw in lesson 3. (In module 3 we'll use statistics.quantiles to do it more rigorously; here, the direct version lets the mechanism show.)

Running it (real output)

We start the Reservo API from lesson 6, read its port, and launch the generator. All of this is real output, run against the server on localhost. First with 200 requests and concurrency 20:

What to expect — 200 Focus/basic/3h quotes with 20 clients at once; all correct (7500), and a p95 latency of a few milliseconds:

$ python3.14 load_generator.py http://127.0.0.1:51568 200 20
requests ............ 200 (concurrency 20)
all returned ........ price_cents=7500 (correct: True)
total duration ...... 0.048 s
throughput .......... 4157.5 req/s
latency min ......... 2.67 ms
latency avg ......... 4.50 ms
latency max ......... 20.20 ms
latency p95 ......... 17.21 ms

There you have it: your first real load measurement. With 20 concurrent clients, Reservo sustained ~4157 requests per second, with a p95 of 17 ms —and all responses were correct (7500)—. Let's raise the concurrency to 50 to see the effect:

What to expect — going from 20 to 50 concurrent, there's more contention: the p95 rises:

$ python3.14 load_generator.py http://127.0.0.1:51568 500 50
requests ............ 500 (concurrency 50)
all returned ........ price_cents=7500 (correct: True)
total duration ...... 0.103 s
throughput .......... 4856.2 req/s
latency min ......... 2.74 ms
latency avg ......... 9.60 ms
latency max ......... 46.19 ms
latency p95 ......... 29.13 ms

The p95 went from 17 to 29 ms when we more than doubled the concurrency: more users at once, more queue, more latency in the tail. This is real performance testing, with thirty lines of Python: you launched load, measured percentile latency, and observed how latency responds to concurrency. Everything that comes in the guide is refining this.

The industrial side: the same test in k6 (content)

Now the other side. This is how the same test —hitting Reservo's POST /quote— would be written in k6. Important label: this script and its summary are CONTENT, not a run of this environment (k6 isn't installed here). They're correct and faithful to k6's official documentation; their in-depth anatomy is module 2.

// CONTENT (not run here): the SAME /quote test, in k6.
// See grafana.com/docs/k6. It would run with: k6 run quote_test.js
import http from "k6/http";
import { check } from "k6";

// options: the "shape" of the load. 50 virtual users for 10 seconds
// (it's the analog of the Python generator's max_workers=50).
export const options = {
  vus: 50,
  duration: "10s",
};

// The default function is what each VU runs in a loop (one "request").
export default function () {
  const url = "http://127.0.0.1:8000/quote";
  const payload = JSON.stringify({ room: "Focus", tier: "basic", hours: 3 });
  const params = { headers: { "Content-Type": "application/json" } };

  const res = http.post(url, payload, params);

  // check(): verifies correctness UNDER load, like the generator's "correct: True".
  check(res, {
    "status is 200": (r) => r.status === 200,
    "price_cents is 7500": (r) => r.json("price_cents") === 7500,
  });
}

Notice the parallelism with the Python generator, line by concept:

ConceptPython generatork6 script
Concurrency (users at once)max_workers=50vus: 50
What each "user" doesone_quote()the default function
The requesturllib.request to /quotehttp.post(url, payload, params)
Verify correctness under loadall(p == 7500 ...)check(res, {...})
How long it laststotal number of requestsduration: "10s"
Measure latency/percentilestime.perf_counter + p95 by handk6 measures it on its own (http_req_duration)

The main difference: in the Python generator you time and compute the p95; in k6, the runtime does it for you and delivers it in the summary. This is what that summary would look like —also content, not run here; the numbers are consistent with what the Python generator actually measured (~4856 req/s, p95 ~29 ms), as it should be, since they do the same thing—:

// CONTENT (not run here): shape of the k6 run summary. See grafana.com/docs/k6
     ✓ status is 200
     ✓ price_cents is 7500

     checks.........................: 100.00%  ✓ 96000     ✗ 0
     data_received..................: 6.1 MB   610 kB/s
     data_sent......................: 5.3 MB   530 kB/s
     http_req_duration..............: avg=9.6ms  min=2.7ms  med=8ms  max=46ms  p(90)=22ms  p(95)=29ms
     http_req_failed................: 0.00%    ✓ 0         ✗ 48000
     http_reqs......................: 48000    4800/s
     iteration_duration.............: avg=10.4ms min=2.9ms  max=48ms
     iterations.....................: 48000    4800/s
     vus............................: 50       min=50      max=50
     vus_max........................: 50       min=50      max=50

Read it with what you already know: checks at 100% (correctness under load, like the generator's correct: True), http_req_duration with its p(95)=29ms (the percentile latency from lesson 3), http_reqs at 4800/s (the throughput), http_req_failed at 0.00% (no request failed), and vus: 50 (the concurrency). It's the same portrait your Python generator painted, done by the professional tool. That correspondence is the whole point of the lesson: k6 isn't magic; it's your thirty-line generator, industrialized.

Common mistakes

Setting max_workers=1 (or few) and believing you're generating load. What happens: the generator is run with concurrency 1, a very low p95 is seen, and it's concluded "it holds up." Why it happens: total gets confused with simultaneity. How to detect it: if CONCURRENCY is 1, there's no contention (you saw it in lesson 2: p95 ≈ average). How to fix it: load is set by concurrency; raise it (20, 50, 100) and observe how the p95 changes.

Measuring latency including things that aren't the request. What happens: someone puts the perf_counter before building the payload or creating the pool, and measures too much. Why it happens: what "the request" is isn't well bounded. How to detect it: suspiciously high latencies or ones that include preparation time. How to fix it: time only the HTTP round trip —just before sending, just after receiving—, as one_quote() does. Everything else (building the JSON, creating the pool) goes outside the stopwatch.

Confusing the k6 script (content) with something that ran here. What happens: someone sees the k6 summary and cites it as a "measured result." Why it happens: the summary looks very real. How to detect it: if you don't have k6 installed, you didn't run k6; its output is content. How to fix it: keep the guide's honesty —the executed numbers are the Python generator's (with its command python3.14 load_generator.py ...); k6's are the expected shape, labeled as content—. If you install k6, the script runs as-is and produces its own real summary.

Exercises

Exercise 1 — Map Python↔k6. For each element of the Python generator, say what its equivalent is in the k6 script. (a) max_workers=50. (b) one_quote(). (c) all(p == 7500 for p in prices). (d) the manual p95 computation with perf_counter.

See solution
  • (a) max_workers=50 ↔ vus: 50 (the concurrency, the virtual users at once).
  • (b) one_quote() ↔ the default function (what each user/VU runs).
  • (c) all(p == 7500 ...) ↔ the check(res, {"price_cents is 7500": ...}) (verifying correctness under load).
  • (d) the manually computed p95 ↔ the http_req_duration with its p(95) that k6 measures and reports on its own (in k6 you don't compute it; the runtime does).

Exercise 2 — Interpret two runs. With 20 concurrent the p95 was 17.21 ms; with 50, it was 29.13 ms. (a) Why did the p95 rise when concurrency rose? (b) The throughput barely changed (4157 → 4856 req/s) while the p95 almost doubled: what does that suggest about the state of the system? (c) If you wanted to find the breaking point, what would you do from here?

See solution
  • (a) More concurrent clients → more requests compete for resources (threads, CPU) → more requests wait their turn → more latency in the tail, which the p95 captures.
  • (b) That throughput barely rises while the p95 spikes suggests the system is approaching saturation: you no longer get much more work per second, and the cost of adding more load is only more wait (more latency). It's the prelude to the breaking point.
  • (c) Keep raising the concurrency in steps (100, 200, 400...) and observe where the p95 gets out of control non-linearly or the error rate stops being 0%. That's a stress test (lesson 4), built with load profiles (module 4).

Exercise 3 — Add the p50 (median). The generator reports min, average, max, and p95. Describe in two or three sentences how you'd add the median (p50) to the report using the already-sorted latencies list, and why comparing p50 with p95 is informative.

See solution

Since latencies is already sorted, the p50 is the value at the 50% position: p50 = latencies[int(len(latencies) * 0.50)] (or statistics.median(latencies), which does exactly that). It would be printed with one more line, just like the p95. Comparing p50 with p95 is informative because the p50 describes the typical experience (half the users saw this or less) and the p95 describes the tail (the worst 5%): if the p95 is much larger than the p50, there's a long tail —contention, occasional spikes— that the typical user doesn't notice but the unlucky one does. That p50↔p95 gap is one of the most useful signals of a system under pressure.

Summary and next step

In this lesson you made your first contact with load, from both sides. On the executable side, you wrote (and actually ran) a mini generator in Python that uses ThreadPoolExecutor to launch N concurrent requests to /quote, measures each latency with time.perf_counter, and reports min/average/max/p95. You saw your first real measurements: p95 of 17 ms with 20 concurrent, 29 ms with 50 —latency responding to concurrency before your eyes, with all responses correct (7500)—. On the industrial side, you saw the same test written in k6 (as labeled content) and its summary, and confirmed the exact parallelism: vus ↔ max_workers, default ↔ one_quote, check ↔ the correctness verification, http_req_duration/p(95) ↔ your manually computed p95.

The idea to take from the lesson is liberating: k6 isn't a magic black box; it does conceptually the same thing as your thirty-line generator, just at scale, with more precision, and with the metrics computed for you. Understanding the generator is understanding k6 inside.

Before moving on you should be able to: explain which line of the generator creates the load (the pool's concurrency) and which line measures latency (the perf_counter around the request); map each part of the generator to its k6 equivalent; and interpret how the p95 rises with concurrency.

What comes next is doing this whole journey with your own hands, from start to finish. In lesson 8, the mini-project: you bring up the Reservo API, write your own generator that hits it with N concurrent requests and reports the real latency, and write the equivalent k6 script as content. It's the synthesis of the whole module.

Resources

← Previous6. The Reservo API and seeing it respondNext →8. Mini-project: your first load measurement