What Latency Are We Measuring?

Description

"Latency" sounds like a single figure — "this run took 185 milliseconds" — but that figure is, really, the sum of several smaller questions, and mixing them up is the first mistake almost anyone measuring latency for the first time makes. This lesson precisely separates two related but distinct questions: how long does an individual tool call take? and how long does the whole run take? The second depends on the first, but knowing book_room takes 120 ms isn't the same as knowing an entire run, with three tool calls, took 185 ms — the second figure tells you something about the user's complete experience; the first tells you exactly where that time went.

Connection to the module

Lesson 01 already previewed the TOOL_LATENCY_MS dictionary, inherited unchanged from Module 1. This lesson uses it for the first time in this module — without formalizing it yet as the central artifact lesson 04 is going to fix — to answer the two questions above over a real run. The full development of the total sum, with its nuance on rejected tool_uses, is lesson 05's job; this lesson deliberately stays with the simple case: a run with no error along the way.

Analogy: the dashboard, with two gauges measuring different things

You already know, from Module 1, the analogy of the car without a dashboard. Now that the dashboard has a temperature gauge — latency — it's worth noting a real dashboard doesn't have a single temperature gauge: a car with several systems — engine, transmission, brakes — could, in theory, show you each one's temperature separately, in addition to a general reading. They serve different questions: a specific component's temperature tells you where to focus a repair; the general reading tells you whether, right now, the whole car is operating within a reasonable range. Neither replaces the other.

That is, precisely, the difference between one tool call's latency and a run's total latency. The first — how long book_room specifically takes — tells you where to focus attention if something's slow. The second — how long the whole run took — tells you whether, overall, that user's experience was reasonable. You need both.

Worked example: the two questions, over the same run

Question 1: how long does an individual tool call take?

This is the simpler of the two questions — it's, literally, a dictionary lookup. TOOL_LATENCY_MS, inherited from Module 1 without changing a single value:

TOOL_LATENCY_MS = {
    "list_rooms": 40,
    "get_quote": 25,
    "book_room": 120,
    "cancel_booking": 90,
}

for tool_name, latency_ms in TOOL_LATENCY_MS.items():
    print(f"{tool_name:15} {latency_ms:4} ms")

What to expect:

list_rooms       40 ms
get_quote        25 ms
book_room       120 ms
cancel_booking   90 ms

Four numbers, four tools, no execution involved at all — this is a reference table, not a measurement. Lesson 04 is going to stop and explain why these four numbers are what they are; for now, it's enough to confirm that answering "how long does book_room take?" is as simple as TOOL_LATENCY_MS["book_room"].

Question 2: how long does the whole run take?

This question is different because a run almost never calls a single tool — it calls several, in sequence, and the total latency is the sum of what each one took. Go back to Sofía's script, from Module 1 lesson 05: no error along the way, Boardroom pro 1 hour.

import reservo_agent as ra

script_sofia = [
    {"stop_reason": "tool_use", "content": [
        {"type": "tool_use", "id": "toolu_01", "name": "list_rooms", "input": {}}]},
    {"stop_reason": "tool_use", "content": [
        {"type": "tool_use", "id": "toolu_02", "name": "get_quote",
         "input": {"room": "Boardroom", "tier": "pro", "hours": 1}}]},
    {"stop_reason": "tool_use", "content": [
        {"type": "tool_use", "id": "toolu_03", "name": "book_room",
         "input": {"room": "Boardroom", "tier": "pro", "hours": 1, "member": "Sofía"}}]},
    {"stop_reason": "end_turn", "content": [
        {"type": "text", "text": "Reservé Boardroom pro por 1 hora para Sofía. Total $64.00. Confirmación #1."}]},
]

final, history = ra.run_reservo_agent("Reserva Boardroom pro 1h para Sofía", script_sofia)

tools_called = [
    block["name"]
    for turn in history if turn["role"] == "assistant" and isinstance(turn["content"], list)
    for block in turn["content"] if block["type"] == "tool_use"
]
total_latency_ms = sum(TOOL_LATENCY_MS[name] for name in tools_called)

print("tools llamadas   :", tools_called)
for name in tools_called:
    print(f"  {name:15} {TOOL_LATENCY_MS[name]:4} ms")
print("latencia total    :", total_latency_ms, "ms")

What to expect:

tools llamadas   : ['list_rooms', 'get_quote', 'book_room']
  list_rooms       40 ms
  get_quote        25 ms
  book_room       120 ms
latencia total    : 185 ms

185 ms, the same figure you already saw in Module 1: 40 + 25 + 120 = 185. This run's total latency is not a new number measured separately — it's, literally, the sum of the three individual latencies Question 1 already knew how to calculate. This code works fine for this case, because no tool call in this script failed — every tool_use appearing in tools_called corresponds to a tool that actually ran. Lesson 05 is going to show you, with Ana's script, why this exact code would be incorrect if some tool call had been rejected on validation.

Why neither question lived in history before this module

It's worth confirming once more, with this module's precise vocabulary: history — the same history you just walked through above — never had a time field. TOOL_LATENCY_MS isn't part of the agent's output; it's an external table you consult, afterward, using the tool name that's already in history (block["name"]) as the key. That distinction matters: latency isn't "extracted" from history the way, say, booking_id would be extracted from a tool_result — it's calculated, combining what history does know (which tool got called) with what TOOL_LATENCY_MS declares (how long that tool "takes," according to this guide's model).

Common mistakes

1. Confusing "a tool's latency" with "a run's latency" when talking in the abstract. They're the same unit (milliseconds) but answer different questions — and a run with a single tool call has, by definition, the same individual latency as total, which can make them look like "the same thing" when in fact they only coincide by that particular case's coincidence.

2. Summing the latency of tools appearing in tools_called without verifying they actually ran. This lesson's code works because Sofía's script has no error at all — but tools_called, as written here, includes any tool_use, whether or not its validation failed. Lesson 05 precisely fixes this.

3. Thinking a run's total latency "averages" its tools' latencies, instead of summing them. No — they're summed. A run with three tool calls doesn't take "the average of the three"; it takes the sum of the three, because (in this guide's sequential model, inherited from agent-fundamentals) the agent dispatches one tool, waits for its result, and only then decides the next one.

4. Looking for the total latency in some field of final or of a single history turn. It doesn't exist. Total latency is always calculated by walking the whole run and summing — it's never a value that comes ready-made in any individual block.

5. Forgetting TOOL_LATENCY_MS is Module 1's same dictionary, not a new one for this module. This lesson reuses it without changing a single value — declaring it again with different numbers would break continuity with every figure you already saw in Module 1.

Exercises

Exercise 1: Calculate the total latency of a single-tool run (Easy)

Without running anything: if a run only calls cancel_booking, with no other tool, what's its total latency? Confirm with code, using TOOL_LATENCY_MS and a one-step script.

script_cancel = [
    {"stop_reason": "tool_use", "content": [
        {"type": "tool_use", "id": "toolu_01", "name": "cancel_booking", "input": {"id": 1}}]},
    {"stop_reason": "end_turn", "content": [
        {"type": "text", "text": "Cancelé la reserva #1."}]},
]
final, history = ra.run_reservo_agent("Cancela la reserva 1", script_cancel)
tools_called = [
    block["name"]
    for turn in history if turn["role"] == "assistant" and isinstance(turn["content"], list)
    for block in turn["content"] if block["type"] == "tool_use"
]
print("latencia total:", sum(TOOL_LATENCY_MS[name] for name in tools_called), "ms")

latencia total: 90 ms

Exercise 2: Do two different tool combinations exist with the same total latency? (Medium)

Using TOOL_LATENCY_MS, investigate whether two different subsets of the four tools exist (with none repeated within the same script) whose total latency is exactly equal. Don't solve it by eyeballing loose combinations — write code that walks every possible combination and confirms it.

from itertools import combinations

values = {"list_rooms": 40, "get_quote": 25, "book_room": 120, "cancel_booking": 90}
sums = {}
for r in range(1, 5):
    for combo in combinations(values.items(), r):
        total = sum(v for _, v in combo)
        names = tuple(sorted(k for k, _ in combo))
        sums.setdefault(total, set()).add(names)

empates = {total: combos for total, combos in sums.items() if len(combos) > 1}
print("sumas con mas de una combinacion de tools distintas:", empates)

sumas con mas de una combinacion de tools distintas: {}

Exercise 3: Design a script whose total latency is greater than any individual tool's, but less than the sum of all four (Hard)

Without running anything first: design a script (using two or three of the four tools, one repeated if needed) whose total latency falls strictly between 120 (the most expensive individual tool) and 275 (the sum of all four, from Module 1). Calculate the expected latency, and confirm with code.

script_mixed = [
    {"stop_reason": "tool_use", "content": [
        {"type": "tool_use", "id": "toolu_01", "name": "get_quote",
         "input": {"room": "Studio", "tier": "basic", "hours": 1}}]},
    {"stop_reason": "tool_use", "content": [
        {"type": "tool_use", "id": "toolu_02", "name": "book_room",
         "input": {"room": "Studio", "tier": "basic", "hours": 1, "member": "Luis"}}]},
    {"stop_reason": "end_turn", "content": [
        {"type": "text", "text": "Reservé Studio basic 1h para Luis. Confirmación #1."}]},
]
final, history = ra.run_reservo_agent("Reserva Studio basic 1h para Luis", script_mixed)
tools_called = [
    block["name"]
    for turn in history if turn["role"] == "assistant" and isinstance(turn["content"], list)
    for block in turn["content"] if block["type"] == "tool_use"
]
print("tools:", tools_called)
print("latencia total:", sum(TOOL_LATENCY_MS[name] for name in tools_called), "ms")

tools: ['get_quote', 'book_room']
latencia total: 145 ms

Summary and next step

• We distinguished two latency questions: one tool call's (a direct lookup on TOOL_LATENCY_MS) and a run's total (the sum of the tools that actually took part).
• We confirmed, with real execution, that Sofía's run — with no error at all along the way — has a total latency of 185 ms, the exact sum of list_rooms (40) + get_quote (25) + book_room (120).
• We deliberately left the case with errors along the way pending — when a tool_use gets rejected on validation, this example's simple sum would give an incorrect number. That nuance is, precisely, lesson 05's content.

Next lesson: 03 — The Honesty Problem: Modeled vs. Real Clock. Before building further on top of TOOL_LATENCY_MS, we answer the underlying question this guide can't avoid: why model latency instead of measuring it with the real clock, and what's lost by doing so?

Additional resources

1. Anthropic — Tool use (function calling) overview — The exact tool_use shape this lesson walks to extract each called tool's name.
2. Python — dictionaries — The data structure behind TOOL_LATENCY_MS, as simple as a lookup table.
3. Python — itertools.combinations — Used in Exercise 2 to confirm, with code, that no distinct tool combination ties in total latency.
4. Python 3.14 — What's New — The version every line of code in this lesson ran on.