Estimating Tokens with `len // 4`

Description

The previous lesson made the unit clear — the token — and why it's the one a language model bills by. This lesson builds observability/cost_calculator.py's first real piece: estimate_tokens(text), the function this entire guide is going to use, without exception, to convert a fragment of text into a token count. The formula is deliberately simple — len(text) // 4 — and this lesson's goal isn't just to show it, but to precisely justify it: where that 4 comes from, how close it is to a real tokenizer, and in which cases it's systematically wrong.

This is, in all honesty, an approximation. It never replaces a real tokenizer like the one claude-sonnet-5 uses internally. Every lesson in this guide that uses estimate_tokens is going to label it an order-of-magnitude estimate — enough to reason about the relative cost of different runs, not enough for an exact invoice.

Connection to the module

This lesson delivers observability/cost_calculator.py's first real function: estimate_tokens. Lesson 04 adds the pricing; lesson 05 combines both into estimate_cost_cents and cost_for_run. Everything that follows in this module — and, per this guide's DISEÑO, in the modules that come after — reuses estimate_tokens unchanged.

Where the 4 comes from: a known approximation, not a made-up number

The len(text) // 4 convention wasn't born in this guide — it's a general rule, widely cited in language-model provider documentation, for English text: on average, one token equals, roughly, four characters. It isn't a physical law or a guarantee — lesson 02 already confirmed that with the JSON tool_result example, with a characters-per-token ratio different from prose — it's a useful average when you don't have access to the real tokenizer and need an order-of-magnitude figure, quick to calculate, with no external dependencies.

This guide inherits that convention from agent-fundamentals-and-tool-calling-guide and context-engineering-guide — the same guides that already used it to reason about context budgets — and turns it, here, into the first time it gets used to calculate real money.

def estimate_tokens(text):
    """Estimación de ORDEN DE MAGNITUD: len(texto) // 4. NUNCA un conteo
    exacto de un tokenizer real (como el que usa claude-sonnet-5
    internamente) -- una aproximación declarada, usada en toda esta guía
    porque no hay acceso a un tokenizer real sin llamar a la API."""
    return len(text) // 4

Integer division (//), not floating-point division — the same arithmetic style you already used for Reservo's pro discount (base * 80 // 100) and that you're going to use in every cost calculation for the rest of this guide. The result is always an int, never a decimal.

Worked example: estimate_tokens over real Reservo text

Before calculating any cost, confirm the function over a variety of real fragments, from shortest to longest:

import json
import reservo_tools as rt

samples = {
    "vacío": "",
    "un carácter": "x",
    "tres caracteres": "xyz",
    "cuatro caracteres": "wxyz",
    "pregunta corta": "Reserva Focus pro 3h para Ana",
    "get_quote result": json.dumps({"price_cents": 6000}),
    "list_rooms result": json.dumps(rt.list_rooms()),
    "respuesta final": "Reservé Focus pro por 3 horas para Ana. Total $60.00. Confirmación #1.",
}
for label, text in samples.items():
    print(f"{label:<20} len={len(text):>4}  estimate_tokens={estimate_tokens(text):>4}")

What to expect:

vacío                len=   0  estimate_tokens=   0
un carácter          len=   1  estimate_tokens=   0
tres caracteres      len=   3  estimate_tokens=   0
cuatro caracteres    len=   4  estimate_tokens=   1
pregunta corta       len=  29  estimate_tokens=   7
get_quote result     len=  21  estimate_tokens=   5
list_rooms result    len= 122  estimate_tokens=  30
respuesta final      len=  70  estimate_tokens=  17

Stop at the first four rows, because they reveal this convention's most important limit: any text under four characters gets estimated at 0 tokens, no exceptions. "x", "xyz" — three characters, almost a real token in any tokenizer that exists — round, via integer division, to exactly 0. This isn't a bug: it's the direct, expected consequence of //, and the reason this guide never calls this estimate "exact." Only with "wxyz" — four characters — does the estimate stop being 0.

Confirming the limit with real Reservo text: short names

The "x" case might look artificial. It isn't — Reservo works with people's names all the time, and several of the names you've already seen in this guide ("Ana", "Luis") have fewer than four characters:

nombres = ["Ana", "Luis", "Nina", "Rui", "Omar", "Carla", "Sofía"]
for nombre in nombres:
    print(f"{nombre!r:<10} len={len(nombre)}  estimate_tokens={estimate_tokens(nombre)}")

What to expect:

'Ana'      len=3  estimate_tokens=0
'Luis'     len=4  estimate_tokens=1
'Nina'     len=4  estimate_tokens=1
'Rui'      len=3  estimate_tokens=0
'Omar'     len=4  estimate_tokens=1
'Carla'    len=5  estimate_tokens=1
'Sofía'    len=5  estimate_tokens=1

"Ana" and "Rui" — two of the real names appearing in this guide's canonical script — get estimated at 0 tokens each, when any real tokenizer would count them as at least one token. This is the exact reason this guide never calculates the cost of a tiny, isolated fragment — like a single name — on its own, and always does it over a complete run's accumulated text, where that rounding error becomes proportionally less important. Lesson 06 is going to confirm, with real numbers, why summing text before applying // 4 is better than applying // 4 to each fragment separately and summing afterward.

What exactly is lost compared to a real tokenizer

It's worth being precise about the two ways this estimate drifts from a real count, because every lesson that follows in this guide inherits both limitations without mentioning them again:

1. It's blind to content, only looks at length. Lesson 02's Exercise 3 already confirmed it: a meaningless text and a real sentence of the same length produce the same estimate. A real tokenizer tells known words apart from odd sequences; len(text) // 4 distinguishes nothing.
2. It's blind to language. The "four characters per token" rule was calibrated, historically, over English text. Spanish — with accents, with words that tend to be longer than their English equivalents — doesn't necessarily follow the same ratio in a real tokenizer. This guide keeps using // 4 uniformly, in Spanish and English, precisely because it's an order-of-magnitude estimate, not one calibrated per language — and it says so, here, in plain terms.

Neither limitation invalidates the use this guide gives the function: reasoning about the relative cost of different runs (which one cost more? why?), and calculating an order of magnitude of total cost, without needing a real API call just to count tokens.

Common mistakes

1. Calling this function "the token count" without the qualifier "estimated." Every mention of estimate_tokens in this guide — in code, in prose — exists, precisely, so it's never read as an exact count. The function's name already says it: estimate_, not count_.

2. Applying the estimate to a tiny fragment and trusting the result. As the names example confirmed, "Ana" and "Rui" get estimated at 0 tokens — a technically correct result according to the formula, but misleading if read as "this name costs nothing." A run's real cost is never calculated over an isolated fragment that short — always over the accumulated text, as lesson 05 shows.

3. Using floating-point division (/) instead of integer division (//). len(text) / 4 gives a float (for example, 7.25) — a fractional number of tokens makes no sense, because a token is a discrete unit. This guide uses // everywhere it estimates a token count, without exception, the same integer-arithmetic discipline you already saw in Reservo's pricing.

4. Thinking installing tiktoken or a real tokenizer would "fix" this guide. That's not the goal. This guide's DISEÑO's hard rule prohibits any external dependency for this, precisely because this guide's point isn't counting tokens with perfect precision — that's a problem existing libraries have already solved — but learning to reason about cost, with an honest, reproducible approximation that requires installing nothing.

5. Summing individual fragments' estimates when the text could be concatenated first. The short-names example already hinted at the problem: every tiny fragment loses information by rounding down separately. Lesson 06 demonstrates it with concrete numbers, over a real scaling case.

Exercises

Exercise 1: Estimate the tokens of three Reservo texts, by hand first (Easy)

Before running anything, calculate by hand the token estimate for: (a) "Cancela la reserva 5" (21 characters), (b) "{\"cancelled\": true}" (19 characters), (c) "Studio" (6 characters). Then confirm with estimate_tokens.

for text in ["Cancela la reserva 5", '{"cancelled": true}', "Studio"]:
    print(f"{text!r:<25} len={len(text):>3}  estimate_tokens={estimate_tokens(text)}")

'Cancela la reserva 5'    len= 21  estimate_tokens=5
'{"cancelled": true}'     len= 19  estimate_tokens=4
'Studio'                  len=  6  estimate_tokens=1

Exercise 2: Find the longest text that still estimates at 0 tokens (Medium)

Without running anything first: what's the maximum length, in characters, a text can have and still estimate at 0 tokens with len(text) // 4? Confirm your answer by testing that exact length and the length immediately above it.

print("3 caracteres:", estimate_tokens("abc"))     # 3 // 4 = 0
print("4 caracteres:", estimate_tokens("abcd"))     # 4 // 4 = 1

3 caracteres: 0
4 caracteres: 1

Exercise 3: Quantify the loss from summing per fragment instead of concatenating (Hard)

Take this list of eight Reservo member names: ["Ana", "Sofia", "Diego", "Luis", "Carla", "Marta", "Nina", "Omar"]. Calculate the token estimate two ways: (a) applying estimate_tokens to each name separately and summing the eight results, (b) concatenating all eight names into a single string and applying estimate_tokens once. Compare both results and explain the difference.

fragments = ["Ana", "Sofia", "Diego", "Luis", "Carla", "Marta", "Nina", "Omar"]

per_fragment_sum = sum(estimate_tokens(f) for f in fragments)
concatenated = estimate_tokens("".join(fragments))

print("longitudes individuales:", [len(f) for f in fragments])
print("longitud total          :", sum(len(f) for f in fragments))
print("(a) suma por fragmento  :", per_fragment_sum)
print("(b) concatenado una vez :", concatenated)

longitudes individuales: [3, 5, 5, 4, 5, 5, 4, 4]
longitud total          : 35
(a) suma por fragmento  : 7
(b) concatenado una vez : 8

Summary and next step

• We built estimate_tokens(text): len(text) // 4, observability/cost_calculator.py's first real function, inherited from the convention already established in agent-fundamentals and context-engineering.
• We confirmed, with real execution, over eight real Reservo fragments, that it's an order-of-magnitude estimate — never an exact count — and that texts of three characters or fewer (including real names like "Ana" and "Rui") always estimate at 0 tokens.
• We precisely named the two limitations this guide accepts on purpose: it's blind to content, and it isn't calibrated per language.
• We confirmed, with real numbers, that summing small fragments' estimates separately loses more precision than concatenating the text and estimating once — the argument lesson 06 picks back up at the scale of thousands of runs.

Next lesson: 04 — The claude-sonnet-5 Pricing. With estimate_tokens now built and tested, we fix the second piece needed to calculate money: the price, in cents, of each million input and output tokens.

Additional resources

1. Anthropic — Token counting — How the Claude API really counts tokens; the exact count this lesson measures its own estimate against.
2. Python — arithmetic operators (//) — Integer division, the exact foundation of estimate_tokens and every cost calculation for the rest of this guide.
3. Python — len() — The function that measures any Python string's length, this estimate's only real input.
4. Anthropic — Building effective agents — On why reasoning about an agentic system's relative cost, even with an approximate estimate, is more valuable than not measuring it at all.
5. Python 3.14 — What's New — The version every line of code in this lesson ran on.