2. Why Enlace desperately needs a cache

Description

In the previous lesson we set the number that governs the module: 100:1, ~4,000 reads per second. But a number by itself convinces no one; you have to feel what it means. This lesson translates that 100:1 into two concrete pressures that make the cache stop being a nice idea and become a necessity. The first is volume: a single database hit 4,000 times per second, all day, becomes the bottleneck of the whole system. The second is latency: each of those reads, if it has to go to the database, costs on the order of 50 milliseconds, while reading the same data from RAM costs on the order of 1 millisecond. Fifty times faster. That gap is the engine of everything that follows.

With those two pressures on the table, the lesson defines precisely what a cache is —a key-value dictionary that lives in RAM, close to the compute— and why a URL shortener is its perfect case study: it reads much more than it writes, and its data is tiny and immutable (a short_code points to a long_url that almost never changes). By the end you'll know, with numbers you reproduce, exactly how much work and how much time putting a cache in front of Enlace's database saves you.

Connection to the module: lesson 1 gave you the map and the vocabulary; this one gives you the quantified motivation. It's the "why" that makes the six lessons that follow mandatory. Lesson 3 will take this motivation and turn it into a concrete pattern —cache-aside—. Lesson 5 will take the latency gap we measure here (1 ms vs 50 ms) and put it into the formula L = h·L_cache + (1−h)·L_db to compute the system's average latency. Here we don't implement yet: here we understand, with numbers, why we have to do it.

The town well and the barrel in the kitchen

Think of it this way. You live in a town with a single well, in the central square. The well's water is the source of truth: it's always there, it never runs out, the whole town drinks from it. But it's far: going to the square, drawing water, and coming back takes fifteen minutes. As long as it's just you, you tolerate the fifteen minutes. The problem appears when the whole town needs water at once: an enormous line forms at the well, and now it's not fifteen minutes, it's two hours of waiting because there are a hundred people ahead of you. The well can't keep up. Not because the water runs out, but because a single well doesn't serve a thousand people at once.

What do you do? You put a barrel in your kitchen. Every morning you fill it with a trip to the well, and during the day you drink from the barrel: the water is two steps away, no line, instantly. You only go back to the well when the barrel empties. And notice the two things you resolved at once. First, the latency: drinking from the barrel takes seconds instead of the well's fifteen minutes. Second, and more important for the whole town, the load on the well: if the thousand people have their barrel, the well goes from receiving a thousand visits a day to receiving a thousand refill trips —far fewer, because each barrel serves dozens of sips per trip to the well—. The line disappears.

Those two reliefs are exactly the two a cache gives, and they're the two pressures of this lesson. The barrel is the cache (cache), in RAM, two steps from the compute. The well is the database (db), the source of truth, far away and with limited capacity for simultaneous service. "Drink from the barrel instead of walking to the well" resolves the latency (1 ms vs 50 ms). "Have the well receive far fewer visits" resolves the volume (the database stops seeing 4,000 reads/s). And that's why Enlace —where the same water, I mean, the same link, is requested over and over— is the perfect town to put barrels in.

A cache resolves two problems at once: it makes each read much faster (latency), and it takes most of the reads off the database (volume). In a read-heavy system like Enlace, both reliefs are enormous, because the same data is requested a great many times.

Pressure number one: the volume on the database

Let's start with the load. A database isn't an infinite source: each query consumes a connection, a bit of CPU, a bit of disk, and a bit of network, and there's a limit to how many it can serve per second before it starts to queue up —just like the well—. That limit depends on the hardware and the type of query, but for a typical relational database serving indexed reads, a single comfortable server handles on the order of hundreds to a few thousand queries per second before the latency starts to shoot up. Enlace wants to do ~4,000 reads per second to it, sustained, all day. That's right at the edge of, or past, what a single instance serves with room to spare.

Let's put it in perspective with the whole-day calculation, reproducing it:

# db_load.py — the read volume that falls on the database
qps_read = 3858                     # reads/s (from module 2)

reads_per_day = qps_read * 86_400
reads_per_year = reads_per_day * 365

print(f"reads/s    = {qps_read:,}")
print(f"reads/day  = {reads_per_day:,}")
print(f"reads/year = {reads_per_year:,}")

What to expect. With python db_load.py:

reads/s    = 3,858
reads/day  = 333,333,333
reads/year = 121,666,666,666

333 million reads a day, more than 121 billion a year, all on a single database. Here's the trap that many people struggle to see: it's not that the database can't serve a read —it serves it in 50 ms without a problem—; it's that it can't serve 4,000 at once every second, forever, without saturating. When it saturates, it doesn't fail cleanly: first it gets slow (the well's line), the queries that took 50 ms start taking 200, 500, a thousand, and the whole system degrades. The cache attacks this at the root: if it catches 90% of the reads, the database goes from seeing 4,000/s to seeing 386/s —you computed it in lesson 1 and you'll formalize it in lesson 5—, and 386/s is a number a single instance breathes easy with. The cache doesn't make the database faster; it makes it get bothered much less.

Pressure number two: the latency gap

Now the other pressure, the one felt on each individual read. When the database resolves a short_code, it doesn't do it by magic: it follows a chain of steps, and each step costs time. It has to open (or take from a pool) a connection, send the query over the network to the database server, have it search its index, have it —if the data isn't already in its own memory— read it from disk, and send the response back over the network. Each link adds up. The most expensive is usually the disk and the network: reading from an SSD and crossing the data center's network are operations of milliseconds, and added up they give on the order of ~50 ms for a database read that touches disk.

Let's compare that with reading the same data from a cache in RAM. RAM has no disk to spin and no —if the cache is on the same network— long trip: a get to Redis resolves in the order of ~1 ms, counting the round trip over the local network. This is the famous latency hierarchy, and it's worth having the orders of magnitude in your head because they're the foundation of why caches exist:

Operation	Order of magnitude	Comparison
Read from RAM (local cache on the network)	~1 ms	the reference
Read from an SSD	~0.1–1 ms on its own	but the complete query to the database adds more
Complete read to the database (network + index + disk)	~50 ms	~50× slower than the cache
Network trip between continents	~100–150 ms	(context, we don't use it here)

The exact numbers vary with the hardware, the load, and the distance, and that's why in this guide we fix L_cache = 1 ms and L_db = 50 ms as round and realistic working values: they're not "the universal truth", they're an honest model to do calculations with. What does not vary is the order: the cache is approximately fifty times faster than going to the database. And that 50× ratio is what turns the hit ratio into gold. Each read the cache catches is one that cost 1 ms instead of 50. Multiply that by 333 million reads a day and you see why the cache isn't a minor optimization: it's the difference between a system that responds instantly and one that drags its feet.

Worked example: how much total time is saved in a day

Let's bring the latency gap down to a tangible number. How much waiting time, summed over all the reads of the day, does the cache save? Let's compare the world without a cache (everything goes to the database, 50 ms each) against the world with a cache at 90% hit ratio.

# time_saved.py — total waiting time with and without a cache, in a day
reads_per_day = 333_333_333
L_cache = 1     # ms per hit
L_db = 50       # ms per miss

# Without cache: everything goes to the database
total_no_cache_ms = reads_per_day * L_db

# With cache at 90%: 90% are hits (1 ms), 10% are misses (50 ms)
h = 0.90
hits = reads_per_day * h
misses = reads_per_day * (1 - h)
total_with_cache_ms = hits * L_cache + misses * L_db

print(f"without cache : {total_no_cache_ms/1000/3600:,.0f} wait-hours/day")
print(f"with cache    : {total_with_cache_ms/1000/3600:,.0f} wait-hours/day")
print(f"savings       : {(1 - total_with_cache_ms/total_no_cache_ms):.0%}")

This sums the waiting time of all the day's reads, to see it at scale. Without a cache, each of the 333 million reads costs 50 ms. With a cache at 90%, 300 million cost 1 ms and only 33 million cost 50 ms.

What to expect. With python time_saved.py:

without cache : 4,630 wait-hours/day
with cache    : 546 wait-hours/day
savings       : 88%

The cache at 90% cuts the accumulated waiting time by 88%: from 4,630 wait-hours a day to 546. It's not that the system literally has 4,630 clock hours —the reads happen in parallel—, it's a way to see the magnitude of the aggregate relief. And notice that the savings (88%) is a bit less than the hit ratio (90%), because the remaining misses are the expensive ones: that's exactly what the formula of lesson 5 formalizes. What matters: the 50× gap between RAM and database, multiplied by Enlace's brutal volume, produces a saving that can't be ignored.

What a cache is, precisely

With the two pressures clear, let's define the tool without metaphors. A cache is a key-value store that lives in RAM, whose job is to keep copies of the most-requested data to serve them without going to the slow source. Three properties define it, and each one explains part of why it works:

• It lives in RAM. That's why it's fast (~1 ms) and why it's volatile: if the process restarts, the RAM is wiped and the cache starts empty. This isn't a serious flaw, because the source of truth —the database— is still intact and the cache repopulates on the fly.
• It's key-value. It works like a Python dictionary: you give it a key and it gives you a value, no complex queries, no joins, no scans. That simplicity is part of its speed. In Enlace, the key is the short_code ("aX9kR2q") and the value is the long_url ("https://example.com/very-long-article...").
• It's partial and disposable. It doesn't store everything: it stores what fits and what's hot (the working set of lesson 6). And if it's wiped, nothing serious happens: it's a copy, not the original.

In Enlace, the cache fits like a glove because of the shape of its data. The record read on the hot path is tiny —a 7-byte short_code pointing to a ~500-byte long_url— and, above all, it almost never changes: when someone shortens a URL, the short_code → long_url mapping stays fixed practically forever (a short link isn't "reassigned" to another destination). Small and immutable data is a cache's dream: a lot fits in little RAM, and you almost never have to worry about the copy becoming stale (the problem of lesson 7, which in Enlace is benign precisely because of this). Compare it with caching, say, a bank account balance —tiny too, but it changes on every transaction—: there the cache gives invalidation headaches that in Enlace hardly exist.

Where the cache lives: between the app and the database

Physically, the cache is placed between the application server and the database. The app server, when it needs to resolve a short_code, no longer talks directly to the database: it first asks the cache.

flowchart LR
    Client([Visitor]) -->|GET /aX9kR2q| App[App server]
    App -->|1. look first| Cache[(cache · Redis · RAM)]
    Cache -.->|2. if missing, then| DB[(db · source of truth · disk)]
    DB -.->|3. populate| Cache

The concrete store used in practice for this is almost always Redis (or its cousin Memcached). Redis is an in-memory data-structure server: a giant key-value dictionary that lives in RAM, with GET key and SET key value commands that respond in submilliseconds, and with native support for expiration (TTL) and the eviction policies we'll see in lessons 4 and 7. When in this guide we write cache.get(short_code), imagine underneath a GET aX9kR2q traveling to a Redis on the same network that responds almost instantly. You don't need to master Redis for this module —we treat it as "the barrel in the kitchen"—, but it's useful to know that our code's abstract cache is, in the real world, that concrete piece.

Common mistakes

Believing the database "handles" 4,000 reads/s just because each one is fast. What happens: someone measures an isolated query, sees it takes 50 ms, and concludes "the database is fast, I don't need a cache". Then, under real load, that same query starts taking 500 ms because there are thousands in line. Why it happens: the latency of one query is confused with the capacity to serve many at once. The well draws water fast for one; with a thousand in line, the wait is hours. How to detect it: measure under load, not isolated. If the latency rises when the queries per second rise, you're near the limit. How to fix it: reduce the number of queries that reach the database —that's exactly pressure number one the cache resolves by catching 90%—. Dropping from 4,000/s to 386/s is what keeps each query at its 50 ms instead of shooting up.

Putting the source of truth in the cache "so it's fast". What happens: someone, seduced by the ~1 ms of RAM, decides to store the data only in the cache and not in the database. Everything goes great until the first Redis restart, when the RAM is wiped and the data disappears for good. Why it happens: RAM's speed tempts you to forget it's volatile. How to detect it: ask yourself "if the cache turns off now, do I lose something I can't reconstruct?". If yes, you have the architecture inverted. How to fix it: the database persists on disk and is the source of truth; the cache is a copy in RAM of the hot stuff. The barrel can be spilled and nothing happens because the well is still full; never the other way around.

Assuming L_db = 50 ms and L_cache = 1 ms are universal constants. What happens: someone takes these numbers as physical laws and is surprised when in their system the database responds in 5 ms (because the data was already in its memory) or the cache in 3 ms (because it's in another data center). Why it happens: a working model is confused with a measurement of the actual system. How to detect it: if you never measured the real latency of your cache and your database, you're using borrowed numbers. How to fix it: use 1 ms and 50 ms to learn the shape of the formula and make back-of-the-envelope estimates —they're realistic orders of magnitude—, but when you design a real system, measure yours and put those into the calculation. What doesn't change between systems is that the cache is much faster than the database; the exact ratio (50×, 20×, 10×) is yours to measure.

Exercises

Exercise 1 — The database with and without a cache. With qps_read = 3858 reads/s, compute how many reads per second actually reach the database in three scenarios: (a) without a cache, (b) with a cache at 80% hit ratio, (c) with a cache at 95%. The formula for those that reach the database is (1 − h) × qps_read.

See solution

• (a) Without cache (h = 0): (1 − 0) × 3858 = 3,858 reads/s to the database. The whole volume falls on it.
• (b) At 80% (h = 0.8): (1 − 0.8) × 3858 = 0.2 × 3858 = 772 reads/s. The cache already took 80% of the load off it.
• (c) At 95% (h = 0.95): (1 − 0.95) × 3858 = 0.05 × 3858 = 193 reads/s.

The jump is the message: going from no-cache to 80% divides the database's load by five (from 3,858 to 772); reaching 95% divides it by twenty (to 193). The database that was drowning with 3,858/s breathes with 193/s. And notice the non-linearity that lesson 5 formalizes: raising the hit ratio from 80% to 95% —only fifteen points— halves (and more) the load on the database again, because what matters is the fraction of misses, not of hits.

Exercise 2 — The 50× gap, in your head. Without running code, answer: if a cache hit costs 1 ms and a miss (going to the database) costs 50 ms, how many cache hits "fit" in the time of a single miss? And if a user does 100 reads in a row, how long does it take in total if all are hits, versus if all are misses?

See solution

In the time of a single miss (50 ms), 50 hits of 1 ms each fit. That's the 50× gap that makes the cache valuable: the time the database spends resolving one read, the cache uses to resolve fifty.

For 100 reads in a row:

• All hits: 100 × 1 ms = 100 ms (a tenth of a second).
• All misses: 100 × 50 ms = 5,000 ms = 5 seconds.

Five seconds against a tenth. The same amount of work, fifty times slower depending on where the data comes from. This mental calculation —"a miss is worth like fifty hits"— is the intuition that makes the hit ratio matter so much: each point of hit ratio you gain turns 50 ms reads into 1 ms reads. That's why lesson 5 obsesses over that number.

Exercise 3 — Why Enlace and not a bank? Enlace's data (short_code → long_url) is small and immutable: once created, it almost never changes. A bank account balance is also small, but it changes on every transaction. Both are read-heavy. Explain in a couple of sentences why caching Enlace's link is easy and caching the bank balance is dangerous, and what problem appears in the second case that in the first hardly exists.

See solution

Caching Enlace's link is easy because the short_code → long_url mapping doesn't change after being created: the copy the cache stores is still correct today, tomorrow, and a year from now. You never serve stale data, because the data doesn't age. This is exactly the ideal scenario of lesson 7.

Caching the bank balance is dangerous because the balance changes constantly: as soon as money comes in or out, the copy in the cache becomes stale. If an ATM reads a cached balance from five minutes ago, it can authorize a withdrawal that it no longer should —money that's no longer there—. The problem that appears is invalidation: every time the database changes, you have to delete or update the copy in the cache instantly, and doing it well is notoriously hard (lesson 7 develops it).

The moral: two systems can be equally read-heavy and still be very different candidates for a cache, according to how often the data changes. Enlace is a happy case because its data is at once very requested and immutable. When you decide to cache anything, don't only ask "is it read a lot?", also ask "how often does it change?" —how much invalidation pain you're buying depends on that second answer.

Summary and next step

In this lesson you translated the 100:1 into the two pressures that make the cache mandatory. The volume one: ~333 million reads a day on a single database, a flow no single instance serves comfortably —and that the cache reduces from 4,000/s to ~386/s by catching 90%—. The latency one: each read to the database costs ~50 ms (network + index + disk) against ~1 ms from RAM, a gap of ~50× that, multiplied by Enlace's volume, saves 88% of the accumulated waiting time. The town well and the barrel in the kitchen resolve both: fewer trips to the well, and water instantly.

You defined the cache precisely —a key-value store in RAM, fast, volatile, partial, and disposable— and you saw why Enlace is its perfect case: its data is small, very requested, and immutable, the ideal combination. And you placed where it lives: between the app and the database, embodied almost always in Redis, with cache.get(short_code) in front of the database.

Before moving on you should be able to: name the two pressures a cache resolves (volume and latency) and give a number for each; explain why L_db ≈ 50 ms and L_cache ≈ 1 ms are a model, not a law; and say which two properties of Enlace's data make it an ideal candidate (small and immutable).

What comes next is to stop talking and start building. In lesson 3 you'll implement the concrete pattern with which the app "looks at the barrel before going to the well": cache-aside, with its miss → database → populate flow. You'll rewrite resolve(short_code) so it queries the cache first, and you'll see the exact diagram of what happens on a hit and what happens on a miss.

Resources

• Latency Numbers Every Programmer Should Know — the classic latency table (RAM, SSD, network, disk) from which the 50× gap between cache and database comes. Look at them as orders of magnitude, not exact figures: the message is that RAM is much faster than going to fetch the data outside.
• Redis — Introduction to Redis — the official documentation of the in-memory store that acts as cache in Enlace. It explains the key-value model (GET/SET) and why it lives in RAM, which are the two properties that give it its speed.
• Designing Data-Intensive Applications, Kleppmann — "Describing Performance" (Ch. 1) — the section where Kleppmann explains why latency is measured under load and not isolated, exactly the common mistake of believing that "a fast query" means "handles a lot of load". The foundation of this lesson's pressure number one.