8. Project: size Enlace's cache

Description

The moment has come to bring the seven lessons together into a single deliverable. In this project you're going to do what a real engineer does when it's time to put a cache into a read-heavy system: size it with numbers. Not "let's add Redis and see what happens", but a defensible design that answers four concrete questions and backs them with arithmetic that runs:

1. What target hit ratio do I choose, and why that one?
2. How much memory does the cache need? (working set × bytes per entry)
3. What average latency results? (the formula L = h·L_cache + (1−h)·L_db)
4. What eviction and TTL policies do I use, and how much load do I take off the database?

The deliverable is a sizing sheet —a table with those numbers— plus a justified policy and a note on what happens the day the hit ratio collapses. You're going to build a small calculator in Python that produces the sheet from your decisions, run it, and defend each choice. By the end you'll have a template that serves to size the cache of any system, not just Enlace's.

Connection to the module: this is the capstone. It uses the pressure of lesson 2, the pattern of lesson 3, the eviction of lesson 4, the formula of lesson 5, the working set of lesson 6, and the TTL of lesson 7 —all at once— to produce a real artifact. It's also the bridge to module 5: when you finish sizing the cache, you'll see what load still reaches the database (386 reads/s plus the ~40 writes/s), and that residual load is exactly the problem module 5 attacks with replicas and sharding.

The project brief

You're the engineer in charge of Enlace's cache layer. The team gives you the anchor numbers —the same ones from the whole guide— and asks you for a sizing proposal. These are the input data, fixed:

Input data	Value	Where it comes from
New URLs per month	100,000,000	canonical requirement
Reads per second (qps_read)	~3,858 (~4,000)	100:1 ratio (module 2)
Writes per second (qps_write)	~39 (~40)	module 2
Size of a cache entry	~500 bytes	short_code + long_url (~500 B) + overhead
Latency of a hit (L_cache)	1 ms	module's model (RAM)
Latency of a miss (L_db)	50 ms	module's model (database)
Total database size	~6 TB	5 years × ~1 KB/record (module 2)

And these are the decisions you make (the design levers): the target hit ratio, the hot fraction of the working set, the eviction policy, the TTL policy, and how much RAM to provision with margin. The project consists of choosing each one, justifying it, and computing the consequences.

The sizing steps

Before seeing the reference solution, here's the process you'll follow. Do it yourself first; the solution is after so you can compare.

Step 1 — Choose the target hit ratio

Look at the table from lesson 5 and decide. Don't aim for 100% (you saw in lesson 6 why it's an expensive mirage: it requires caching the whole cold tail). Don't settle for 0.50 (half the reads would still hit the database). The sweet spot for Enlace is at 0.90–0.95: good hit ratio, modest memory, single-digit latency. Choose one and have its justification at hand: what latency it gives and how much load it takes off the database.

Step 2 — Compute the working-set memory

Apply lesson 6: working_set = new_links_per_day × hot_fraction, and memory = working_set × bytes_per_entry. With the 80/20 fraction (0.20) and ~500 bytes per entry, you get the cache size. Also decide how much RAM to provision: always a bit more than the exact working set, to leave margin for spikes and Redis's overhead (a common rule is 1.5–2× the working set).

Step 3 — Compute the resulting latency

Apply the formula from lesson 5 with your target hit ratio: L = h·L_cache + (1−h)·L_db. This is the number you promise the rest of the system: "with this cache, a read takes on average L ms".

Step 4 — Choose eviction and TTL, and compute the database relief

From lesson 4, choose the maxmemory-policy (for a pure cache like Enlace's, you already know which). From lesson 7, choose the TTL policy (remembering that Enlace's data is almost immutable, which allows you a luxury). And compute, with (1 − h) × qps_read, how many reads/s still reach the database —the number module 5 will inherit—.

Reference solution

Here's a complete and defensible sizing. It's not the only correct answer —another target hit ratio or another TTL window are also defensible—, but it's a solid proposal with each number justified. First the calculator that produces it:

# enlace_cache_sizing.py — the sizing sheet, run
# --- Fixed inputs (anchor numbers) ---
NEW_URLS_PER_MONTH = 100_000_000
qps_read = 3858
qps_write = 39

# --- Design decisions ---
target_hit_ratio = 0.90         # step 1
hot_fraction = 0.20             # 80/20 rule
entry_bytes = 500               # short_code + long_url + overhead
L_cache = 1.0                   # ms
L_db = 50.0                     # ms

# Step 2: working set and memory
new_links_per_day = NEW_URLS_PER_MONTH / 30
working_set = new_links_per_day * hot_fraction
mem_bytes = working_set * entry_bytes

# Step 3: resulting latency
L = target_hit_ratio * L_cache + (1 - target_hit_ratio) * L_db

# Step 4: database relief
to_db = (1 - target_hit_ratio) * qps_read

print(f"target hit ratio   : {target_hit_ratio:.0%}")
print(f"working set        : {working_set:,.0f} entries")
print(f"memory (at {entry_bytes}B) : {mem_bytes/1e6:,.0f} MB ({mem_bytes/1e9:.2f} GB)")
print(f"RAM to provision   : ~1 GB (2x the working set, roomy)")
print(f"average latency    : {L:.2f} ms  (no cache: {L_db:.0f} ms, {L_db/L:.1f}x faster)")
print(f"reads to the DB    : {to_db:,.0f}/s  (no cache: {qps_read:,}/s)")

What to expect. With python enlace_cache_sizing.py:

target hit ratio   : 90%
working set        : 666,667 entries
memory (at 500B) : 333 MB (0.33 GB)
RAM to provision   : ~1 GB (2x the working set, roomy)
average latency    : 5.90 ms  (no cache: 50 ms, 8.5x faster)
reads to the DB    : 386/s  (no cache: 3,858/s)

The sizing sheet

Those numbers, presented as the deliverable you'd show the team:

Metric	Value	Justification
Target hit ratio	90%	Sweet spot: single-digit latency with modest memory (lesson 5)
Working set	666,667 entries	20% of the ~3.33 M new links/day (80/20 rule, lesson 6)
Working-set memory	~333 MB	666,667 × 500 bytes/entry
RAM to provision	~1 GB	~2× the working set: margin for spikes and Redis overhead
Resulting average latency	5.90 ms	0.9·1 + 0.1·50 (lesson 5); 8.5× faster than without a cache
Reads/s to the database	386/s	(1 − 0.9) × 3,858; the cache absorbs 90%
Eviction policy	allkeys-lru	Pure cache, traffic with changing fashions (lesson 4)
TTL policy	Long (24 h) + jitter	Almost immutable data: long TTL for hit ratio, jitter against stampede (lesson 7)

How each decision is defended

• Hit ratio 0.90. Gives 5.90 ms of average latency (8.5× better than the 50 ms without a cache) and takes 90% of the load off the database. You could aim for 0.95 (3.45 ms, 193 reads/s to the database) in exchange for more RAM to cache more of the tail; 0.90 is the point where the memory is minimal (~333 MB) and the latency is already excellent. Rising to 0.95 is worth it if the database suffers with 386/s; if not, 0.90 is enough.
• Memory ~333 MB, provision ~1 GB. The exact working set fits in 333 MB, but ~1 GB is provisioned (about 2× the working set plus a cushion) to absorb high-traffic days, Redis's metadata overhead, and some of the cold tail that raises the hit ratio for free. A Redis of 1–2 GB is cheap and common: there's no reason to skimp here.
• allkeys-lru. Enlace's cache is a pure cache —all its content is reconstructed from the database—, so all the keys are legitimate eviction candidates, and LRU over all of them exploits the temporal locality of the viral/recent links (lesson 4). noeviction would reject writes when it fills up (bad); allkeys-lfu would get polluted with old glories given Enlace's changing-fashion traffic.
• Long TTL (24 h) with jitter. Since the short_code → long_url mapping is almost immutable (lesson 7), a long TTL almost never serves stale data and maximizes the hit ratio. The jitter (24 h ± 10%) avoids many entries expiring at once and causing a stampede (lesson 5). Explicit invalidation is only needed when deleting a link (expiration): delete_link deletes the cache entry, database first.

What happens the day the hit ratio collapses

A good sizing doesn't only describe the good day; it anticipates the bad one. Include this section in your delivery, because it's what distinguishes a naive proposal from a robust one. The scenario: Redis restarts and the cache starts empty, or a deployment empties it. The hit ratio drops to 0 for a few seconds or minutes, and:

• The average latency jumps from 5.90 ms to 50 ms (everything is misses).
• The database goes from 386 reads/s to 3,858 reads/s suddenly —ten times its usual load—, and if it can't handle that peak, it saturates and the real latency exceeds 50 ms.

The mitigations that go in the proposal (all seen in the module):

1. Warm up the cache before sending it traffic after a restart (populate it in a staggered way with the hottest known links), so as not to expose the database to the cold avalanche.
2. Jitter in the TTLs (already in the policy), so the expirations don't synchronize and produce periodic mini-stampedes.
3. Size the database to survive the peak, not just the steady state —or, better, have read replicas that absorb the blow—. And this is where this project hands the baton to module 5: the cache protects the database 99% of the time, but the 1% it fails, the database has to handle it alone. Replicas and sharding are the net beneath the net.

Common mistakes

Sizing by the total database size, not by the working set. What happens: someone sees the 6 TB and requests a huge Redis (or discards the cache because "it doesn't all fit"). The proposal comes out very expensive or nonexistent. Why it happens: "all the data" is confused with "the hot data" (lesson 6). How to detect it: if your proposed memory comes close to the 6 TB, you're not using the 80/20 rule. How to fix it: size by the working set (~333 MB), not by the total. The cache catches the concentrated traffic, which is a tiny fraction of the data.

Promising a latency without saying at what hit ratio. What happens: someone writes "the cache gives 5.9 ms" without clarifying that it assumes a sustained hit ratio of 0.90. When the real hit ratio turns out to be 0.80 (because the working set was larger than estimated), the real latency is 10.8 ms and the promise breaks. Why it happens: the nice number is reported without its condition. How to detect it: if your promised latency isn't tied to a concrete hit ratio and RAM, it's a loose figure. How to fix it: every latency promise carries its hit ratio and its memory: "5.9 ms at hit ratio 0.90, with ~1 GB of RAM". And verify the real hit ratio in production, not just the estimated one.

Forgetting the plan for the bad day. What happens: someone sizes the steady state perfectly and says nothing about what happens if the cache empties. On restart day, the database receives the avalanche, saturates, and the system goes down —exactly when the cache was supposed to protect it—. Why it happens: it's easy to design for the happy case and forget the mass miss (lesson 5). How to detect it: if your proposal doesn't mention what happens at hit ratio 0, it's incomplete. How to fix it: always include the "what happens if the hit ratio collapses" section with its mitigations (warm-up, jitter, database/replicas that handle the peak). A sizing without a plan for the bad day is half a proposal.

Exercises

Exercise 1 — Resize for hit ratio 0.95. The team asks to lower the average latency below 4 ms. Recompute the sizing sheet aiming for hit ratio 0.95 instead of 0.90: (a) the resulting latency, (b) the reads/s that reach the database, and (c) reason whether the memory has to rise and why.

See solution

• (a) Latency: L = 0.95×1 + 0.05×50 = 0.95 + 2.50 = **3.45 ms**. Below the requested 4 ms: it meets it.
• (b) Reads to the database: (1 − 0.95) × 3,858 = 0.05 × 3,858 = **193/s** (half of the 386/s from the 0.90 plan). The database breathes even more.
• (c) The memory rises, yes. Raising the hit ratio from 0.90 to 0.95 requires caching beyond the hottest 20% —you have to catch part of the colder tail, which is what produces the additional 5% of hits—. In approximate numbers, maybe going from 20% to 30–40% of the working set, that is from ~333 MB to ~500–670 MB. It's still little RAM (it fits comfortably in the 1 GB Redis already provisioned, or you go up to 2 GB). The tradeoff from lesson 5 in action: the last points of hit ratio cost more memory per point, because you cache ever-colder data —but for Enlace the absolute cost is still trivial—.

Conclusion: 0.95 is perfectly achievable and gives 3.45 ms; the only cost is a bit more RAM (which is cheap). If the team wants sub-4-ms latency and less database load, it's worth it; if 5.9 ms was already enough, 0.90 saves a bit of memory. Both are defended with numbers.

Exercise 2 — The bigger entry. Suppose Enlace decides to store in each cache entry not just the long_url (~500 B) but also metadata (page title, created_at, clicks counter), taking the entry to ~1,500 bytes. (a) How much memory does the 666,667-entry working set need now? (b) Does it still fit comfortably in RAM? (c) What design decision would you make?

See solution

• (a) Memory: 666,667 × 1,500 = 1,000,000,500 B ≈ **1,000 MB ≈ 1.0 GB**. By tripling the entry size (from 500 to 1,500 B), the memory triples (from 333 MB to 1 GB), as expected: memory is linear in the bytes per entry.
• (b) Yes, it still fits comfortably. 1 GB is perfectly manageable in a modern Redis (which can have 8, 16, 32 GB). There's no capacity drama; you'd just have to provision ~2 GB instead of ~1 GB to keep the same margin.
• (c) The key decision: do you really need the metadata on the hot path? Enlace's hot path is the redirect (resolve), which only needs the long_url. The title, created_at, and clicks are for analytics or the management page, which are queried much less. A good decision is to cache only what the hot path needs (the long_url, ~500 B) and leave the metadata in the database, queried on demand by the cold paths. That way you keep the entry small, more working set fits in the same RAM, and the redirect's hit ratio doesn't get diluted. Putting fat metadata in the hot cache is a case of "caching more than the fast path requests", which wastes RAM (an echo of lesson 3). Rule: the hot cache stores exactly what the hot path reads, not a byte more.

Exercise 3 — Defend your design against three objections. A colleague questions your proposal (hit ratio 0.90, ~1 GB, allkeys-lru, TTL 24 h + jitter). Answer each objection in a couple of sentences, with numbers. (a) "Why don't we cache the whole 6 TB to have 100% hit ratio?" (b) "Why a 24-hour TTL? Don't we serve old data?" (c) "Why allkeys-lru and not noeviction, which doesn't delete anything?"

See solution

• (a) Not caching the 6 TB: because it would be very expensive and buy very little. Going from hit ratio 0.90 (with ~333 MB) to 1.00 (with 6 TB) lowers the average latency from 5.90 ms to 1.00 ms —an improvement of ~5 ms— in exchange for multiplying the RAM by ~18,000. The 80/20 rule says the hot working set (~333 MB) catches 90% of the traffic; the rest is cold tail that generates almost no reads. Caching 100% is fighting diminishing returns (lessons 5 and 6).
• (b) 24-hour TTL: we don't serve old data because Enlace's data is almost immutable —the short_code → long_url mapping doesn't change after being created (lesson 7)—. With no changes, there's no staleness, so a long TTL is safe and maximizes the hit ratio. The only real invalidation is when deleting a link (expiration), which we handle with delete_link (database first, then delete the cache). The jitter avoids many entries expiring together.
• (c) allkeys-lru and not noeviction: because noeviction rejects new writes when the cache fills up, and the cache must be full (it's its normal state). With noeviction, on filling up we'd stop populating the cache with new links and the hit ratio would stall, plus we'd receive OOM errors. allkeys-lru evicts the least recently used to make room for the new —exactly what a pure cache needs—. All its content is reconstructible from the database, so evicting never loses anything (lesson 4).

The lesson of this exercise: a sizing isn't just a table of numbers, it's a set of decisions you know how to defend. Each number —the hit ratio, the RAM, the policy— answers a tradeoff you can explain with the module's arithmetic. That's what turns "we added Redis" into an engineering design.

Summary and next step

In this project you sized Enlace's complete cache, the module's capstone. You started from the anchor numbers and produced a defensible sizing sheet: target hit ratio 0.90, working set of 666,667 entries ≈ 333 MB, ~1 GB of RAM provisioned, resulting average latency of 5.90 ms (8.5× faster than without a cache), 386 reads/s residual to the database, eviction policy allkeys-lru, and long TTL with jitter —each decision justified with the arithmetic of the seven lessons—. And you added what separates a naive proposal from a robust one: the plan for the bad day, when the hit ratio collapses and the database receives the avalanche.

With this you close the caching module. You know why a read-heavy system demands it (100:1, the 50× gap), how to implement it (cache-aside, miss → database → populate), how to manage it when the RAM runs out (LRU), how to measure its effect (L = h·L_cache + (1−h)·L_db), how much it costs (the working set and the 80/20 rule), and how to keep it correct (TTL and invalidation). And you know how to size it, which is the skill that ties everything else together.

What comes next is the load the cache does not absorb. Your own sheet shows it: even with hit ratio 0.90, the database still receives 386 reads/s plus the ~40 writes/s, and on mass-miss day it receives thousands. A single database has a limit. In module 5 you'll see how to scale that database with replication (read replicas that spread the reads that do arrive) and sharding (splitting the data across nodes), with the verification of consistent hashing —how many keys are remapped when adding a node (should be ~K/N, not all of them)—. The cache was the first line of defense; the replicas and sharding are the second.

Resources

• Caching — System Design Primer — the comprehensive review of everything you sized here (hit ratio, patterns, eviction, invalidation) in the context of a complete system design. Ideal for reviewing the whole module at a glance before moving on to replicas.
• Redis — Memory optimization and maxmemory — the official guide for computing and adjusting a Redis's real memory (per-key overhead, maxmemory, policies), which is exactly what you provisioned in step 2. Useful for going from the napkin estimate to the real configuration.
• Designing Data-Intensive Applications, Kleppmann — Chapters 1 and 5 (performance and replication) — chapter 1 backs all the latency and load reasoning of this project; chapter 5 (replication) is the direct bridge to module 5, where you'll scale the database the cache can't protect alone.