Module 2: Caching Patterns and TTL
Cache Stampede + the Caching Pattern Lab Mini-project
Overview
We're one step away from closing module 2. So far you've mastered the 3 caching patterns and the TTL and invalidation strategies. But there's a problem that only shows up in production with real traffic: cache stampede. It's when a popular value expires (or gets invalidated) and a thousand simultaneous requests detect the miss at the same time, they all hit PostgreSQL, and they all regenerate the cache. The DB saturates, latencies spike, and in the worst case, there's a cascading failure.
This is the bug that shows up at 9am on Monday when every cache expires at the same time. You won't find it on localhost or in staging — only in production. But we can prevent it by design with two techniques: locking (only one regenerates, the rest wait or serve stale) and probabilistic early expiration (each request has a small probability of regenerating before the real TTL, spreading out the load).
After cache stampede, you close the module with the Caching Pattern Lab mini-project: a CLI tool that implements the 3 patterns over the same dataset, simulates realistic traffic, and reports hit rates, latency, and throughput. It's the consolidation of everything you've learned — you'll be able to compare the patterns with numbers, not intuition. When you finish this capsule you'll have a complete module: 5 capsules, ~4,500 lines, deep command of caching strategies. Ready to get into Rate Limiting in module 3.
The problem: cache stampede
Imagine this scenario:
- You have a popular endpoint:
GET /products(the homepage) - You cache it with a 5-minute TTL
- At 10:00am, 1,000 users open the app simultaneously
- 5 minutes go by. The cache expires at 10:05am
- At 10:05:00, 1,000 requests arrive at the same time. They all detect a cache miss. They all run a SELECT against PostgreSQL. They all regenerate the cache.
PostgreSQL receives 1,000 identical queries at the same instant. If the query is heavy (~100 ms), PostgreSQL saturates its connection pool. Latencies climb from 100 ms to 5,000 ms. Other endpoints start failing too. If your API has autoscaling, you scale up to more workers — which also hit PostgreSQL — making the situation worse.
This is cache stampede. It's also known as "thundering herd" or "dogpiling."
Visualization
Time Requests Cache hit? → DB?
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
10:00 100/sec ✓ (all) → 0
10:01 100/sec ✓ (all) → 0
10:02 100/sec ✓ (all) → 0
10:03 100/sec ✓ (all) → 0
10:04 100/sec ✓ (all) → 0
10:05 1000/sec ✗ ALL MISS → 1000! ◄── STAMPEDE
10:06 100/sec ✓ (all) → 0
During the second at 10:05, PostgreSQL receives 100x its normal load. If your DB has a pool of 100 connections, it saturates. Requests 101-1000 wait in a queue. Some time out. Some return a 503.
When cache stampede is real (not theoretical)
- Very popular data with a short TTL: the homepage, the default search, trending posts
- After a deploy: every cache resets
- After a Redis crash: an empty cache, everything is a miss
- Mass invalidation: you bump the version, and every read of the new version is a simultaneous miss
Solution 1: Locking (a single regenerator)
The idea: when there's a cache miss, only one worker regenerates the cache. The rest wait for it to finish, or they get the stale data.
Implementation with SETNX
import time
import json
import redis
r = redis.Redis(host='localhost', port=6379, decode_responses=True)
def db_get_products():
"""Simulates a slow query to PostgreSQL."""
time.sleep(0.1) # 100ms
return [{"id": i, "name": f"Product {i}"} for i in range(1, 51)]
def get_products_with_lock(timeout_ms=5000):
"""Cache-aside with a lock to prevent a stampede."""
cache_key = "cache:products"
lock_key = "cache:products:lock"
# 1. Try the cache
cached = r.get(cache_key)
if cached:
return json.loads(cached)
# 2. Cache miss → try to acquire the lock
lock_acquired = r.set(lock_key, "1", nx=True, ex=10)
if lock_acquired:
# I AM the regenerator. The others wait.
try:
products = db_get_products()
r.set(cache_key, json.dumps(products), ex=300)
return products
finally:
r.delete(lock_key)
else:
# ANOTHER worker is regenerating. Wait and retry.
start = time.time()
while (time.time() - start) * 1000 < timeout_ms:
time.sleep(0.05) # 50ms
cached = r.get(cache_key)
if cached:
return json.loads(cached)
# Timeout: regenerate yourself too (degraded mode)
return db_get_products()Visualizing the behavior with a lock
Time Worker Action DB query?
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
10:05.000 W1 Cache miss. Lock acquired. Regenerating... ✓ (1)
10:05.001 W2 Cache miss. Lock NOT acquired. Waiting...
10:05.002 W3 Cache miss. Lock NOT acquired. Waiting...
...
10:05.999 W1000 Cache miss. Lock NOT acquired. Waiting...
10:05.100 W1 Regeneration complete. SET cache. DEL lock.
10:05.105 W2 Retry loop: cache hit → return ✓
10:05.105 W3 Retry loop: cache hit → return ✓
...
1 DB query instead of 1,000. The worker holding the lock regenerates; the others wait ~100 ms and read from the regenerated cache.
A variant: serve stale during regeneration
Instead of blocking while you wait, serve the old data:
def get_products_swr_with_lock(stale_ttl=600):
"""SWR + a lock: nobody waits, everyone gets stale data during the regeneration."""
cache_key = "cache:products"
stale_key = "cache:products:stale"
lock_key = "cache:products:lock"
# 1. Try the fresh cache
cached = r.get(cache_key)
if cached:
return json.loads(cached)
# 2. The cache is stale: try the stale data + start the regeneration
stale = r.get(stale_key)
lock_acquired = r.set(lock_key, "1", nx=True, ex=10)
if lock_acquired:
# Start the regeneration in a thread (non-blocking)
import threading
threading.Thread(
target=regenerate_products,
args=(cache_key, stale_key, lock_key),
daemon=True
).start()
if stale:
return json.loads(stale) # serve stale
# There's nothing (a cold start). Now we do have to regenerate synchronously.
return regenerate_products(cache_key, stale_key, lock_key)
def regenerate_products(cache_key, stale_key, lock_key):
try:
products = db_get_products()
pipe = r.pipeline()
pipe.set(cache_key, json.dumps(products), ex=300)
pipe.set(stale_key, json.dumps(products), ex=3600) # a longer stale window
pipe.execute()
return products
finally:
r.delete(lock_key)With this version, nobody ever waits (except for the absolute cold start). Reads during the regeneration get the stale data immediately.
Solution 2: Probabilistic Early Expiration
Another elegant technique: each request has a small probability of regenerating the cache before the real TTL. You spread the load out over time instead of concentrating it at the moment of expiration.
The concept
Imagine TTL = 300 seconds:
- At 270 seconds (90% of the TTL), each request has a 5% probability of regenerating
- At 285 seconds (95% of the TTL), each request has a 20% probability
- At 295 seconds (98% of the TTL), an 80% probability
- At 300 seconds, it expires "naturally" — but by then there are 0 requests because someone regenerated it earlier
The regenerations spread out over a window of ~30 seconds before the TTL, eliminating the stampede spike.
Implementation (the XFetch algorithm)
import time
import math
import json
import random
import redis
r = redis.Redis(host='localhost', port=6379, decode_responses=True)
def db_get_products():
time.sleep(0.1)
return [{"id": i, "name": f"Product {i}"} for i in range(1, 51)]
def get_products_xfetch(beta=1.0):
"""
A cache with the XFetch algorithm to prevent a stampede.
beta: the aggressiveness factor. >1 = regenerates earlier (more aggressive).
<1 = waits closer to the real TTL.
The algorithm:
1. Read the cache + delta (how long the last regeneration took)
2. If current_time + (delta * beta * log(random)) >= expiry_time, regenerate
3. If not, return from the cache
"""
cache_key = "cache:products"
meta_key = "cache:products:meta"
cached = r.get(cache_key)
meta = r.hgetall(meta_key)
if cached and meta:
delta = float(meta.get("delta_seconds", 0.1))
expiry = float(meta.get("expiry_time", 0))
# The probabilistic check: do we regenerate early?
current_time = time.time()
random_uniform = random.random()
early_check = current_time - (delta * beta * math.log(random_uniform))
if early_check < expiry:
# Don't regenerate yet: return the cached value
return json.loads(cached)
# Regenerate (early or because of a miss)
start = time.time()
products = db_get_products()
delta = time.time() - start
pipe = r.pipeline()
pipe.set(cache_key, json.dumps(products), ex=300)
pipe.hset(meta_key, mapping={
"delta_seconds": delta,
"expiry_time": time.time() + 300,
})
pipe.expire(meta_key, 300)
pipe.execute()
return productsHow it works in practice
Without XFetch (a fixed TTL):
Time: 0 50 100 150 200 250 290 299 300 301
Load: 0 0 0 0 0 0 0 0 █ █ ◄── a spike
1000
misses
With XFetch (a rising probability):
Time: 0 50 100 150 200 250 280 290 295 299
Load: 0 0 0 0 0 1 3 8 20 45 ◄── spread out
The regenerations spread over ~30 seconds before the TTL, eliminating the spike.
When to use XFetch vs Locking
| Characteristic | Locking | XFetch |
|---|---|---|
| Complexity | Medium | High |
| Guarantees | Only 1 regenerates | Probabilistic (no guarantee) |
| Latency for the waiters | High (they wait) | Low (always from the cache) |
| Load distribution | Concentrated in 1 worker | Spread out over time |
| When to use it | Critical data, very heavy queries | Popular data, reducing spikes |
Recommendation: Locking is simpler and enough for 95% of cases. XFetch is for apps with very high traffic where reducing spikes matters.
Solution 3: Pre-populating with cron jobs
Sometimes the simplest solution is the best one: regenerate the cache periodically with a cron job, without waiting for it to expire naturally.
import time
import schedule # pip install schedule
def refresh_products_cache():
"""Refreshes the products cache every 4 minutes (before the 5 min TTL)."""
print("Refreshing the products cache...")
products = db_get_products()
r.set("cache:products", json.dumps(products), ex=300)
print(f" Cached {len(products)} products")
# Schedule it every 4 minutes
schedule.every(4).minutes.do(refresh_products_cache)
def run_scheduler():
refresh_products_cache() # the initial populate
while True:
schedule.run_pending()
time.sleep(1)Pros: simple, predictable, zero stampede risk. Cons: your app is coupled to the scheduler. If the job fails, the cache eventually expires and you're back to the stampede.
When to use it: data where the TTL can be predictable and the cost of regenerating is always low.
Mini-project: Caching Pattern Lab
It's time to consolidate module 2 in code. You're going to build Caching Pattern Lab: a tool that implements the 3 patterns over the same dataset and compares the results with real benchmarks.
Specifications
The Lab simulates a "products API" over a dataset of 1000 products. It implements the 3 patterns and compares them:
=== Caching Pattern Lab ===
Configuration:
- Dataset: 1000 products
- Workload: 5000 requests with an 80/20 distribution (80% of the traffic to 20% of the products)
- Simulated DB latency: 50ms
Results:
Pattern Total req Avg latency p50 p95 p99 Hit rate
─────────────────────────────────────────────────────────────────────────────
No cache 5000 50.2ms 50.1ms 53.5ms 55.2ms N/A
Cache-aside 5000 5.8ms 0.9ms 51.3ms 53.0ms 89.2%
Write-through 5000 5.6ms 0.9ms 50.5ms 52.8ms 95.0%
Write-behind* 5000 0.6ms 0.4ms 1.5ms 2.8ms N/A
* Write-behind only applies to writes; here we show write throughput
Project structure
caching-pattern-lab/
├── .venv/
├── lab.py
└── requirements.txt
Implementation: lab.py
"""
Caching Pattern Lab — an experimental comparison of the 3 patterns.
Mini-project for Module 2 of the Redis & Caching Strategies Guide.
"""
import time
import json
import random
import statistics
import threading
import redis
# ═══════════════════════════════════════════════════════════
# Setup
# ═══════════════════════════════════════════════════════════
r = redis.Redis(host='localhost', port=6379, decode_responses=True)
DATASET_SIZE = 1000
DB_LATENCY_SECONDS = 0.05 # 50ms
# We simulate PostgreSQL
fake_db = {
i: {
"id": i,
"name": f"Product {i}",
"price": 100 + (i * 7) % 500,
"stock": (i * 13) % 100,
}
for i in range(1, DATASET_SIZE + 1)
}
def db_get(product_id):
time.sleep(DB_LATENCY_SECONDS)
return fake_db.get(product_id)
def db_update(product_id, data):
time.sleep(DB_LATENCY_SECONDS)
if product_id in fake_db:
fake_db[product_id].update(data)
return fake_db[product_id]
return None
# ═══════════════════════════════════════════════════════════
# Pattern 1: No cache (the baseline)
# ═══════════════════════════════════════════════════════════
class NoCachePattern:
name = "No cache"
def get(self, product_id):
return db_get(product_id)
def update(self, product_id, data):
return db_update(product_id, data)
# ═══════════════════════════════════════════════════════════
# Pattern 2: Cache-aside
# ═══════════════════════════════════════════════════════════
class CacheAsidePattern:
name = "Cache-aside"
def __init__(self, ttl=300):
self.ttl = ttl
self.hits = 0
self.misses = 0
def get(self, product_id):
cache_key = f"ca:product:{product_id}"
cached = r.get(cache_key)
if cached:
self.hits += 1
return json.loads(cached)
self.misses += 1
product = db_get(product_id)
if product:
r.set(cache_key, json.dumps(product), ex=self.ttl)
return product
def update(self, product_id, data):
updated = db_update(product_id, data)
if updated:
r.delete(f"ca:product:{product_id}") # invalidate
return updated
# ═══════════════════════════════════════════════════════════
# Pattern 3: Write-through
# ═══════════════════════════════════════════════════════════
class WriteThroughPattern:
name = "Write-through"
def __init__(self, ttl=600):
self.ttl = ttl
self.hits = 0
self.misses = 0
def get(self, product_id):
cache_key = f"wt:product:{product_id}"
cached = r.get(cache_key)
if cached:
self.hits += 1
return json.loads(cached)
self.misses += 1
product = db_get(product_id)
if product:
r.set(cache_key, json.dumps(product), ex=self.ttl)
return product
def update(self, product_id, data):
updated = db_update(product_id, data)
if updated:
# Write-through: it updates the cache too
r.set(f"wt:product:{product_id}", json.dumps(updated), ex=self.ttl)
return updated
# ═══════════════════════════════════════════════════════════
# Pattern 4: Write-behind (for writes only)
# ═══════════════════════════════════════════════════════════
class WriteBehindPattern:
name = "Write-behind"
BUFFER_KEY = "wb:buffer"
FLUSH_INTERVAL = 5
def __init__(self):
self.flushed_count = 0
self._stop_event = threading.Event()
self._flusher_thread = None
def start_flusher(self):
self._flusher_thread = threading.Thread(target=self._run_flusher, daemon=True)
self._flusher_thread.start()
def stop_flusher(self):
self._stop_event.set()
def _run_flusher(self):
while not self._stop_event.is_set():
time.sleep(self.FLUSH_INTERVAL)
self.flush()
def update(self, product_id, data):
# To the buffer only
r.rpush(self.BUFFER_KEY, json.dumps({"id": product_id, "data": data}))
def flush(self):
pipe = r.pipeline()
pipe.lrange(self.BUFFER_KEY, 0, -1)
pipe.delete(self.BUFFER_KEY)
items_raw, _ = pipe.execute()
for raw in items_raw:
item = json.loads(raw)
db_update(item["id"], item["data"])
self.flushed_count += len(items_raw)
return len(items_raw)
# ═══════════════════════════════════════════════════════════
# Workload generation: an 80/20 distribution
# ═══════════════════════════════════════════════════════════
def generate_workload(num_requests, hot_pct=0.8):
"""
Generates N requests with an 80/20 distribution:
80% of the traffic goes to 20% of the products (the "popular" ones).
"""
workload = []
hot_products = list(range(1, int(DATASET_SIZE * 0.2) + 1))
all_products = list(range(1, DATASET_SIZE + 1))
for _ in range(num_requests):
if random.random() < hot_pct:
workload.append(random.choice(hot_products))
else:
workload.append(random.choice(all_products))
return workload
# ═══════════════════════════════════════════════════════════
# READ benchmark
# ═══════════════════════════════════════════════════════════
def benchmark_reads(pattern, workload):
print(f"\n--- Benchmarking READS: {pattern.name} ---")
# Clean up the previous cache
if pattern.name != "No cache":
for key_prefix in ["ca", "wt", "wb"]:
cursor = 0
while True:
cursor, keys = r.scan(cursor=cursor, match=f"{key_prefix}:*", count=100)
if keys:
r.delete(*keys)
if cursor == 0:
break
latencies = []
start_total = time.time()
for product_id in workload:
start = time.time()
pattern.get(product_id)
latencies.append((time.time() - start) * 1000)
total_elapsed = (time.time() - start_total) * 1000
return {
"name": pattern.name,
"total_ms": total_elapsed,
"avg_ms": statistics.mean(latencies),
"p50_ms": statistics.median(latencies),
"p95_ms": sorted(latencies)[int(len(latencies) * 0.95)],
"p99_ms": sorted(latencies)[int(len(latencies) * 0.99)],
"hits": getattr(pattern, "hits", None),
"misses": getattr(pattern, "misses", None),
}
# ═══════════════════════════════════════════════════════════
# WRITE benchmark
# ═══════════════════════════════════════════════════════════
def benchmark_writes(pattern, num_writes):
print(f"\n--- Benchmarking WRITES: {pattern.name} ---")
latencies = []
for i in range(num_writes):
product_id = (i % DATASET_SIZE) + 1
start = time.time()
pattern.update(product_id, {"price": random.randint(50, 500)})
latencies.append((time.time() - start) * 1000)
return {
"name": pattern.name,
"avg_ms": statistics.mean(latencies),
"p50_ms": statistics.median(latencies),
"p95_ms": sorted(latencies)[int(len(latencies) * 0.95)],
}
# ═══════════════════════════════════════════════════════════
# Reporting
# ═══════════════════════════════════════════════════════════
def report_reads(results):
print("\n" + "=" * 90)
print("READ PERFORMANCE")
print("=" * 90)
header = f"{'Pattern':<20} {'Avg':<10} {'p50':<10} {'p95':<10} {'p99':<10} {'Hit rate':<12}"
print(header)
print("-" * 90)
for r in results:
hit_rate = ""
if r["hits"] is not None and r["misses"] is not None:
total = r["hits"] + r["misses"]
hit_rate = f"{(r['hits'] / total * 100):.1f}%" if total > 0 else "0%"
else:
hit_rate = "N/A"
print(f"{r['name']:<20} {r['avg_ms']:<10.2f} {r['p50_ms']:<10.2f} {r['p95_ms']:<10.2f} {r['p99_ms']:<10.2f} {hit_rate:<12}")
def report_writes(results):
print("\n" + "=" * 60)
print("WRITE PERFORMANCE")
print("=" * 60)
header = f"{'Pattern':<20} {'Avg':<10} {'p50':<10} {'p95':<10}"
print(header)
print("-" * 60)
for r in results:
print(f"{r['name']:<20} {r['avg_ms']:<10.2f} {r['p50_ms']:<10.2f} {r['p95_ms']:<10.2f}")
# ═══════════════════════════════════════════════════════════
# Main
# ═══════════════════════════════════════════════════════════
def main():
print("=" * 60)
print(" CACHING PATTERN LAB")
print(" Module 2 mini-project")
print("=" * 60)
print(f"\nConfiguration:")
print(f" Dataset: {DATASET_SIZE} products")
print(f" DB latency: {DB_LATENCY_SECONDS * 1000:.0f}ms")
# Verify Redis
try:
r.ping()
except redis.ConnectionError:
print("\n❌ Redis isn't running. Start it with:")
print(" docker start redis-dev")
return
# Clean up Redis
r.flushdb()
# Generate the workload
NUM_REQUESTS = 1000
print(f" Workload: {NUM_REQUESTS} requests with an 80/20 distribution\n")
workload = generate_workload(NUM_REQUESTS)
# === READ benchmark ===
read_results = []
no_cache = NoCachePattern()
read_results.append(benchmark_reads(no_cache, workload))
cache_aside = CacheAsidePattern()
read_results.append(benchmark_reads(cache_aside, workload))
write_through = WriteThroughPattern()
read_results.append(benchmark_reads(write_through, workload))
report_reads(read_results)
# === WRITE benchmark ===
print("\n" + "=" * 60)
print("Benchmarking writes (100 operations)...")
print("=" * 60)
NUM_WRITES = 100
write_results = []
write_results.append(benchmark_writes(NoCachePattern(), NUM_WRITES))
write_results.append(benchmark_writes(CacheAsidePattern(), NUM_WRITES))
write_results.append(benchmark_writes(WriteThroughPattern(), NUM_WRITES))
# Write-behind
wb = WriteBehindPattern()
wb.start_flusher()
write_results.append(benchmark_writes(wb, NUM_WRITES))
wb.stop_flusher()
report_writes(write_results)
# === Analysis ===
print("\n" + "=" * 60)
print("ANALYSIS")
print("=" * 60)
no_cache_avg = read_results[0]["avg_ms"]
cache_aside_avg = read_results[1]["avg_ms"]
speedup = no_cache_avg / cache_aside_avg
print(f"\n Cache-aside speedup vs no-cache: {speedup:.1f}x")
cache_aside_hit_rate = read_results[1]["hits"] / (read_results[1]["hits"] + read_results[1]["misses"]) * 100
print(f" Cache-aside hit rate: {cache_aside_hit_rate:.1f}%")
write_through_hit_rate = read_results[2]["hits"] / (read_results[2]["hits"] + read_results[2]["misses"]) * 100
print(f" Write-through hit rate: {write_through_hit_rate:.1f}%")
no_cache_write = write_results[0]["avg_ms"]
write_behind_write = write_results[3]["avg_ms"]
write_speedup = no_cache_write / write_behind_write
print(f" Write-behind speedup vs no-cache: {write_speedup:.1f}x")
print("\n💡 Conclusions:")
print(f" • For reads, cache-aside cut latency by ~{int((1 - cache_aside_avg/no_cache_avg)*100)}%")
print(f" • Write-through has a higher hit rate ({write_through_hit_rate:.0f}% vs {cache_aside_hit_rate:.0f}%) — no cold cache problem")
print(f" • Write-behind is {int(write_speedup)}x faster on writes (from the client's perspective)")
if __name__ == "__main__":
main()requirements.txt
redis>=7.4
Running it
cd ~/projects/redis-guide/module-02-patterns
mkdir -p caching-pattern-lab && cd caching-pattern-lab
python -m venv .venv
source .venv/bin/activate
pip install redis
# Copy the lab.py code from above
python lab.pyExpected output:
============================================================
CACHING PATTERN LAB
Module 2 mini-project
============================================================
Configuration:
Dataset: 1000 products
DB latency: 50ms
Workload: 1000 requests with an 80/20 distribution
--- Benchmarking READS: No cache ---
--- Benchmarking READS: Cache-aside ---
--- Benchmarking READS: Write-through ---
==========================================================================================
READ PERFORMANCE
==========================================================================================
Pattern Avg p50 p95 p99 Hit rate
------------------------------------------------------------------------------------------
No cache 50.21 50.12 53.45 55.30 N/A
Cache-aside 5.42 0.92 51.20 52.85 89.4%
Write-through 4.85 0.91 50.55 52.65 90.2%
============================================================
Benchmarking writes (100 operations)...
============================================================
--- Benchmarking WRITES: No cache ---
--- Benchmarking WRITES: Cache-aside ---
--- Benchmarking WRITES: Write-through ---
--- Benchmarking WRITES: Write-behind ---
============================================================
WRITE PERFORMANCE
============================================================
Pattern Avg p50 p95
------------------------------------------------------------
No cache 50.31 50.21 52.45
Cache-aside 50.42 50.18 52.95
Write-through 51.23 50.92 53.45
Write-behind 0.34 0.28 0.55
============================================================
ANALYSIS
============================================================
Cache-aside speedup vs no-cache: 9.3x
Cache-aside hit rate: 89.4%
Write-through hit rate: 90.2%
Write-behind speedup vs no-cache: 148.0x
💡 Conclusions:
• For reads, cache-aside cut latency by ~89%
• Write-through has a higher hit rate (90% vs 89%) — no cold cache problem
• Write-behind is 148x faster on writes (from the client's perspective)
Analyzing the results
Reads:
- No cache: ~50 ms per read. PostgreSQL handles everything. If you have 1000 reqs/sec, that's 50,000 ms-seconds of query time.
- Cache-aside: ~5 ms on average (an 89% hit rate). A 9x speedup. The p99 is still 53 ms (the misses are slow).
- Write-through: ~5 ms on average (a 90% hit rate). A minimal difference from cache-aside because the workload has few writes.
Writes:
- No cache, cache-aside, write-through: all at 50 ms (the bottleneck is the DB).
- Write-behind: 0.3 ms (148x faster). The client never waits for the DB.
The decisions this reinforces:
- For reads, almost any cache is an enormous improvement (10x). Cache-aside is enough for 80% of cases.
- Write-through only pays off when there are a lot of reads on freshly written data (immediate consistency).
- Write-behind is transformational for massive writes — but remember the data loss risk.
The mini-project's success criteria
- The script runs without errors
- Reports are generated with real data
- The cache-aside hit rate is > 80%
- The cache-aside speedup vs no-cache is > 5x
- The write-behind speedup vs no-cache is > 50x
- You understand WHY each pattern got those numbers
If all 6 are ✅, you've completed module 2.
Troubleshooting
Problem 1: The stampede keeps happening even though you use locking
Cause: The lock's timeout is too short, so the waiting workers give up and regenerate too.
Solution: Raise the lock's timeout to the maximum reasonable time for the query (e.g., if the query takes 200 ms, a lock TTL of 5s leaves margin):
lock = r.set(lock_key, "1", nx=True, ex=5) # 5 secondsProblem 2: XFetch regenerates too early (an always-fresh cache)
Cause: beta is too high (>1.0), so it regenerates very early.
Solution: Start with beta=1.0 (the default). If you want to reduce the frequency, use beta=0.5. Measure it in production.
Problem 3: The cron job fails and nobody notices
Cause: The pre-populate strategy depends on a job that can fail silently.
Solution:
- Logs + alerts if the job fails
- Also keep a natural TTL as a fallback ("belt and suspenders")
- Hit rate monitoring: if it drops unexpectedly, that's a sign the job didn't run
Problem 4: Mini-project: Redis is down
Cause: You forgot to start Redis.
Solution: The script detects it and tells you:
❌ Redis isn't running. Start it with:
docker start redis-dev
Problem 5: The benchmark's hit rate is very low (<50%)
Cause: The workload doesn't generate enough key repetition.
Solution: Check that hot_pct=0.8 and that the dataset is small enough for the popular products to repeat across a workload of N=1000 requests.
Problem 6: Write-behind doesn't flush in the benchmark
Cause: The _flusher_thread isn't running, or the script finishes before the periodic flush.
Solution: Call wb.flush() explicitly before the stop, or lower FLUSH_INTERVAL to 1 second for testing.
Module 2 summary
You close module 2 with command of:
Cache stampede:
- It's real: it happens with popular data + short TTLs + concurrent traffic
- Locking with SETNX: only 1 worker regenerates, the rest wait
- Probabilistic Early Expiration (XFetch): it spreads regenerations out over time
- Pre-populating with cron jobs: simple if the TTL is predictable
The Caching Pattern Lab mini-project:
- An implementation of the 3 patterns over the same dataset
- A realistic workload (an 80/20 distribution)
- Metrics: avg, p50, p95, p99, hit rate
- A comparison of reads and writes
What you achieve by closing this module:
- ✅ Command of cache-aside, write-through, and write-behind with selection criteria
- ✅ TTL strategies (fixed, sliding, per type of data)
- ✅ The 3 invalidation strategies: manual, event-driven, version-based
- ✅ Stale-while-revalidate for minimal latency
- ✅ Cache stampede protection with locking and XFetch
- ✅ Real metrics: hit rate, latency, throughput
What's coming in module 3: Rate Limiting & Sessions. You'll take module 1's sorted sets (which you learned but haven't yet applied with their full purpose) and build professional rate limiting with a sliding window, a token bucket, and session storage that complements JWT.
Additional resources
- Cache Stampede on Wikipedia — The formal definition and the solutions
- Redis: Distributed Locks — The Redlock algorithm for robust locks
- XFetch Algorithm Paper — The original "Optimal Probabilistic Cache Stampede Prevention" paper
- Facebook: Caching at Scale — How Facebook solves cache problems at scale
- Pinterest: Cache Invalidation Strategies — A real case of cache invalidation in production
- System Design: Cache Stampede — A newsletter explaining the solutions
What's next?
You've closed Module 2: Caching Patterns & TTL. In Module 3: Rate Limiting & Session Storage you apply module 1's sorted sets to the most important API protection algorithm — sliding window rate limiting — and learn to use Redis as a session store complementing JWT.
Before moving on, make sure you:
- Have run the Caching Pattern Lab
- See hit rates >80% with cache-aside
- Get a >5x speedup on reads and >50x on writes with write-behind
- Understand WHEN to use each pattern (review capsule 03's decision matrix)
If all 4 are ✅, you're ready. On to module 3.