Module 3: Data Model and Short Code Generation

1. Module introduction: from the number to the model

Description

In the two previous modules Enlace was, above all, arithmetic. Module 1 gave you the method for approaching an open problem —clarify requirements, scope the problem, draw the "single box"— and module 2 put you to reproduce the back-of-the-envelope math: ~40 writes per second, ~4,000 reads per second, 6 billion records, and ~6 TB at five years. Those numbers were the diagnosis. This module is the first recipe: how a link is really stored and how its short code is manufactured.

It's a concrete leap. Up to now, when you said "Enlace stores the URL", that was a box with an arrow. From this lesson on, "storing the URL" is a table with five columns, each with its type and its size, one of them marked as the key. And "returning a short code" stops being magic: it's an algorithm you can write in twenty lines, run, and —this is the important part— break in the three or four classic ways if you don't understand what it does inside. By the end of the module you'll have the Link record designed, a defensible answer to "SQL or NoSQL for this?", and a short_code generator that runs and that you know why you chose.

Connection to the module: this lesson is the map, not the territory. Here you don't design the table yet (that's lesson 2) or run base62_encode (that's lesson 6). Here you install the three ideas that make the other seven make sense. First: a shortener's data model is surprisingly small —a ~1 KB record, five fields— and that smallness has consequences for which store to choose. Second: the access pattern rules —Enlace almost only does "give me the URL of this code", a lookup by key, and that pushes toward a key-value store or a unique index, not a loaded relational schema—. And third: generating the code is a decision with three candidates —hash, counter+base62, random+verification—, each with a trade-off you'll be able to recite from memory because you'll measure it, not memorize it.

A note on the module's tone, because it marks everything that follows: here you run code. It's a design guide, yes, but ID design has parts that are real code and real numbers. You'll run the base62 generator and see the round-trip give True. You'll check that 62⁷ is exactly 3,521,614,606,208 by having Python multiply it, not quoting it. And you'll measure the collision rate of a truncated hash with a simulation that runs on your machine, so that "collisions are a problem" stops being a slogan and becomes a 29.80% you saw come out in the terminal. Every number in this module came from running the code.

The warehouse with numbered lockers

Think of Enlace as an enormous warehouse of lockers, the kind you find at a train station. Every time someone brings a long URL to store, the system does two things: it puts the URL in a free locker and hands the client a ticket with a short number —the short_code—. Later, when anyone shows up with that ticket, the system reads the number, goes straight to the locker, and takes out what was inside. That's all a shortener does: store in a locker and return by ticket.

With that image in mind, the module's three big questions become concrete and even obvious.

The first: what's stored inside the locker? Not just the long URL. Also when it was stored, when the ticket expires (if it expires), and how many times someone has used it. That's the Link record, and you design it in lesson 2. The answer matters because that's where the "~1 KB per record" you already used in the estimation comes from: it wasn't a magic number, it's the sum of the sizes of those columns.

The second: how is the warehouse organized? Is it a filing cabinet with cross-referenced index cards where you can look up "all the URLs stored on a Tuesday by a premium user" (that's a relational database, with its joins and its rich queries), or is it a wall of lockers where the only thing you know how to do is "number → contents" (that's a key-value store)? The answer depends on what questions you'll ask the warehouse, and it turns out Enlace asks it almost only one: "give me the contents of this ticket". That's the SQL vs NoSQL discussion of lesson 3.

And the third, the juiciest: how is the ticket's number decided? Here there are three philosophies, and each has a problem the other doesn't. You could derive the number from the URL itself —run it through a grinder (a hash) and keep a few digits—, but then two different URLs can land on the same number (a collision), and two people would fight over the same locker. You could have a counter that goes 1, 2, 3, 4… and convert that number into a short ticket; there are never collisions, but anyone who receives the ticket 4c93 knows the previous one was 4c92 and can go snooping. Or you could throw a random number and check whether the locker is free; if it is, done, and if not, throw again. Those are the three strategies —hash, counter+base62, random+verification— and choosing between them with judgment is the module's central muscle.

The idea that holds up M3: a shortener is a warehouse of lockers. The record is small, the access is "ticket → contents", and all the interesting difficulty is in how the ticket's number is manufactured without collisions, without making it guessable, and without running out of tickets.

The Link record, in one sentence

Before lesson 2 develops it, it's worth having the complete record in your head, because it's the module's thread. These are its five fields, with the identifiers in English (like all the code in the guide) and the explanation in Spanish:

# The record Enlace stores for each shortened URL.
from dataclasses import dataclass
from datetime import datetime


@dataclass
class Link:
    short_code: str        # the "ticket": 7 chars base62, e.g. "aX9kR2q" — it's the KEY
    long_url: str          # the original URL, up to ~500 bytes
    created_at: datetime   # when it was shortened
    expires_at: datetime   # when it expires (or None if it doesn't expire)
    clicks: int            # how many times it has been resolved

Five fields. No relationship with another table, no joins. The short_code is what the outside world uses to request the record —it's the key— and everything else is the locker's contents. Notice from the start something we'll squeeze: Enlace's hot operation is resolve(short_code) -> long_url, that is, "given the short_code, give me back the long_url". It's a lookup by key, exact, a single row. There's no WHERE created_at BETWEEN..., no JOIN users, no GROUP BY. That poverty of access patterns —which sounds like a limitation— is actually the best design clue we have, and lesson 3 turns it into a storage decision.

The three strategies, at a glance

The module's main course is lessons 4, 5, and 6, where each generation strategy is developed and run. Here I only present them with their summary sentence and their one-line trade-off, so you know where we're headed:

StrategyHow it manufactures the codeIts characteristic problemLesson
Hash of the URLbase62(truncate(hash(long_url)))Collisions: two different URLs can give the same code. The shorter the code, the more frequent.4
Counter + base62a global counter 1,2,3,… passed through base62_encodeGuessable (sequential codes) and the global counter is a single coordination point5
Random + verificationthrows 7 random chars, checks that the locker is freeRetries when the space fills up; needs a check per write5

None is "the correct one" in the abstract. The correct one is the one that best pays Enlace's criteria, and we order those criteria in lesson 7. An honest preview: all three are viable for Enlace at this scale, and the real choice is usually a combination —for example, a counter started at a high offset so the codes don't begin at 1, or random + a unique index that does the verification for you—. The skill you build isn't memorizing an answer, it's defending yours with the trade-off in hand.

Worked example: the module's numeric anchor

Every module of the guide has a numeric anchor that's run, not quoted. M3's is this: do 7 base62 characters suffice for all the links Enlace will create in 5 years?

The calculation has two sides. The demand side you already computed in module 2: 100 million new URLs a month, times 12 months, times 5 years. The supply side is how many distinct codes fit in 7 characters of a 62-symbol alphabet (0-9, a-z, A-Z): that's 62⁷. We'll have Python compute both and compare them, instead of trusting anyone:

# Do 7 base62 chars suffice for 5 years of Enlace?
supply = 62 ** 7                      # distinct codes in 7 base62 chars
demand = 100_000_000 * 12 * 5         # new URLs in 5 years (100M/month)

print("supply  62^7   =", f"{supply:,}")
print("demand  5 years =", f"{demand:,}")
print("supply / demand =", supply / demand, "times")
print("fraction used   =", f"{demand / supply:.4%}")

What to expect. Running this with Python 3.14.0 gives, exactly:

supply  62^7   = 3,521,614,606,208
demand  5 years = 6,000,000,000
supply / demand = 586.9357677013334 times
fraction used   = 0.1704%

Read it slowly, because it's the permission the whole module gives us to choose any strategy with peace of mind. The supply —62⁷, three and a half trillion codes— is almost 587 times larger than the 5-year demand. Even after creating 6 billion links, we would have used 0.17% of the namespace. We're not scraping the bottom of the barrel; we're taking a glass of water from the ocean. This has an enormous practical consequence: since there's so much space to spare, the "throw a random code and verify" strategy almost never collides (you'll measure it in lesson 5), and "7 characters" isn't a tight number but a roomy one. If tomorrow the demand doubled, there would still be 293 times to spare.

Keep the number: 62⁷ = 3,521,614,606,208, and 5 years of Enlace use 0.17% of that. It's the module's checksum. Every time a lesson says "the short code suffices", this is the reason, and you'll be able to reproduce it.

The module's map

Eight lessons that go from understanding the record to choosing and building the generator. They're not in any order: first we fix what's stored and where, then we attack the three generation strategies one by one (with the math that distinguishes them), then we decide, and at the end we build.

LessonWhat it installsWhat you walk away with
1. From the number to the model (this one)The Link record in one sentence, the three strategies, the 62⁷ vs 6 billion anchorThe map and the why of each decision to come
2. The Link recordThe five fields with types and sizes, the short_code key, the PostgreSQL DDLDesign the table and justify where the "~1 KB per record" comes from
3. SQL vs NoSQL for EnlaceThe "lookup by key" access pattern, key-value vs relational, the decisionArgue which store fits and why, without dogma
4. Hash and collisionsHash+truncate, the collision simulation run, the birthday paradoxExplain why the pure hash isn't enough, with the real rate measured
5. Counter + base62 (and random)The monotonic counter to base62, its two costs, random+verificationCompare the two collision-free strategies and their price
6. base62 runThe divmod algorithm, the round-trip, 62⁷ reproducedImplement and run base62_encode/base62_decode with confidence
7. Choosing the strategyThe trade-off table against Enlace's criteriaMake and defend the design decision
8. Project: the generatorThe complete id_generator, with verification and expiration, runningDeliver a short_code generator that works and know why

Notice the shape of the arc. Lessons 2 and 3 fix the model and the store (what and where). Lessons 4, 5, and 6 are the algorithm (how the code is manufactured, run). Lesson 7 is the decision. And lesson 8 is the construction. When you finish, you won't have an opinion about shorteners; you'll have a generator that runs and a trade-off table you can defend in an interview or a design doc.

The boundary: what's taught here and what in M5

This module deliberately stops at the edge of a topic that's so big it has its own module: what happens when the store doesn't fit on a single machine. Here we design the record and the generator assuming a single database. Replication (read copies), sharding (splitting the data across machines), and consistent hashing are module 5. It's worth knowing the line from the start, so you don't expect from here something that belongs to M5:

TopicWhat's covered here (M3)Where it's developed
The record and its typesComplete: the five fields, sizes, the key— (it's this module's)
Choosing SQL vs NoSQLThe decision for Enlace's access pattern— (it's this module's)
Generating the short_codeThe three strategies, run and compared— (it's this module's)
Replicas and replication lagOnly mentioned that the global counter is a coordination pointModule 5
Sharding and consistent hashingOnly named that scaling the counter is splitting itModule 5
Caching the short_codeNothing: the hot read path is the next moduleModule 4

The mechanical rule to remember it: if the question is "how do I model the link and how do I generate its code?", it's this module. If it's "how do I make that model hold up across many machines?", it's module 5. And if it's "how do I make the 4,000 reads per second not touch the disk?", it's module 4, which starts right when we close this one. Keeping the boundary clear is what lets you learn the data model without drowning yet in distribution.

Common mistakes

Over-modeling the record "just in case". What happens: someone, used to apps with many entities, designs Enlace with a users table, a campaigns table, a domains table, three relationships, and eight indexes, before having a single user. Why it happens: "a serious system" is confused with "a complicated schema". How to detect it: if your data model has joins and Enlace, at its core, only needs "code → URL", you overshot. How to fix it: start with the minimal record that solves the requirements —the five fields of Link— and add complexity only when a requirement demands it. Module 2 gave you the numbers precisely so the design is tailored, not just-in-case.

Believing that "NoSQL is more modern / faster" and choosing it for that. What happens: someone chooses a key-value store (or, conversely, PostgreSQL) because "it's what's used now", without looking at the access pattern. Why it happens: the decision is made by fashion or habit, not by the problem. How to detect it: if you can't say in one sentence what query your system does 99% of the time, you're not in a position to choose the store. How to fix it: first name the access pattern (in Enlace: "exact lookup by short_code, 100:1 read ratio"), and then let that pattern choose the store. Lesson 3 does exactly that, and the answer is less dogmatic than fashion suggests.

Choosing the ID strategy without knowing its trade-off. What happens: someone chooses "hash of the URL" because it sounds elegant (the same code for the same URL, for free!) and discovers in production that two URLs collide, or chooses "counter" and finds out that competitors scrape their links by incrementing the code. Why it happens: they choose by the visible advantage and ignore the cost, which only appears at scale. How to detect it: if you can't name the problem of the strategy you chose, you didn't choose it, you adopted it. How to fix it: this whole module. By the time you reach lesson 7 you'll be able to say, of each strategy, its advantage and its measured cost —and that's the only honest basis for deciding.

Exercises

Exercise 1 — Name the access pattern. Without writing code, write in a single sentence the query Enlace runs 99% of the time (hint: it's what happens every time someone clicks a short link). Then write a query Enlace does not need to do, and explain in one sentence why that difference is a design clue.

See solution

The 99% query: "given a short_code, return its long_url" — an exact lookup by key, that brings a single row. In pseudo-SQL it would be SELECT long_url FROM links WHERE short_code = ?. It's the resolve operation, and it happens on every redirect; with Enlace's 100:1 ratio, it's ~4,000 times per second against ~40 writes.

A query Enlace does not need: something like "give me all the links created last Tuesday ordered by number of clicks" (SELECT ... WHERE created_at ... ORDER BY clicks), or any join against a users table. Enlace doesn't do ad-hoc analytics or relate entities on the hot path.

Why it's a design clue: when the access pattern is only "lookup by key", you're using almost none of what a relational database offers (joins, rich queries, aggregations). A key-value store does exactly that operation and nothing more, and that's why it can be simpler and faster for this case. The access pattern, not the fashion, is what should choose the store —and that's exactly what lesson 3 develops.

Exercise 2 — Reproduce the anchor. With Python (or even a calculator that handles big integers), compute 62⁷ and compare it with Enlace's 5-year demand (100M/month). Answer: how many times larger is the supply than the demand, and what percentage of the space is used? Then answer the question that matters: if Enlace doubled its traffic to 200M/month, would 7 characters still suffice?

See solution
supply = 62 ** 7                 # 3,521,614,606,208
demand = 100_000_000 * 12 * 5    # 6,000,000,000
print(supply / demand)           # 586.9357677013334
print(f"{demand / supply:.4%}")  # 0.1704%

The supply is ~587 times the demand; 0.17% of the space is used in 5 years.

If the traffic doubled to 200M/month, the 5-year demand would be 12,000,000,000 (12 billion). The supply is still 62⁷ = 3,521,614,606,208. The ratio becomes 3,521,614,606,208 / 12,000,000,000 ≈ 293 times, and the fraction used ~0.34%. Yes, they still suffice with plenty to spare: even at double the traffic, a third of one percent is used. That giant margin is why "7 characters" isn't a tight bet, and why the random+verification strategy will almost never collide (you measure it in lesson 5). The number that makes all this possible is that 62⁷ is enormous compared to the human demand for links.

Exercise 3 — Match the strategy with its problem. For each symptom a team reports in production, say which of the three generation strategies most likely caused it, and in one sentence why. (a) "Two different clients shortened two different URLs and received the same code; the second overwrote the first." (b) "A competitor is scraping all our links simply by requesting the code following the one they received." (c) "Under heavy write load, all the inserts are being serialized against a single component and that component is the bottleneck."

See solution
  • (a) Hash of the URL. The symptom is a collision: two different inputs produced the same truncated code. It's the characteristic problem of deriving the code from a hash and keeping few bits; you'll measure it in lesson 4 (at 16 bits with 50k URLs, almost 30% collide). A counter or random+verification don't produce this symptom (the first never collides; the second detects the clash and retries).
  • (b) Counter + base62. The symptom is that the codes are sequential and guessable: 4c92, 4c93, 4c94… If your code comes from a counter that goes 1,2,3,…, the next link is predictable. It's the counter's privacy/enumeration cost, and it's mitigated with an offset or permutation, or by choosing random.
  • (c) Counter + base62. The symptom is that the global counter is a single coordination point: each write has to ask "the next number" from the same place, and that serializes. It's why scaling the counter is a distribution problem —and the boundary with module 5, where it's resolved by splitting it (sharding) or using per-server ranges.

The lesson: each strategy has its problem, and recognizing the symptom in production is the other face of choosing with judgment. Lesson 7 puts the three in a table so the decision is deliberate, not a surprise.

Summary and next step

In this lesson you made the leap from the number to the model. Enlace stopped being a box and became a warehouse of lockers: a small record is stored (Link, five fields, ~1 KB), it's accessed almost always with a single operation ("ticket → contents", that is resolve(short_code) -> long_url), and all the interesting difficulty is in how the ticket's number is manufactured. You met the three strategies the module compares —hash (collides), counter+base62 (guessable, global counter), random+verification (retries)— and ran the numeric anchor: 62⁷ = 3,521,614,606,208, almost 587 times the 5-year demand, with only 0.17% of the space used. That giant margin is the permission that gives us freedom to choose.

Before moving on you should be able to: name Enlace's dominant access pattern in one sentence (exact lookup by short_code); list the five fields of the Link record; recite the three generation strategies with their characteristic problem; and reproduce the 62⁷ vs 6 billion anchor by saying what percentage of the space is used (0.17%).

What comes next is bringing the record down to earth. In lesson 2 you design the real Link table: each column with its type, the why of short_code as the primary key, exactly where the "~1 KB per record" you used in the estimation comes from, and the PostgreSQL DDL that creates it. It's the foundation the store decisions (lesson 3) and generation decisions (lessons 4 to 7) rest on.

Resources