Module 4: Native Partitioning in PostgreSQL
When to partition and when not to: the decision matrix before the syntax
Capsule description
Partitioning a table is surgery. If you don't need it and you do it, you add operational complexity, schemas that are harder to maintain, queries that can become slower, and foreign key limitations you didn't have before. If you need it and you don't do it, you're left with multi-second queries, autovacuum that takes hours, massive DELETEs that block the app, and a disk that fills up with no clean way to free it.
This capsule teaches you to decide before learning how to do it. The module's most common trap is jumping to the syntax (CREATE TABLE ... PARTITION BY RANGE) without having landed the "should I partition this table?". You're going to learn the quantitative criteria that separate "partition" from "don't partition yet," the decision tree for choosing among the 3 types (range / list / hash), and the explicit trade-offs so you know exactly what you're paying.
By the end you'll be able to receive a table description (volume, query pattern, retention, schema) and issue a technical verdict: partition or not, which type and why. That decision is the module's most expensive one to get wrong, and it's the first one you're going to make in any real ticket.
Mental model: partitioning divides the table physically, not virtually
When you think about "partitioning," the first instinct is something like indexes or views — a logical layer on top of a single table. It isn't that.
A partitioned table in PostgreSQL is many physical tables (the child partitions) coordinated by a parent table that's only metadata. The difference matters because it changes the costs and the benefits:
┌─────────────────────────────────────────────────────────────┐
│ NON-partitioned table (events) │
│ │
│ ┌───────────────────────────────────────────────────┐ │
│ │ events (50M rows, 30 GB on disk) │ │
│ │ - 1 big file │ │
│ │ - 1 B-tree index per indexed column (giant) │ │
│ │ - autovacuum over everything: takes hours │ │
│ │ - DELETE WHERE created_at < ... blocks │ │
│ └───────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────┐
│ Table partitioned by month (events) │
│ │
│ ┌──────────┐ events (parent, metadata only) │
│ │ events │ PARTITION BY RANGE (created_at) │
│ └─────┬────┘ │
│ │ │
│ ┌─────┴────────┬──────────────┬──────────────┬─────────┐ │
│ │ events_2025_ │ events_2025_ │ events_2026_ │ ... │ │
│ │ 11 (2GB) │ 12 (2GB) │ 01 (2GB) │ │ │
│ └──────────────┴──────────────┴──────────────┴─────────┘ │
│ │
│ - Each partition is an independent physical table │
│ - Local indexes per partition (small, they fit in RAM) │
│ - autovacuum per partition (fast, can be parallel) │
│ - DROP PARTITION events_2024_05: instant, frees disk │
│ │
└─────────────────────────────────────────────────────────────┘
Three ideas to internalize:
-
Each partition is a real table. It has its file on disk, its own indexes, its own autovacuum. PostgreSQL operates on each partition as if it were an independent table.
-
The planner decides which partitions to touch (partition pruning). If your query has
WHERE created_at >= '2026-01-01' AND created_at < '2026-02-01', the planner only scansevents_2026_01. The other 11 partitions don't even get opened. -
The cost per insert/update/delete changes. Before, the insert went straight to "the table." Now the insert has an extra step: the planner looks at the
created_atvalue, decides which partition it goes to, and writes there. It's micro overhead (~5%), but it exists.
This explains why partitioning wins on large tables with range queries (each partition is small, the indexes fit in RAM, pruning avoids scanning what doesn't apply) and why it loses on small tables (the routing overhead and the operational complexity aren't compensated by a real benefit).
The 4 reasons to partition
These are the only 4 legitimate reasons. If your case doesn't fit at least one, don't partition.
Reason 1: range queries over massive tables
Symptom: your table has >50M rows, and the most common queries filter by a predictable column (typically created_at, event_date, tenant_id).
Why partitioning helps: partition pruning makes the planner only scan the partitions of the requested range. A 50M-row table without partitioning scans (or uses a big index) over all the rows. Partitioned by month, a query for the current month only touches 2M rows. The difference is 25× less work, before we even talk about indexes.
Typical case: analytics dashboards, listings of events from the last N days, recent activity reports.
-- On a NON-partitioned 50M-row table:
EXPLAIN ANALYZE
SELECT count(*) FROM events
WHERE created_at >= '2026-04-01' AND created_at < '2026-05-01';
-- Bitmap Heap Scan or Index Scan, ~3.2 seconds
-- On the same table partitioned by month:
EXPLAIN ANALYZE
SELECT count(*) FROM events
WHERE created_at >= '2026-04-01' AND created_at < '2026-05-01';
-- Only scans events_2026_04, ~40 milliseconds
Reason 2: autovacuum takes too long
Symptom: pg_stat_progress_vacuum shows that events is being vacuumed for hours. Meanwhile, indexes are partially blocked, writes pile up, and the "next vacuum" is already behind.
Why partitioning helps: autovacuum operates per table. If you have a 30 GB table, vacuum has to scan 30 GB. If you divide it into 12 partitions of 2.5 GB each, vacuum operates on each one independently (and can run in parallel if you configure autovacuum_max_workers). Each vacuum finishes in minutes, not hours.
Typical case: tables with frequent UPDATE/DELETE (sessions, job statuses, like counters). Pure-INSERT event tables (without UPDATE) suffer less from this problem, but they still win for reason 1.
Reason 3: you need to delete old data without touching production
Symptom: compliance, costs, or performance force you to delete old data. DELETE FROM events WHERE created_at < NOW() - INTERVAL '1 year' takes 40 minutes, holds locks that block inserts, and leaves the table with bloat (it needs VACUUM FULL or pg_repack afterwards).
Why partitioning helps: DROP PARTITION events_2024_01 is instantaneous. PostgreSQL just removes the metadata entry and dereferences the files on disk. It doesn't scan rows, it doesn't take long locks, it frees disk immediately. The difference is minutes vs milliseconds.
Typical case: audit logs, events, sessions, metrics, sensor readings. Any data with a retention policy.
-- On a NON-partitioned table:
DELETE FROM events WHERE created_at < '2025-01-01';
-- 40 minutes, SHARE UPDATE lock, bloat afterwards.
-- On a partitioned table:
DROP TABLE events_2024_12; -- instant
ALTER TABLE events DETACH PARTITION events_2024_12; -- alternative, keeps the table but removes it from the parent
Reason 4: insert performance on tables with heavy indexes
Symptom: the inserts into events started fast (microseconds) and now take milliseconds. When you profile, the cost is in maintaining the giant B-tree indexes.
Why partitioning helps: the inserts only go to the "now" partition (typically the latest one). That partition is small and its indexes are small. The old partitions' indexes aren't touched on the insert. Inserts go back to microseconds.
Typical case: append-only tables with several indexes. Logs, events, chat messages.
The 4 contraindications for NOT partitioning
If your case fits any of these, don't partition (yet).
Contraindication 1: a small table
Quantitative criterion: fewer than 10 million rows and less than 10 GB on disk.
At that size, a well-indexed table answers queries in milliseconds without partitioning. Adding partitioning only adds complexity without benefit: slower insert routing, a schema that's harder to change, foreign keys with limitations, an EXPLAIN that's more complex to read.
Exception: if the table is going to grow predictably and fast (you know it's going to be at 50M rows in 6 months), partitioning from the start can make sense to avoid the expensive migration later. But document the reason ("partitioned prematurely because the plan is 50M rows in Q4") so the team doesn't question it.
Contraindication 2: the queries don't use the partition key
Symptom: you have a users table with 50M rows that's growing. You think "let's partition by created_at because that's what's natural in partitioning." But 95% of the queries are SELECT * FROM users WHERE email = '...' or WHERE id = .... Those queries don't use created_at in the WHERE. Partition pruning doesn't apply. PostgreSQL scans all the partitions for every query.
Why it happens: partition pruning only works if the WHERE filters by the partition key (or by an expression the planner can evaluate against the partitions' ranges). Without that, partitioning is worse than not partitioning (overhead with no benefit).
How to detect it before partitioning: review pg_stat_statements and look at the most common queries. Do they have a consistent WHERE over the column you're thinking of using as the partition key? If not, don't partition by that column.
Contraindication 3: you need unique constraints or foreign keys that don't include the partition key
Symptom: you want to partition orders by created_at. But you have UNIQUE (order_number) and order_number doesn't include created_at. PostgreSQL doesn't allow that unique constraint on partitioned tables — unique constraints must include the partition key.
Why it happens: PostgreSQL can't guarantee global uniqueness without scanning all the partitions. The restriction is: any UNIQUE/PRIMARY KEY on a partitioned table must include the partition key's column.
Workaround: if you need global uniqueness over a column that isn't the partition key, use an auxiliary "reservations" table or an app-level validation index. But this is additional complexity. If the constraint is critical and you can't relax it, don't partition that table.
Foreign keys: PG 12+ supports foreign keys toward partitioned tables (an improvement over older versions), but there are caveats. If your table has many incoming FKs, validate that the pattern works in your specific version.
Contraindication 4: the team has no operational experience with partitioned tables
Symptom: you're the only one on the team who understands partitioning. If you leave, nobody knows how to debug why an insert lands in the default partition, or how to recreate the setup in staging.
Why it matters: partitioning adds operations. Creating future partitions, dropping old ones, monitoring the default partition (which shouldn't grow), understanding why a query doesn't use pruning, handling ALTER TABLE over the parent table. If the team isn't prepared, partitioning becomes the "dirty area" nobody touches and everybody fears breaking.
Mitigation: invest in pg_partman (capsule 07) to automate what can be automated, document the decisions extensively, and make at least one other person on the team understand the setup. If you can't do any of the three, consider deferring partitioning until the team is ready.
Decision matrix with quantitative criteria
Land your case against this table. If you hesitate, partitioning is the wrong decision.
| Criterion | Partition | Don't partition (yet) |
|---|---|---|
| Rows in the table | > 50M (ideally >100M) | < 10M |
| Size on disk | > 50 GB | < 10 GB |
| Predictable monthly growth | > 5M rows/month | < 500k rows/month |
| Dominant query pattern | By date range or by tenant_id | Random access (by id, email, etc.) |
| Retention policy | Yes, with a fixed window (e.g. 18 months) | No retention (permanent data) |
| Autovacuum on the table | Takes > 30 minutes | Takes < 5 minutes |
Range DELETE blocking | Yes, a real operational problem | No, the DELETEs are rare or small |
| Unique constraints without the partition key | No (or they can be included) | Yes, and they're critical |
| Team operationally prepared | Yes (at least 2 people) | No, it would be a dirty area |
General rule: you need at least 3 "partition" criteria and no strong "don't partition" to justify the surgery. If you only have 1-2, the indexes and the tuning from #12 usually suffice.
Decision tree: range, list, or hash
If you decided you are partitioning, now you choose the type. PostgreSQL offers three:
Which column dominates your queries' filter?
│
┌──────────────────────┼──────────────────────┐
▼ ▼ ▼
A continuous date A discrete category None obvious /
(created_at, (tenant_id, region, uniform
event_date, country, status) distribution wanted
timestamp) (user_id, random hash)
│ │ │
▼ ▼ ▼
┌────────────────┐ ┌────────────────┐ ┌────────────────┐
│ RANGE │ │ LIST │ │ HASH │
└────────────────┘ └────────────────┘ └────────────────┘
│ │ │
▼ ▼ ▼
The 80% case: Multi-tenant case: Edge case:
time-series, SaaS by tenant, distribute rows
events, logs, apps by region, with no natural
metrics job statuses criterion
│ │ │
▼ ▼ ▼
Capsule 03 Capsule 04 Capsule 05
Range partitioning is the most used (probably 80% of real cases). You partition by an ordered column (dates, incrementing IDs) into contiguous ranges. Ideal for time-series and data with retention.
List partitioning works when the column is discrete and the values are known (a fixed set of tenants, regions, statuses). Ideal for multi-tenant SaaS.
Hash partitioning is used when there's no natural criterion but you want to distribute rows evenly among N partitions (typically to parallelize I/O or reduce contention). It's the least common but it exists for a reason.
Worked example: 3 cases, 3 decisions
Let's apply the decision matrix to three concrete cases.
Case 1: the Blog API's events table
Data:
- Current volume: 50M rows, 30 GB on disk
- Growth: 2M rows/month (predictable, it comes from user tracking)
- Dominant queries:
WHERE created_at >= ? AND created_at < ?(dashboards),WHERE post_id = ? ORDER BY created_at DESC LIMIT 100(feed) - Retention: 18 months (compliance)
- Autovacuum: 25 minutes
- DELETE of old data: currently broken (40 min, blocks the API)
- Unique constraints: only
id BIGSERIAL PRIMARY KEY(it can be converted into a composite PK(id, created_at))
Decision matrix:
- ✅ Volume > 50M
- ✅ Size > 10 GB
- ⚠️ Growth 2M/month (below the 5M/month threshold, but predictable and sustained)
- ✅ Range queries dominant
- ✅ 18-month retention
- ✅ Blocking DELETE = a real problem
- ✅ The PK can be
(id, created_at) - ✅ The team understands it (you learned it)
Verdict: partition. Type: RANGE by created_at, monthly granularity (24 partitions max at a time = 18 months of retention + 6 future months). This is what you're going to do in capsule 08.
Case 2: the users table of the same Blog API
Data:
- Volume: 8M users
- Growth: 200k users/month
- Dominant queries:
WHERE id = ?(auth),WHERE email = ?(login),WHERE username = ?(profile) - Retention: none (users are permanent)
- Autovacuum: 3 minutes
- DELETE: very rare (soft-delete with
deleted_at) - Critical unique constraints:
email,username(they don't includecreated_at)
Decision matrix:
- ❌ Volume < 10M
- ❌ Manageable size
- ❌ Random-access queries (by id/email/username)
- ❌ No retention
- ❌ Fast autovacuum
- ❌ DELETEs aren't a problem
- ❌ Unique constraints over email/username — they CAN'T be included in the partition key without breaking the logic
Verdict: don't partition. Your users table doesn't need partitioning and isn't going to benefit from it. Keep indexes on email, username, id (which you already have) and you'll be fine for years. If you reach 100M users some day, reevaluate — but not before.
Case 3: the tenant_data table of a B2B SaaS app
Data:
- Volume: 200M rows (data for 500 tenants, unevenly distributed: 5 tenants have 50M rows each, 495 have <1M each)
- Growth: 10M/month
- Dominant queries:
WHERE tenant_id = ?(always — it's in the RLS) - Retention: per tenant, configurable (default 36 months)
- Autovacuum: 1 hour
- DELETE: per tenant when a tenant cancels
- Unique constraints:
(tenant_id, external_id)already includes tenant_id
Decision matrix:
- ✅ Volume > 100M
- ✅ Growth > 5M/month
- ✅ The dominant query uses
tenant_id(the potential partition key) - ✅ Slow autovacuum
- ✅ DELETE per tenant is real
- ✅ Compatible unique constraint
- ✅ Team prepared
Verdict: partition. Type: here it gets interesting. The instinct is LIST by tenant_id (1 partition per tenant). But with 500 tenants, that's 500 partitions — the planner overhead is high. A better option: LIST by tenant groups (the 5 big ones in their dedicated partitions, the 495 small ones spread across 4-8 partitions by hash of tenant_id). It's the "partitioned by tenant tier" case — a pattern documented by Citus. Capsule 04 develops it.
Why does this matter in real work?
1. The decision "let's not partition this table yet" has as much value as "let's partition it." Partitioning is weeks of work (migration + testing + monitoring). If you convince the team not to do it when it doesn't apply, you save those weeks and avoid permanent operational complexity. Your technical authority grows when you can defend a "no" with data instead of an enthusiastic "yes."
2. The decision "let's partition, but not this one, this other one first" gets made all the time. In any system with several large tables, you have to prioritize. The highest-volume one isn't always the winner — the one with range queries that hurt is the one that benefits first. Knowing how to prioritize uses this capsule's decision matrix.
3. The quantitative criteria protect you from cargo cult. "Famous company X partitions everything" isn't a reason to partition your table. What works is: "this table has Y rows, grows Z/month, autovacuum takes W minutes, DELETE blocks — let's partition." That argument passes code review in any serious team.
4. It's the question they're going to ask you in senior interviews. "When would you partition a table and when wouldn't you?" is a standard question for senior backend roles. The weak answer ("when it's very big") lowers your level. The solid answer (quantitative criteria + decision tree + trade-offs) puts you where you want to be.
Traps and common mistakes
Mistake 1 (conceptual): assuming partitioning always speeds up queries
Symptom: you partition a table. The query SELECT * FROM events WHERE user_id = 42 is slower than before.
Why it happens: the query doesn't use the partition key (created_at) in the WHERE. PostgreSQL can't prune and it scans all the partitions, one by one. Before it was one table; now it's 24 partitions. The planner spends more time planning, and the executor opens 24 files instead of 1.
How to tell: review pg_stat_statements before partitioning. Do the dominant queries filter by the column you're going to use as the partition key? If not, don't partition by that column (or don't partition at all).
Fix: if the queries are random access (by id, email, etc.), partitioning probably doesn't help. Consider: specific indexes, a query rewrite, materialized views (module 5), or accepting the current latency if it's within your SLO.
Mistake 2 (practical): choosing the wrong partitioning granularity
Symptom: you partition events by year because "it's simpler." Each partition has 25M rows. Queries for the last month still scan 25M rows (the current year's partition).
Why it happens: the partitioning granularity has to align with the typical query ranges. If your queries ask for last month's data, partition by month. If they ask for the last day, consider partitioning by day (and use pg_partman to manage the volume of partitions).
How to decide the granularity:
- If a typical partition has >10M rows, the granularity is probably too coarse.
- If you have hundreds of partitions simultaneously, it's probably too fine (the planner overhead grows).
- Sweet spot: 12-48 partitions, each between 500k and 10M rows.
Fix: adjust the granularity. For events with 2M/month, monthly is perfect (2M per partition, 24 partitions). For metrics with 100M/month, daily makes more sense.
Mistake 3 (conceptual): thinking partitioning solves "a very big table" regardless of how it's used
Symptom: a products table (5M rows, queries by category_id with a join) doesn't perform well. You decide to partition by created_at because "that's what's natural." The queries get worse.
Why it's confusing: partitioning isn't "generic tuning" for large tables. It's an optimization specific to range queries on the partition key. If your problem is a JOIN with another table, the solution is better indexes (covering, composite), not partitioning.
How to tell: is the problem that EXPLAIN shows it scans too many rows because the queries are by range? Partitioning can help. Is the problem that the JOIN is slow or the index isn't used? That isn't a partitioning problem — it's an indexing/query-plan problem. Go back to guide #12.
Mistake 4 (practical): underestimating the impact on migrations and schema changes
Symptom: after partitioning events, you decide to add a new column. ALTER TABLE events ADD COLUMN device_type TEXT looks the same as before. But now it affects 24 partitions, each with its inserts, indexes, and autovacuum. The ALTER takes 10× longer and blocks inserts to all 24 partitions simultaneously.
Why it happens: an ALTER on the parent table propagates to all the children. If your app has frequent migrations and events's schema changes a lot, partitioning adds friction to every change.
How to prevent it: stabilize the schema before partitioning. If you know you're going to touch the schema 5 times in the next quarter, postpone the migration to partitioned until the schema stabilizes. Use the zero-downtime migration techniques (#13) that apply.
Mistake 5 (conceptual): treating the default partition as a "safe overflow partition"
Symptom: you create CREATE TABLE events_default PARTITION OF events DEFAULT. It reassures you to think "if something out of range comes in, it goes there." You forget to monitor it. Six months later it has 50M rows because there was a bug and all the inserts were coming in with created_at = NULL or weird dates.
Why it happens: the default partition is a useful safety net — but it isn't a place for data to live. If rows end up in default, it's a sign of a bug (a badly defined range, invalid dates, a future partition not created in time).
How to prevent it: monitor SELECT count(*) FROM events_default periodically (alert if > 1000). Configure pg_partman to create future partitions ahead of time. If default grows, investigate why before deleting.
Exercises
Exercise 1: apply the decision matrix to 4 tables
For each of these tables, decide whether to partition or not, and if so, which type:
a) sessions — 30M rows, 8 GB, grows 5M/month. Queries: WHERE user_id = ? AND created_at > NOW() - INTERVAL '30 days'. Retention: 90 days. Autovacuum: 15 min. The retention DELETE blocks for 20 min every night.
b) products — 2M rows, 1 GB. Grows 10k/month. Queries: WHERE category_id = ? (with a join), WHERE id = ?. No retention. Autovacuum: 30 seconds.
c) metrics_minutely — 800M rows, 200 GB. Grows 200M/month (1 row per metric per minute). Queries: WHERE metric_name = ? AND ts BETWEEN ? AND ? with a typical range of 24 hours. Retention: 30 days raw, then aggregated into metrics_hourly.
d) audit_log for B2B SaaS — 100M rows, 40 GB. Grows 8M/month. Queries: WHERE tenant_id = ? AND created_at > ? (90% of the time, it comes from the tenant's admin dashboard). Retention: 5 years (compliance). Autovacuum: 90 min.
See solution
a) sessions: Partition, RANGE by created_at with weekly or monthly granularity.
- ✅ 30M rows, grows fast (5M/month), 90-day retention, slow autovacuum, blocking DELETE.
- The query includes
created_at > ...so pruning applies (even though it also filters byuser_id). - Granularity: monthly gives 3-4 simultaneous partitions (90 days of retention + 1 buffer). Weekly gives 13. Monthly is fine to start.
- Bonus: combine it with a local index over
(user_id, created_at)for the user-specific queries.
b) products: Don't partition.
- ❌ 2M rows << 10M.
- ❌ Random-access queries (by
category_idwith a join, not by range). - ❌ No retention, no blocking DELETE.
- Correct indexes over
category_idandidsolve everything.
c) metrics_minutely: Partition, RANGE by ts with daily granularity.
- ✅ 800M rows is massive, grows 200M/month, 30-day raw retention, queries by time range.
- Granularity: daily gives 30-31 simultaneous partitions. If it were monthly, each partition would have 200M rows — still gigantic.
pg_partmanis mandatory here (create a future partition every day, drop the old ones).- Bonus: consider a
BRINindex onts(an alternative to a B-tree for sequential data).
d) audit_log: Partition, RANGE by created_at with monthly granularity, but consider composite partitioning if you have version 16+.
- ✅ All the criteria aligned.
- Monthly with 5-year retention = 60 simultaneous partitions. It's at the high limit but manageable.
- Bonus: you can sub-partition by
tenant_id(LIST) inside each monthly partition if you have very few large tenants dominating the volume. It's complex but powerful. If the team isn't ready, monthly without sub-partitioning is a good base. - Combine it with RLS by
tenant_id(guide #13) for isolation.
General pattern: the three critical questions are: (a) >10M rows and predictable growth? (b) do the dominant queries filter by a stable column? (c) is there retention or a blocking DELETE? If all three are yes, partition. If one fails, evaluate hard.
Exercise 2: defend a "don't partition yet"
Your lead asks you to partition the notifications table:
- 8M rows, 4 GB.
- Grows 500k/month.
- Queries:
WHERE user_id = ? AND read = false ORDER BY created_at DESC LIMIT 50(95% of the time). - Retention: 6 months (old notifs can be deleted).
- Autovacuum: 4 minutes.
Your lead says "we're going to hit 50M rows at some point, better to start now." Argue technically why not to partition now (or when to do it).
See solution
The argument against partitioning now:
-
The current volume is far from the threshold: 8M rows and 4 GB are well within the range where correct indexes solve it. PostgreSQL handles 50M rows with an efficient composite B-tree without partitioning.
-
The query pattern: the dominant query filters by
user_id(not bycreated_at). Even if we partitioned bycreated_at, the query would scan all the partitions (without useful pruning) unless we combine it with an explicit date filter in the app. That would require a code change. -
Uncertain granularity: with growth of 500k/month, monthly gives partitions of 500k rows — too small to justify the planner overhead. Quarterly or semi-annual would be more reasonable, but then you lose granularity for
DROP PARTITIONby retention. -
The cost of migrating later is manageable: you have zero-downtime migration techniques (#13). When the table is at 30-50M rows and the queries start hurting, you can partition then. It isn't like multi-DB sharding.
-
A better alternative for now: a composite index
(user_id, read, created_at DESC)covers the dominant query in 1ms. For retention, aDELETEwithLIMITin small batches (technique from #13) can run during low-traffic hours without blocking.
When to actually partition:
- When
notificationsis at 30M+ rows and - When the dominant queries include a
created_atfilter (checkpg_stat_statements) or - When autovacuum goes over 30 minutes or
- When the retention
DELETEblocks the app in an observable way.
A phrase to use with your lead: "Partitioning now gives us 0 observable benefit and adds permanent complexity. Let's set a trigger: when we reach 30M rows or autovacuum > 30min, we do it. Meanwhile, we use a compound index that solves the dominant query in 1ms."
Exercise 3: choose the partitioning granularity
You have an api_calls table that's going to be partitioned by called_at (the call's date). Volume: 5M calls/day = ~150M/month. Retention: 6 months. Dominant queries: dashboards showing the last 24 hours.
What granularity do you choose and why?
See solution
Choose daily granularity. Reasons:
-
Size per partition: 5M rows per partition is optimal (each one manageable, indexes fit in RAM, fast vacuum).
-
Alignment with the queries: the dashboards ask for 24 hours. With daily partitions, most queries touch 1-2 partitions (the current day and possibly the previous one). With weekly partitions, they'd always touch the current week's partition (35M rows) — far more data scanned.
-
Granular retention: dropping data older than 180 days is trivial (
DROP PARTITION api_calls_2025_11_03). If it were weekly, you'd lose granularity for precise retention. -
Number of partitions: 180 simultaneous partitions. That's high but manageable with
pg_partman. If it were hourly, it'd be 4320 partitions — the planner overhead grows, a problem.
Accepted trade-offs:
- More partitions to create: without
pg_partman, maintaining 180 partitions manually would be hell. Withpg_partmanautomated, it's trivial. - Slower ALTER TABLE: a schema change affects 180 partitions. Acceptable because api_calls should have a stable schema.
- More verbose EXPLAIN: 180 partitions show up in some plans. Manageable.
Recommended configuration:
SELECT partman.create_parent(
p_parent_table => 'public.api_calls',
p_control => 'called_at',
p_type => 'native',
p_interval => '1 day',
p_premake => 7 -- always creates 7 future partitions
);
UPDATE partman.part_config
SET retention = '180 days',
retention_keep_table = false -- DROP the old ones, don't DETACH
WHERE parent_table = 'public.api_calls';
If you go up to 50M calls/day: consider hourly partitioning. Each partition would be 2M rows. But then 4320 simultaneous partitions — you need to validate that your workload tolerates the planner overhead. The alternative is sub-partitioning (daily + hourly inside), but that's high complexity. For the 5M/day case, daily is the right call.
Exercise 4: identify the blocking contraindication
You have an orders table:
- 80M rows, 50 GB.
- Grows 4M/month.
- Dominant queries:
WHERE order_number = ?(lookup),WHERE customer_id = ? ORDER BY created_at DESC LIMIT 20(history),WHERE status = 'pending' AND created_at > ...(the pending queue). - Retention: never deleted (financial compliance).
- Critical constraint:
UNIQUE (order_number)— the entire system depends on globally unique order numbers.
Your teammate proposes partitioning by created_at monthly. Is there a blocker?
See solution
Yes, there's a serious blocker. The UNIQUE (order_number) doesn't include the partition key (created_at). PostgreSQL doesn't allow that constraint on a partitioned table — UNIQUE constraints must include all the partition key's columns.
What happens if you try:
-- This FAILS in PostgreSQL:
CREATE TABLE orders (
id BIGSERIAL,
order_number TEXT NOT NULL,
customer_id BIGINT,
created_at TIMESTAMPTZ NOT NULL,
status TEXT,
PRIMARY KEY (id),
UNIQUE (order_number) -- ERROR: UNIQUE on a partitioned table must include the partition key
) PARTITION BY RANGE (created_at);
-- ERROR: unique constraint on partitioned table must include all partitioning columns
-- DETAIL: UNIQUE constraint on table "orders" lacks column "created_at"
Workarounds:
-
Change the constraint to
UNIQUE (order_number, created_at). It works technically but it breaks the global uniqueness guarantee — two orders with the sameorder_numberin different months would be valid. Unacceptable for the use case. -
Validate uniqueness at the app level. Add
SELECT 1 FROM orders WHERE order_number = ?before every insert. It's a race condition (two simultaneous requests can both pass the check), it requires an advisory lock or a retry. High operational complexity. -
An auxiliary "reservations" table. Create
order_numbers_reserved (order_number TEXT PK), not partitioned. Before inserting intoorders, insert into the auxiliary one (UNIQUE guaranteed). If it fails, retry with another number. Medium complexity. -
Don't partition this table. Keep
ordersnon-partitioned. 80M rows with good indexes is manageable. If the range queries (status = 'pending' AND created_at > ...) are slow, consider a partial index overWHERE status = 'pending'— that does help without partitioning.
Verdict: option 4 is the right one for the short term. If in 2-3 years the table is at 500M rows and autovacuum becomes unsustainable, evaluate option 3 (the auxiliary table) plus partitioning.
Lesson: unique constraints are a serious contraindication for partitioning. When you detect them, the decision changes from "how to partition" to "how to architect the uniqueness before partitioning."
Exercise 5: convince the team NOT to partition
Your team is convinced that partitioning all the large tables is going to "solve the performance problems." There are 4 tables under discussion: users (10M), posts (3M), events (50M), audit_logs (200M).
Write a short document (5-10 lines) justifying the decision: which ones to partition, which ones not to, and why.
See solution
Document — Partitioning decision, Q2 2026:
After applying the decision matrix to the 4 candidate tables:
users(10M rows): DON'T partition. The dominant queries are random access (by id, email, username). Partition pruning wouldn't apply. The current indexes solve it in <5ms. We'll reevaluate if we reach 100M+ users.
posts(3M rows): DON'T partition. Volume well below the threshold (10M). Range queries are rare (most are by id or by author). Indexes and FTS (module 3) cover the rest.
events(50M rows): YES, partition —RANGEbycreated_at, monthly. It meets all the criteria: volume, predictable growth (2M/month), dominant range queries, 18-month retention, and the DELETE currently blocks the API. Expected benefit: dashboards 4s → <100ms; DROP of old data in milliseconds.
audit_logs(200M rows): YES, partition —RANGEbycreated_at, monthly. Same criteria aseventsbut with compliance (5-year retention). Combine with RLS for per-tenant isolation.Plan: start with
events(biggest immediate impact, serves as the team's learning case).audit_logsin Q3 once the operation withpg_partmanis consolidated.
A phrase for defending the "no" on users: "Partitioning users doesn't solve any current problem and it adds limitations (we can't have UNIQUE on email without including the partition key, schema changes would be 8× slower). When the problem shows up, we'll solve it. Right now it doesn't apply."
Lesson: the right decision isn't "all or nothing." It's "these yes, these no, in this order, with this criterion." Technical seniors defend that precision.
Summary and next step
In this capsule you learned to decide before learning how to do it:
-
Partitioning physically divides the table into sub-tables (it isn't a logical layer). That explains why it wins on large tables with range queries and loses on small tables or ones with random queries.
-
The 4 reasons to partition: range queries over massive tables, autovacuum that's too slow, a blocking DELETE for retention, insert performance with heavy indexes. If your case doesn't fit at least one, don't partition.
-
The 4 serious contraindications: a small table (<10M), queries that don't use the partition key, unique constraints without the partition key, a team that isn't operationally prepared.
-
A decision matrix with quantitative criteria: concrete thresholds (>50M rows, >50 GB, etc.) that protect you from cargo cult and give you the vocabulary to defend technical decisions.
-
The decision tree of the 3 types: RANGE for continuous dates/IDs (80% of cases), LIST for discrete categories (multi-tenant), HASH for uniform distribution (edge cases).
Before moving on you should be able to:
- Receive a table's description and issue a "partition / don't partition yet" verdict with quantitative justification.
- Choose among RANGE, LIST, and HASH based on the query pattern and the nature of the candidate column.
- Identify blocking contraindications (unique constraints, random queries, an unprepared team) before starting the migration.
- Defend a "let's not partition this table" with data, not with hunches.
Next capsule — Range partitioning by date. You already know when and which type. Now the syntax. Capsule 03 teaches you the 80% case: partitioning events by created_at monthly using PostgreSQL 16's declarative partitioning. You're going to see the complete SQL, the gotchas (default partition, propagated indexes, foreign keys), and the EXPLAIN ANALYZE that demonstrates partition pruning. You're going to come out able to apply it in a real Alembic migration.
Resources
- PostgreSQL 16 — Table Partitioning, the "Declarative Partitioning" section — the official reference. The "Limitations" section is critical for understanding what does NOT work.
- Crunchy Data — When to Use Partitioning in PostgreSQL — pragmatism from the Crunchy team with real-world examples.
- Hironobu Suzuki — The Internals of PostgreSQL: Partitioning — how PostgreSQL implements partitioning internally. Useful for understanding why pruning works and when it doesn't.
- AWS RDS — Best Practices for PostgreSQL Partitioning — operational, from a mainstream cloud provider. It considers RDS but the principles apply to any deployment.
- Citus Data — Distributing Postgres Tables for Scale — combining partitioning with sharding (advanced, out of scope for this module, but good context on when partitioning alone isn't enough).
- depesz — Why PostgreSQL Partitioning is great (when used right) — an analysis of when partitioning helps and when it doesn't, with examples of large tables in production.
Module 4 — Advanced PostgreSQL for Backend Guide
Next capsule: Range partitioning by date — the 80% case, with complete SQL and before/after EXPLAIN.