Module 4: Native Partitioning in PostgreSQL

When to partition and when not to: the decision matrix before the syntax

Capsule description

Partitioning a table is surgery. If you don't need it and you do it, you add operational complexity, schemas that are harder to maintain, queries that can become slower, and foreign key limitations you didn't have before. If you need it and you don't do it, you're left with multi-second queries, autovacuum that takes hours, massive DELETEs that block the app, and a disk that fills up with no clean way to free it.

This capsule teaches you to decide before learning how to do it. The module's most common trap is jumping to the syntax (CREATE TABLE ... PARTITION BY RANGE) without having landed the "should I partition this table?". You're going to learn the quantitative criteria that separate "partition" from "don't partition yet," the decision tree for choosing among the 3 types (range / list / hash), and the explicit trade-offs so you know exactly what you're paying.

By the end you'll be able to receive a table description (volume, query pattern, retention, schema) and issue a technical verdict: partition or not, which type and why. That decision is the module's most expensive one to get wrong, and it's the first one you're going to make in any real ticket.


Mental model: partitioning divides the table physically, not virtually

When you think about "partitioning," the first instinct is something like indexes or views — a logical layer on top of a single table. It isn't that.

A partitioned table in PostgreSQL is many physical tables (the child partitions) coordinated by a parent table that's only metadata. The difference matters because it changes the costs and the benefits:

┌─────────────────────────────────────────────────────────────┐
│  NON-partitioned table (events)                             │
│                                                             │
│   ┌───────────────────────────────────────────────────┐     │
│   │ events (50M rows, 30 GB on disk)                  │     │
│   │  - 1 big file                                     │     │
│   │  - 1 B-tree index per indexed column (giant)      │     │
│   │  - autovacuum over everything: takes hours        │     │
│   │  - DELETE WHERE created_at < ... blocks           │     │
│   └───────────────────────────────────────────────────┘     │
│                                                             │
└─────────────────────────────────────────────────────────────┘

                              ▼

┌─────────────────────────────────────────────────────────────┐
│  Table partitioned by month (events)                        │
│                                                             │
│   ┌──────────┐       events (parent, metadata only)         │
│   │ events   │       PARTITION BY RANGE (created_at)        │
│   └─────┬────┘                                              │
│         │                                                   │
│   ┌─────┴────────┬──────────────┬──────────────┬─────────┐ │
│   │ events_2025_ │ events_2025_ │ events_2026_ │  ...    │ │
│   │   11 (2GB)   │   12 (2GB)   │   01 (2GB)   │         │ │
│   └──────────────┴──────────────┴──────────────┴─────────┘ │
│                                                             │
│   - Each partition is an independent physical table         │
│   - Local indexes per partition (small, they fit in RAM)    │
│   - autovacuum per partition (fast, can be parallel)        │
│   - DROP PARTITION events_2024_05: instant, frees disk      │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Three ideas to internalize:

  1. Each partition is a real table. It has its file on disk, its own indexes, its own autovacuum. PostgreSQL operates on each partition as if it were an independent table.

  2. The planner decides which partitions to touch (partition pruning). If your query has WHERE created_at >= '2026-01-01' AND created_at < '2026-02-01', the planner only scans events_2026_01. The other 11 partitions don't even get opened.

  3. The cost per insert/update/delete changes. Before, the insert went straight to "the table." Now the insert has an extra step: the planner looks at the created_at value, decides which partition it goes to, and writes there. It's micro overhead (~5%), but it exists.

This explains why partitioning wins on large tables with range queries (each partition is small, the indexes fit in RAM, pruning avoids scanning what doesn't apply) and why it loses on small tables (the routing overhead and the operational complexity aren't compensated by a real benefit).


The 4 reasons to partition

These are the only 4 legitimate reasons. If your case doesn't fit at least one, don't partition.

Reason 1: range queries over massive tables

Symptom: your table has >50M rows, and the most common queries filter by a predictable column (typically created_at, event_date, tenant_id).

Why partitioning helps: partition pruning makes the planner only scan the partitions of the requested range. A 50M-row table without partitioning scans (or uses a big index) over all the rows. Partitioned by month, a query for the current month only touches 2M rows. The difference is 25× less work, before we even talk about indexes.

Typical case: analytics dashboards, listings of events from the last N days, recent activity reports.

-- On a NON-partitioned 50M-row table:
EXPLAIN ANALYZE
SELECT count(*) FROM events
WHERE created_at >= '2026-04-01' AND created_at < '2026-05-01';
-- Bitmap Heap Scan or Index Scan, ~3.2 seconds

-- On the same table partitioned by month:
EXPLAIN ANALYZE
SELECT count(*) FROM events
WHERE created_at >= '2026-04-01' AND created_at < '2026-05-01';
-- Only scans events_2026_04, ~40 milliseconds

Reason 2: autovacuum takes too long

Symptom: pg_stat_progress_vacuum shows that events is being vacuumed for hours. Meanwhile, indexes are partially blocked, writes pile up, and the "next vacuum" is already behind.

Why partitioning helps: autovacuum operates per table. If you have a 30 GB table, vacuum has to scan 30 GB. If you divide it into 12 partitions of 2.5 GB each, vacuum operates on each one independently (and can run in parallel if you configure autovacuum_max_workers). Each vacuum finishes in minutes, not hours.

Typical case: tables with frequent UPDATE/DELETE (sessions, job statuses, like counters). Pure-INSERT event tables (without UPDATE) suffer less from this problem, but they still win for reason 1.

Reason 3: you need to delete old data without touching production

Symptom: compliance, costs, or performance force you to delete old data. DELETE FROM events WHERE created_at < NOW() - INTERVAL '1 year' takes 40 minutes, holds locks that block inserts, and leaves the table with bloat (it needs VACUUM FULL or pg_repack afterwards).

Why partitioning helps: DROP PARTITION events_2024_01 is instantaneous. PostgreSQL just removes the metadata entry and dereferences the files on disk. It doesn't scan rows, it doesn't take long locks, it frees disk immediately. The difference is minutes vs milliseconds.

Typical case: audit logs, events, sessions, metrics, sensor readings. Any data with a retention policy.

-- On a NON-partitioned table:
DELETE FROM events WHERE created_at < '2025-01-01';
-- 40 minutes, SHARE UPDATE lock, bloat afterwards.

-- On a partitioned table:
DROP TABLE events_2024_12;  -- instant
ALTER TABLE events DETACH PARTITION events_2024_12;  -- alternative, keeps the table but removes it from the parent

Reason 4: insert performance on tables with heavy indexes

Symptom: the inserts into events started fast (microseconds) and now take milliseconds. When you profile, the cost is in maintaining the giant B-tree indexes.

Why partitioning helps: the inserts only go to the "now" partition (typically the latest one). That partition is small and its indexes are small. The old partitions' indexes aren't touched on the insert. Inserts go back to microseconds.

Typical case: append-only tables with several indexes. Logs, events, chat messages.


The 4 contraindications for NOT partitioning

If your case fits any of these, don't partition (yet).

Contraindication 1: a small table

Quantitative criterion: fewer than 10 million rows and less than 10 GB on disk.

At that size, a well-indexed table answers queries in milliseconds without partitioning. Adding partitioning only adds complexity without benefit: slower insert routing, a schema that's harder to change, foreign keys with limitations, an EXPLAIN that's more complex to read.

Exception: if the table is going to grow predictably and fast (you know it's going to be at 50M rows in 6 months), partitioning from the start can make sense to avoid the expensive migration later. But document the reason ("partitioned prematurely because the plan is 50M rows in Q4") so the team doesn't question it.

Contraindication 2: the queries don't use the partition key

Symptom: you have a users table with 50M rows that's growing. You think "let's partition by created_at because that's what's natural in partitioning." But 95% of the queries are SELECT * FROM users WHERE email = '...' or WHERE id = .... Those queries don't use created_at in the WHERE. Partition pruning doesn't apply. PostgreSQL scans all the partitions for every query.

Why it happens: partition pruning only works if the WHERE filters by the partition key (or by an expression the planner can evaluate against the partitions' ranges). Without that, partitioning is worse than not partitioning (overhead with no benefit).

How to detect it before partitioning: review pg_stat_statements and look at the most common queries. Do they have a consistent WHERE over the column you're thinking of using as the partition key? If not, don't partition by that column.

Contraindication 3: you need unique constraints or foreign keys that don't include the partition key

Symptom: you want to partition orders by created_at. But you have UNIQUE (order_number) and order_number doesn't include created_at. PostgreSQL doesn't allow that unique constraint on partitioned tables — unique constraints must include the partition key.

Why it happens: PostgreSQL can't guarantee global uniqueness without scanning all the partitions. The restriction is: any UNIQUE/PRIMARY KEY on a partitioned table must include the partition key's column.

Workaround: if you need global uniqueness over a column that isn't the partition key, use an auxiliary "reservations" table or an app-level validation index. But this is additional complexity. If the constraint is critical and you can't relax it, don't partition that table.

Foreign keys: PG 12+ supports foreign keys toward partitioned tables (an improvement over older versions), but there are caveats. If your table has many incoming FKs, validate that the pattern works in your specific version.

Contraindication 4: the team has no operational experience with partitioned tables

Symptom: you're the only one on the team who understands partitioning. If you leave, nobody knows how to debug why an insert lands in the default partition, or how to recreate the setup in staging.

Why it matters: partitioning adds operations. Creating future partitions, dropping old ones, monitoring the default partition (which shouldn't grow), understanding why a query doesn't use pruning, handling ALTER TABLE over the parent table. If the team isn't prepared, partitioning becomes the "dirty area" nobody touches and everybody fears breaking.

Mitigation: invest in pg_partman (capsule 07) to automate what can be automated, document the decisions extensively, and make at least one other person on the team understand the setup. If you can't do any of the three, consider deferring partitioning until the team is ready.


Decision matrix with quantitative criteria

Land your case against this table. If you hesitate, partitioning is the wrong decision.

CriterionPartitionDon't partition (yet)
Rows in the table> 50M (ideally >100M)< 10M
Size on disk> 50 GB< 10 GB
Predictable monthly growth> 5M rows/month< 500k rows/month
Dominant query patternBy date range or by tenant_idRandom access (by id, email, etc.)
Retention policyYes, with a fixed window (e.g. 18 months)No retention (permanent data)
Autovacuum on the tableTakes > 30 minutesTakes < 5 minutes
Range DELETE blockingYes, a real operational problemNo, the DELETEs are rare or small
Unique constraints without the partition keyNo (or they can be included)Yes, and they're critical
Team operationally preparedYes (at least 2 people)No, it would be a dirty area

General rule: you need at least 3 "partition" criteria and no strong "don't partition" to justify the surgery. If you only have 1-2, the indexes and the tuning from #12 usually suffice.


Decision tree: range, list, or hash

If you decided you are partitioning, now you choose the type. PostgreSQL offers three:

                    Which column dominates your queries' filter?
                                       │
                ┌──────────────────────┼──────────────────────┐
                ▼                      ▼                      ▼
       A continuous date       A discrete category      None obvious /
        (created_at,           (tenant_id, region,      uniform
         event_date,           country, status)         distribution wanted
         timestamp)                                     (user_id, random hash)
                │                      │                      │
                ▼                      ▼                      ▼
       ┌────────────────┐     ┌────────────────┐     ┌────────────────┐
       │     RANGE      │     │     LIST       │     │     HASH       │
       └────────────────┘     └────────────────┘     └────────────────┘
                │                      │                      │
                ▼                      ▼                      ▼
        The 80% case:          Multi-tenant case:     Edge case:
        time-series,           SaaS by tenant,        distribute rows
        events, logs,          apps by region,        with no natural
        metrics                job statuses           criterion
                │                      │                      │
                ▼                      ▼                      ▼
        Capsule 03             Capsule 04              Capsule 05

Range partitioning is the most used (probably 80% of real cases). You partition by an ordered column (dates, incrementing IDs) into contiguous ranges. Ideal for time-series and data with retention.

List partitioning works when the column is discrete and the values are known (a fixed set of tenants, regions, statuses). Ideal for multi-tenant SaaS.

Hash partitioning is used when there's no natural criterion but you want to distribute rows evenly among N partitions (typically to parallelize I/O or reduce contention). It's the least common but it exists for a reason.


Worked example: 3 cases, 3 decisions

Let's apply the decision matrix to three concrete cases.

Case 1: the Blog API's events table

Data:

  • Current volume: 50M rows, 30 GB on disk
  • Growth: 2M rows/month (predictable, it comes from user tracking)
  • Dominant queries: WHERE created_at >= ? AND created_at < ? (dashboards), WHERE post_id = ? ORDER BY created_at DESC LIMIT 100 (feed)
  • Retention: 18 months (compliance)
  • Autovacuum: 25 minutes
  • DELETE of old data: currently broken (40 min, blocks the API)
  • Unique constraints: only id BIGSERIAL PRIMARY KEY (it can be converted into a composite PK (id, created_at))

Decision matrix:

  • ✅ Volume > 50M
  • ✅ Size > 10 GB
  • ⚠️ Growth 2M/month (below the 5M/month threshold, but predictable and sustained)
  • ✅ Range queries dominant
  • ✅ 18-month retention
  • ✅ Blocking DELETE = a real problem
  • ✅ The PK can be (id, created_at)
  • ✅ The team understands it (you learned it)

Verdict: partition. Type: RANGE by created_at, monthly granularity (24 partitions max at a time = 18 months of retention + 6 future months). This is what you're going to do in capsule 08.

Case 2: the users table of the same Blog API

Data:

  • Volume: 8M users
  • Growth: 200k users/month
  • Dominant queries: WHERE id = ? (auth), WHERE email = ? (login), WHERE username = ? (profile)
  • Retention: none (users are permanent)
  • Autovacuum: 3 minutes
  • DELETE: very rare (soft-delete with deleted_at)
  • Critical unique constraints: email, username (they don't include created_at)

Decision matrix:

  • ❌ Volume < 10M
  • ❌ Manageable size
  • ❌ Random-access queries (by id/email/username)
  • ❌ No retention
  • ❌ Fast autovacuum
  • ❌ DELETEs aren't a problem
  • ❌ Unique constraints over email/username — they CAN'T be included in the partition key without breaking the logic

Verdict: don't partition. Your users table doesn't need partitioning and isn't going to benefit from it. Keep indexes on email, username, id (which you already have) and you'll be fine for years. If you reach 100M users some day, reevaluate — but not before.

Case 3: the tenant_data table of a B2B SaaS app

Data:

  • Volume: 200M rows (data for 500 tenants, unevenly distributed: 5 tenants have 50M rows each, 495 have <1M each)
  • Growth: 10M/month
  • Dominant queries: WHERE tenant_id = ? (always — it's in the RLS)
  • Retention: per tenant, configurable (default 36 months)
  • Autovacuum: 1 hour
  • DELETE: per tenant when a tenant cancels
  • Unique constraints: (tenant_id, external_id) already includes tenant_id

Decision matrix:

  • ✅ Volume > 100M
  • ✅ Growth > 5M/month
  • ✅ The dominant query uses tenant_id (the potential partition key)
  • ✅ Slow autovacuum
  • ✅ DELETE per tenant is real
  • ✅ Compatible unique constraint
  • ✅ Team prepared

Verdict: partition. Type: here it gets interesting. The instinct is LIST by tenant_id (1 partition per tenant). But with 500 tenants, that's 500 partitions — the planner overhead is high. A better option: LIST by tenant groups (the 5 big ones in their dedicated partitions, the 495 small ones spread across 4-8 partitions by hash of tenant_id). It's the "partitioned by tenant tier" case — a pattern documented by Citus. Capsule 04 develops it.


Why does this matter in real work?

1. The decision "let's not partition this table yet" has as much value as "let's partition it." Partitioning is weeks of work (migration + testing + monitoring). If you convince the team not to do it when it doesn't apply, you save those weeks and avoid permanent operational complexity. Your technical authority grows when you can defend a "no" with data instead of an enthusiastic "yes."

2. The decision "let's partition, but not this one, this other one first" gets made all the time. In any system with several large tables, you have to prioritize. The highest-volume one isn't always the winner — the one with range queries that hurt is the one that benefits first. Knowing how to prioritize uses this capsule's decision matrix.

3. The quantitative criteria protect you from cargo cult. "Famous company X partitions everything" isn't a reason to partition your table. What works is: "this table has Y rows, grows Z/month, autovacuum takes W minutes, DELETE blocks — let's partition." That argument passes code review in any serious team.

4. It's the question they're going to ask you in senior interviews. "When would you partition a table and when wouldn't you?" is a standard question for senior backend roles. The weak answer ("when it's very big") lowers your level. The solid answer (quantitative criteria + decision tree + trade-offs) puts you where you want to be.


Traps and common mistakes

Mistake 1 (conceptual): assuming partitioning always speeds up queries

Symptom: you partition a table. The query SELECT * FROM events WHERE user_id = 42 is slower than before.

Why it happens: the query doesn't use the partition key (created_at) in the WHERE. PostgreSQL can't prune and it scans all the partitions, one by one. Before it was one table; now it's 24 partitions. The planner spends more time planning, and the executor opens 24 files instead of 1.

How to tell: review pg_stat_statements before partitioning. Do the dominant queries filter by the column you're going to use as the partition key? If not, don't partition by that column (or don't partition at all).

Fix: if the queries are random access (by id, email, etc.), partitioning probably doesn't help. Consider: specific indexes, a query rewrite, materialized views (module 5), or accepting the current latency if it's within your SLO.

Mistake 2 (practical): choosing the wrong partitioning granularity

Symptom: you partition events by year because "it's simpler." Each partition has 25M rows. Queries for the last month still scan 25M rows (the current year's partition).

Why it happens: the partitioning granularity has to align with the typical query ranges. If your queries ask for last month's data, partition by month. If they ask for the last day, consider partitioning by day (and use pg_partman to manage the volume of partitions).

How to decide the granularity:

  • If a typical partition has >10M rows, the granularity is probably too coarse.
  • If you have hundreds of partitions simultaneously, it's probably too fine (the planner overhead grows).
  • Sweet spot: 12-48 partitions, each between 500k and 10M rows.

Fix: adjust the granularity. For events with 2M/month, monthly is perfect (2M per partition, 24 partitions). For metrics with 100M/month, daily makes more sense.

Mistake 3 (conceptual): thinking partitioning solves "a very big table" regardless of how it's used

Symptom: a products table (5M rows, queries by category_id with a join) doesn't perform well. You decide to partition by created_at because "that's what's natural." The queries get worse.

Why it's confusing: partitioning isn't "generic tuning" for large tables. It's an optimization specific to range queries on the partition key. If your problem is a JOIN with another table, the solution is better indexes (covering, composite), not partitioning.

How to tell: is the problem that EXPLAIN shows it scans too many rows because the queries are by range? Partitioning can help. Is the problem that the JOIN is slow or the index isn't used? That isn't a partitioning problem — it's an indexing/query-plan problem. Go back to guide #12.

Mistake 4 (practical): underestimating the impact on migrations and schema changes

Symptom: after partitioning events, you decide to add a new column. ALTER TABLE events ADD COLUMN device_type TEXT looks the same as before. But now it affects 24 partitions, each with its inserts, indexes, and autovacuum. The ALTER takes 10× longer and blocks inserts to all 24 partitions simultaneously.

Why it happens: an ALTER on the parent table propagates to all the children. If your app has frequent migrations and events's schema changes a lot, partitioning adds friction to every change.

How to prevent it: stabilize the schema before partitioning. If you know you're going to touch the schema 5 times in the next quarter, postpone the migration to partitioned until the schema stabilizes. Use the zero-downtime migration techniques (#13) that apply.

Mistake 5 (conceptual): treating the default partition as a "safe overflow partition"

Symptom: you create CREATE TABLE events_default PARTITION OF events DEFAULT. It reassures you to think "if something out of range comes in, it goes there." You forget to monitor it. Six months later it has 50M rows because there was a bug and all the inserts were coming in with created_at = NULL or weird dates.

Why it happens: the default partition is a useful safety net — but it isn't a place for data to live. If rows end up in default, it's a sign of a bug (a badly defined range, invalid dates, a future partition not created in time).

How to prevent it: monitor SELECT count(*) FROM events_default periodically (alert if > 1000). Configure pg_partman to create future partitions ahead of time. If default grows, investigate why before deleting.


Exercises

Exercise 1: apply the decision matrix to 4 tables

For each of these tables, decide whether to partition or not, and if so, which type:

a) sessions — 30M rows, 8 GB, grows 5M/month. Queries: WHERE user_id = ? AND created_at > NOW() - INTERVAL '30 days'. Retention: 90 days. Autovacuum: 15 min. The retention DELETE blocks for 20 min every night.

b) products — 2M rows, 1 GB. Grows 10k/month. Queries: WHERE category_id = ? (with a join), WHERE id = ?. No retention. Autovacuum: 30 seconds.

c) metrics_minutely — 800M rows, 200 GB. Grows 200M/month (1 row per metric per minute). Queries: WHERE metric_name = ? AND ts BETWEEN ? AND ? with a typical range of 24 hours. Retention: 30 days raw, then aggregated into metrics_hourly.

d) audit_log for B2B SaaS — 100M rows, 40 GB. Grows 8M/month. Queries: WHERE tenant_id = ? AND created_at > ? (90% of the time, it comes from the tenant's admin dashboard). Retention: 5 years (compliance). Autovacuum: 90 min.

See solution

a) sessions: Partition, RANGE by created_at with weekly or monthly granularity.

  • ✅ 30M rows, grows fast (5M/month), 90-day retention, slow autovacuum, blocking DELETE.
  • The query includes created_at > ... so pruning applies (even though it also filters by user_id).
  • Granularity: monthly gives 3-4 simultaneous partitions (90 days of retention + 1 buffer). Weekly gives 13. Monthly is fine to start.
  • Bonus: combine it with a local index over (user_id, created_at) for the user-specific queries.

b) products: Don't partition.

  • ❌ 2M rows << 10M.
  • ❌ Random-access queries (by category_id with a join, not by range).
  • ❌ No retention, no blocking DELETE.
  • Correct indexes over category_id and id solve everything.

c) metrics_minutely: Partition, RANGE by ts with daily granularity.

  • ✅ 800M rows is massive, grows 200M/month, 30-day raw retention, queries by time range.
  • Granularity: daily gives 30-31 simultaneous partitions. If it were monthly, each partition would have 200M rows — still gigantic.
  • pg_partman is mandatory here (create a future partition every day, drop the old ones).
  • Bonus: consider a BRIN index on ts (an alternative to a B-tree for sequential data).

d) audit_log: Partition, RANGE by created_at with monthly granularity, but consider composite partitioning if you have version 16+.

  • ✅ All the criteria aligned.
  • Monthly with 5-year retention = 60 simultaneous partitions. It's at the high limit but manageable.
  • Bonus: you can sub-partition by tenant_id (LIST) inside each monthly partition if you have very few large tenants dominating the volume. It's complex but powerful. If the team isn't ready, monthly without sub-partitioning is a good base.
  • Combine it with RLS by tenant_id (guide #13) for isolation.

General pattern: the three critical questions are: (a) >10M rows and predictable growth? (b) do the dominant queries filter by a stable column? (c) is there retention or a blocking DELETE? If all three are yes, partition. If one fails, evaluate hard.

Exercise 2: defend a "don't partition yet"

Your lead asks you to partition the notifications table:

  • 8M rows, 4 GB.
  • Grows 500k/month.
  • Queries: WHERE user_id = ? AND read = false ORDER BY created_at DESC LIMIT 50 (95% of the time).
  • Retention: 6 months (old notifs can be deleted).
  • Autovacuum: 4 minutes.

Your lead says "we're going to hit 50M rows at some point, better to start now." Argue technically why not to partition now (or when to do it).

See solution

The argument against partitioning now:

  1. The current volume is far from the threshold: 8M rows and 4 GB are well within the range where correct indexes solve it. PostgreSQL handles 50M rows with an efficient composite B-tree without partitioning.

  2. The query pattern: the dominant query filters by user_id (not by created_at). Even if we partitioned by created_at, the query would scan all the partitions (without useful pruning) unless we combine it with an explicit date filter in the app. That would require a code change.

  3. Uncertain granularity: with growth of 500k/month, monthly gives partitions of 500k rows — too small to justify the planner overhead. Quarterly or semi-annual would be more reasonable, but then you lose granularity for DROP PARTITION by retention.

  4. The cost of migrating later is manageable: you have zero-downtime migration techniques (#13). When the table is at 30-50M rows and the queries start hurting, you can partition then. It isn't like multi-DB sharding.

  5. A better alternative for now: a composite index (user_id, read, created_at DESC) covers the dominant query in 1ms. For retention, a DELETE with LIMIT in small batches (technique from #13) can run during low-traffic hours without blocking.

When to actually partition:

  • When notifications is at 30M+ rows and
  • When the dominant queries include a created_at filter (check pg_stat_statements) or
  • When autovacuum goes over 30 minutes or
  • When the retention DELETE blocks the app in an observable way.

A phrase to use with your lead: "Partitioning now gives us 0 observable benefit and adds permanent complexity. Let's set a trigger: when we reach 30M rows or autovacuum > 30min, we do it. Meanwhile, we use a compound index that solves the dominant query in 1ms."

Exercise 3: choose the partitioning granularity

You have an api_calls table that's going to be partitioned by called_at (the call's date). Volume: 5M calls/day = ~150M/month. Retention: 6 months. Dominant queries: dashboards showing the last 24 hours.

What granularity do you choose and why?

See solution

Choose daily granularity. Reasons:

  1. Size per partition: 5M rows per partition is optimal (each one manageable, indexes fit in RAM, fast vacuum).

  2. Alignment with the queries: the dashboards ask for 24 hours. With daily partitions, most queries touch 1-2 partitions (the current day and possibly the previous one). With weekly partitions, they'd always touch the current week's partition (35M rows) — far more data scanned.

  3. Granular retention: dropping data older than 180 days is trivial (DROP PARTITION api_calls_2025_11_03). If it were weekly, you'd lose granularity for precise retention.

  4. Number of partitions: 180 simultaneous partitions. That's high but manageable with pg_partman. If it were hourly, it'd be 4320 partitions — the planner overhead grows, a problem.

Accepted trade-offs:

  • More partitions to create: without pg_partman, maintaining 180 partitions manually would be hell. With pg_partman automated, it's trivial.
  • Slower ALTER TABLE: a schema change affects 180 partitions. Acceptable because api_calls should have a stable schema.
  • More verbose EXPLAIN: 180 partitions show up in some plans. Manageable.

Recommended configuration:

SELECT partman.create_parent(
  p_parent_table => 'public.api_calls',
  p_control => 'called_at',
  p_type => 'native',
  p_interval => '1 day',
  p_premake => 7  -- always creates 7 future partitions
);

UPDATE partman.part_config
SET retention = '180 days',
    retention_keep_table = false  -- DROP the old ones, don't DETACH
WHERE parent_table = 'public.api_calls';

If you go up to 50M calls/day: consider hourly partitioning. Each partition would be 2M rows. But then 4320 simultaneous partitions — you need to validate that your workload tolerates the planner overhead. The alternative is sub-partitioning (daily + hourly inside), but that's high complexity. For the 5M/day case, daily is the right call.

Exercise 4: identify the blocking contraindication

You have an orders table:

  • 80M rows, 50 GB.
  • Grows 4M/month.
  • Dominant queries: WHERE order_number = ? (lookup), WHERE customer_id = ? ORDER BY created_at DESC LIMIT 20 (history), WHERE status = 'pending' AND created_at > ... (the pending queue).
  • Retention: never deleted (financial compliance).
  • Critical constraint: UNIQUE (order_number) — the entire system depends on globally unique order numbers.

Your teammate proposes partitioning by created_at monthly. Is there a blocker?

See solution

Yes, there's a serious blocker. The UNIQUE (order_number) doesn't include the partition key (created_at). PostgreSQL doesn't allow that constraint on a partitioned table — UNIQUE constraints must include all the partition key's columns.

What happens if you try:

-- This FAILS in PostgreSQL:
CREATE TABLE orders (
  id BIGSERIAL,
  order_number TEXT NOT NULL,
  customer_id BIGINT,
  created_at TIMESTAMPTZ NOT NULL,
  status TEXT,
  PRIMARY KEY (id),
  UNIQUE (order_number)  -- ERROR: UNIQUE on a partitioned table must include the partition key
) PARTITION BY RANGE (created_at);

-- ERROR: unique constraint on partitioned table must include all partitioning columns
-- DETAIL: UNIQUE constraint on table "orders" lacks column "created_at"

Workarounds:

  1. Change the constraint to UNIQUE (order_number, created_at). It works technically but it breaks the global uniqueness guarantee — two orders with the same order_number in different months would be valid. Unacceptable for the use case.

  2. Validate uniqueness at the app level. Add SELECT 1 FROM orders WHERE order_number = ? before every insert. It's a race condition (two simultaneous requests can both pass the check), it requires an advisory lock or a retry. High operational complexity.

  3. An auxiliary "reservations" table. Create order_numbers_reserved (order_number TEXT PK), not partitioned. Before inserting into orders, insert into the auxiliary one (UNIQUE guaranteed). If it fails, retry with another number. Medium complexity.

  4. Don't partition this table. Keep orders non-partitioned. 80M rows with good indexes is manageable. If the range queries (status = 'pending' AND created_at > ...) are slow, consider a partial index over WHERE status = 'pending' — that does help without partitioning.

Verdict: option 4 is the right one for the short term. If in 2-3 years the table is at 500M rows and autovacuum becomes unsustainable, evaluate option 3 (the auxiliary table) plus partitioning.

Lesson: unique constraints are a serious contraindication for partitioning. When you detect them, the decision changes from "how to partition" to "how to architect the uniqueness before partitioning."

Exercise 5: convince the team NOT to partition

Your team is convinced that partitioning all the large tables is going to "solve the performance problems." There are 4 tables under discussion: users (10M), posts (3M), events (50M), audit_logs (200M).

Write a short document (5-10 lines) justifying the decision: which ones to partition, which ones not to, and why.

See solution

Document — Partitioning decision, Q2 2026:

After applying the decision matrix to the 4 candidate tables:

  • users (10M rows): DON'T partition. The dominant queries are random access (by id, email, username). Partition pruning wouldn't apply. The current indexes solve it in <5ms. We'll reevaluate if we reach 100M+ users.

  • posts (3M rows): DON'T partition. Volume well below the threshold (10M). Range queries are rare (most are by id or by author). Indexes and FTS (module 3) cover the rest.

  • events (50M rows): YES, partition — RANGE by created_at, monthly. It meets all the criteria: volume, predictable growth (2M/month), dominant range queries, 18-month retention, and the DELETE currently blocks the API. Expected benefit: dashboards 4s → <100ms; DROP of old data in milliseconds.

  • audit_logs (200M rows): YES, partition — RANGE by created_at, monthly. Same criteria as events but with compliance (5-year retention). Combine with RLS for per-tenant isolation.

Plan: start with events (biggest immediate impact, serves as the team's learning case). audit_logs in Q3 once the operation with pg_partman is consolidated.

A phrase for defending the "no" on users: "Partitioning users doesn't solve any current problem and it adds limitations (we can't have UNIQUE on email without including the partition key, schema changes would be 8× slower). When the problem shows up, we'll solve it. Right now it doesn't apply."

Lesson: the right decision isn't "all or nothing." It's "these yes, these no, in this order, with this criterion." Technical seniors defend that precision.


Summary and next step

In this capsule you learned to decide before learning how to do it:

  • Partitioning physically divides the table into sub-tables (it isn't a logical layer). That explains why it wins on large tables with range queries and loses on small tables or ones with random queries.

  • The 4 reasons to partition: range queries over massive tables, autovacuum that's too slow, a blocking DELETE for retention, insert performance with heavy indexes. If your case doesn't fit at least one, don't partition.

  • The 4 serious contraindications: a small table (<10M), queries that don't use the partition key, unique constraints without the partition key, a team that isn't operationally prepared.

  • A decision matrix with quantitative criteria: concrete thresholds (>50M rows, >50 GB, etc.) that protect you from cargo cult and give you the vocabulary to defend technical decisions.

  • The decision tree of the 3 types: RANGE for continuous dates/IDs (80% of cases), LIST for discrete categories (multi-tenant), HASH for uniform distribution (edge cases).

Before moving on you should be able to:

  • Receive a table's description and issue a "partition / don't partition yet" verdict with quantitative justification.
  • Choose among RANGE, LIST, and HASH based on the query pattern and the nature of the candidate column.
  • Identify blocking contraindications (unique constraints, random queries, an unprepared team) before starting the migration.
  • Defend a "let's not partition this table" with data, not with hunches.

Next capsule — Range partitioning by date. You already know when and which type. Now the syntax. Capsule 03 teaches you the 80% case: partitioning events by created_at monthly using PostgreSQL 16's declarative partitioning. You're going to see the complete SQL, the gotchas (default partition, propagated indexes, foreign keys), and the EXPLAIN ANALYZE that demonstrates partition pruning. You're going to come out able to apply it in a real Alembic migration.


Resources

  1. PostgreSQL 16 — Table Partitioning, the "Declarative Partitioning" section — the official reference. The "Limitations" section is critical for understanding what does NOT work.
  2. Crunchy Data — When to Use Partitioning in PostgreSQL — pragmatism from the Crunchy team with real-world examples.
  3. Hironobu Suzuki — The Internals of PostgreSQL: Partitioning — how PostgreSQL implements partitioning internally. Useful for understanding why pruning works and when it doesn't.
  4. AWS RDS — Best Practices for PostgreSQL Partitioning — operational, from a mainstream cloud provider. It considers RDS but the principles apply to any deployment.
  5. Citus Data — Distributing Postgres Tables for Scale — combining partitioning with sharding (advanced, out of scope for this module, but good context on when partitioning alone isn't enough).
  6. depesz — Why PostgreSQL Partitioning is great (when used right) — an analysis of when partitioning helps and when it doesn't, with examples of large tables in production.

Module 4 — Advanced PostgreSQL for Backend Guide

Next capsule: Range partitioning by date — the 80% case, with complete SQL and before/after EXPLAIN.