Every write is a new snapshot

Description

This lesson creates Kiosko's second Iceberg table: kiosko.dim_product, with a four-column schema — product_id, product_name, category, unit_cost — and no history column at all. It loads it, for the first time, with the V1 values: Kiosko's four products, with P002 Energy Bar still at category='snacks', unit_cost=0.60. By the end of this lesson you're going to have the first half of this module's complete experiment: a table with exactly one snapshot, the "before" photo of the change lesson 3 is going to trigger.

Connection to the module. Lesson 1 promised two real writes against kiosko.dim_product. This lesson does the first one. The mechanism — table.append() creates a new snapshot — you already saw, exactly the same way, in module 1's lesson 6, against kiosko.fact_orders. This lesson doesn't discover any new PyIceberg method; what it contributes is the case — a table designed specifically for the P002 change — that lessons 3 through 7 of this module are going to build the complete trip through time on top of.

An analogy: installing the shelf, before the first photo

Picking up lesson 1's analogy: before the supermarket clerk can take the first photo of a shelf, the shelf itself has to exist — empty, with its slots marked, ready to receive merchandise. This lesson does exactly that: it creates kiosko.dim_product with its declared schema — four slots, no more — and then fills it, for the first time, with the V1 merchandise. The moment the merchandise reaches the shelf is, precisely, the moment the first photo gets taken: this table's first snapshot.

Worked example: the table, and its first snapshot

Step 1 — Create kiosko.dim_product, with no history column at all

With the kiosko catalog already loaded — the same SqlCatalog backed by SQLite you installed in module 1 — declare the new table's schema:

# create_dim_product.py
import os

from pyiceberg.catalog import load_catalog
from pyiceberg.schema import Schema
from pyiceberg.types import DoubleType, NestedField, StringType

warehouse_path = os.path.abspath("kiosko_warehouse")
catalog_db_path = os.path.abspath("kiosko_catalog.db")

catalog = load_catalog(
    "kiosko", type="sql",
    uri=f"sqlite:///{catalog_db_path}", warehouse=f"file://{warehouse_path}",
)

dim_product_schema = Schema(
    NestedField(field_id=1, name="product_id", field_type=StringType(), required=True),
    NestedField(field_id=2, name="product_name", field_type=StringType(), required=True),
    NestedField(field_id=3, name="category", field_type=StringType(), required=True),
    NestedField(field_id=4, name="unit_cost", field_type=DoubleType(), required=True),
)

table = catalog.create_table("kiosko.dim_product", schema=dim_product_schema)

print("Table created:", table.name())
print()
print(table.schema())
print()
print("Tables in the kiosko namespace:", catalog.list_tables("kiosko"))

What to expect (verified by running the actual script, with module 1's kiosko.fact_orders already loaded in the same catalog):

Table created: ('kiosko', 'dim_product')

table {
  1: product_id: required string
  2: product_name: required string
  3: category: required string
  4: unit_cost: required double
}

Tables in the kiosko namespace: [('kiosko', 'dim_product'), ('kiosko', 'fact_orders')]

Notice the four columns, and what's not there: no valid_from, no valid_to, no is_current, no dbt_scd_id. This isn't an oversight — it's, precisely, this whole module's central point. catalog.list_tables("kiosko") confirms the kiosko namespace now holds two tables: module 1's fact_orders, and this freshly created, still-empty dim_product.

Step 2 — The V1 values: P002 is still snacks/0.60

# load_v1.py -- kiosko.dim_product, first load, P002 = snacks / 0.60
import os

import pyarrow as pa
from pyiceberg.catalog import load_catalog

warehouse_path = os.path.abspath("kiosko_warehouse")
catalog_db_path = os.path.abspath("kiosko_catalog.db")

catalog = load_catalog(
    "kiosko", type="sql",
    uri=f"sqlite:///{catalog_db_path}", warehouse=f"file://{warehouse_path}",
)
table = catalog.load_table("kiosko.dim_product")

dim_product_pa_schema = pa.schema([
    pa.field("product_id", pa.string(), nullable=False),
    pa.field("product_name", pa.string(), nullable=False),
    pa.field("category", pa.string(), nullable=False),
    pa.field("unit_cost", pa.float64(), nullable=False),
])

# V1 -- dim_product's state BEFORE the P002 change, effective through 2026-08-15
DIM_PRODUCT_V1 = [
    {"product_id": "P001", "product_name": "Bottled Water 600ml", "category": "beverages", "unit_cost": 0.40},
    {"product_id": "P002", "product_name": "Energy Bar", "category": "snacks", "unit_cost": 0.60},
    {"product_id": "P003", "product_name": "Instant Coffee Sachet", "category": "beverages", "unit_cost": 0.35},
    {"product_id": "P004", "product_name": "Phone Charger Cable", "category": "electronics", "unit_cost": 2.10},
]

pa_table_v1 = pa.Table.from_pylist(DIM_PRODUCT_V1, schema=dim_product_pa_schema)
table.append(pa_table_v1)

print("table.scan().to_arrow() after the first load:")
for row in table.scan().to_arrow().to_pylist():
    print(f"  {row['product_id']}  {row['product_name']:<22} {row['category']:<10} unit_cost={row['unit_cost']}")

What to expect (verified by running the actual script):

table.scan().to_arrow() after the first load:
  P001  Bottled Water 600ml    beverages  unit_cost=0.4
  P002  Energy Bar             snacks     unit_cost=0.6
  P003  Instant Coffee Sachet  beverages  unit_cost=0.35
  P004  Phone Charger Cable    electronics unit_cost=2.1

Four rows, one per product — the same DIM_PRODUCT you already know from the six previous guides, with P002 still in its original state. None of this is mechanically new: it's exactly the same table.append() you already used in module 1's lesson 6.

Step 3 — Confirm the first snapshot, captured in a variable

snap_v1 = table.current_snapshot().snapshot_id

print("dim_product's first snapshot, captured in snap_v1")
print("type(snap_v1):", type(snap_v1).__name__)
print("table.history() has", len(table.history()), "entry(entries)")

What to expect (snap_v1 is an integer Iceberg assigns at commit time, different on every run — never hardcoded; see this module's lesson 4 for the full rule):

dim_product's first snapshot, captured in snap_v1
type(snap_v1): int
table.history() has 1 entry(entries)

Notice something important for the rest of this module: you captured snap_v1 in a variable immediately after the write, not later. This discipline — capturing the snapshot_id in the same block of code that generated it, never "later, when I need it" — is what lesson 4 is going to turn into an explicit rule. For now, keep snap_v1: it's the "before" photo you're going to use to travel through time in lesson 5.

Diagram: kiosko.dim_product's first photo

flowchart LR
    A["catalog.create_table\nkiosko.dim_product\n(4 columns, 0 rows)"] --> B["table.append(V1)\nP002 = snacks / 0.60"]
    B --> C["snapshot snap_v1\noperation: append\nadded-records: 4"]
    C -.->|"captured right away"| D["snap_v1 = table.current_snapshot().snapshot_id"]
    D -.->|"lesson 5"| E["table.scan(snapshot_id=snap_v1)\nrecovers this photo, later on"]

On disk, after this lesson, kiosko_warehouse/kiosko/dim_product/ has the same shape you already saw for fact_orders in module 1: a Parquet data file with the four rows, a manifest file listing it, a manifest list pointing to that manifest file, and two metadata files — one from the empty table (step 1), another from the first snapshot (step 3).

Going deeper: what counts as "a write" to Iceberg

It's worth being precise about which operations create a new snapshot, because the rest of this module — and a good part of the ones that follow — depends on this distinction. table.append(), table.overwrite() (lesson 3), table.delete(), and table.upsert() (module 6) are all operations that modify the table's data, and each of them produces, at minimum, one new snapshot — lesson 3 is going to show overwrite() can produce more than one. What does not count as a data write, in this precise sense, is a schema evolution — table.update_schema(), which you're only going to use starting in module 4: adding or dropping a column changes the metadata file, but doesn't touch any Parquet file or add a new entry to the data snapshot history. This distinction — data changes produce snapshots; schema changes are a different kind of operation, over the same metadata chain — is exactly what all of module 4 is going to develop in depth. For now, what you need to hold onto is simpler: every time you ask Iceberg to add, replace, or delete rows, you get a new photo, filed alongside every earlier one.

Common mistakes

Creating kiosko.dim_product without having loaded module 1's kiosko.fact_orders first, and being surprised the namespace already exists. What happens: someone, starting this module in a new directory without module 1's state, runs step 1 of this lesson and gets an error because the kiosko namespace isn't registered. Why it happens: this lesson deliberately assumes you're continuing from the state modules 1 and 2 left — the same kiosko_catalog.db and kiosko_warehouse/ as always — not starting from scratch. How to spot it: if catalog.create_table("kiosko.dim_product", ...) fails with NoSuchNamespaceError, your catalog doesn't have the kiosko namespace created yet. How to fix it: run catalog.create_namespace("kiosko") first — the same step from module 1's lesson 5 — or, if you want to start from scratch for this specific module, use lesson 8's project, which rebuilds all the necessary state in a single script.

Assuming table.append() on dim_product replaces existing rows with the same product_id. What happens: someone, already planning lesson 3's P002 change, tries to solve it with a second table.append() that only includes P002's updated row, expecting it to replace the old row. Why it happens: in many databases, an implicit UPSERT by primary key is common behavior, and it's easy to assume append() works that way. How to spot it: if after "updating" P002 you see two rows with product_id='P002' in table.scan().to_arrow(), instead of a single updated one, that's this exact mistake's symptom. How to fix it: table.append() always adds, never replaces — to replace the full content of a one-row-per-key table like dim_product, the correct operation is table.overwrite(), exactly the one this module's lesson 3 uses. (Module 6 of this guide teaches a third alternative, table.upsert(), designed specifically to update by key without replacing the whole table.)

Exercises

Exercise 1 — Reproduce the creation and first load yourself. With module 1's kiosko catalog available, run this lesson's three steps on your own machine. Confirm you see V1's four rows with P002 at category='snacks', unit_cost=0.6, and that table.history() reports exactly one entry.

See solution

If your catalog already had the kiosko namespace (inherited from module 1), your output should exactly match this lesson's: the table created with four columns, V1's four rows in the same order, and table.history() with a single entry. Your snap_v1 is going to be a different integer than this lesson's — that's exactly expected, not an error.

Exercise 2 — Explain why this lesson doesn't show snap_v1's literal value. In 1-2 sentences, explain why this lesson's step 3 "What to expect" block doesn't print the actual snapshot_id number, unlike, for example, type(snap_v1), which is a fixed, reproducible value.

See solution

The snapshot_id is an identifier Iceberg assigns at the exact moment of the commit — it doesn't depend on Kiosko's business data, but on Iceberg's own internal identifier-generation mechanism — so it's going to be different on every run, even running the same script twice from scratch. Showing a specific number in "What to expect" would incorrectly imply that number is the "correct" result you should get, when the only valid check is really that the value exists (isn't None) and is of the right type (int). type(snap_v1), on the other hand, is reproducible — it's always going to be int, no matter when you run the script — so it is shown as a literal value.

Exercise 3 — Prediction: what would happen if you ran load_v1.py a second time, without having created a new table? Without running it yet, predict: if you run step 2 of this lesson twice in a row against the same kiosko.dim_product table, how many rows do you expect to see in table.scan().to_arrow() after the second run? Justify your answer with what you already know about table.append() from module 1.

See solution

Eight rows: two copies of each of the four products. table.append() adds rows, it never replaces existing content — exactly the same behavior you already saw in module 1 when running load_into_iceberg.py twice against kiosko.fact_orders gave 80 rows instead of 40. For a dimension table like dim_product, where the correct grain is "one row per product," running append() twice with the same data breaks that grain — it's exactly the mistake this lesson's "Common mistakes" section warns about, and the reason lesson 3 uses table.overwrite() instead of a second append() for the P002 change.

Summary and next step

In this lesson you created kiosko.dim_product, with four columns and none for history, and loaded it for the first time with the V1 values — P002 still snacks/0.60. You confirmed that first write created exactly one snapshot, captured it right away in the snap_v1 variable, and saw, in the Going deeper section, the precise distinction between which operations create a data snapshot and which don't.

Before moving on you should be able to: create an Iceberg dimension table with no history columns; load it with table.append(); and explain why capturing snap_v1 immediately after the write, instead of "later, when needed," is a discipline and not a formality.

kiosko.dim_product now has its first photo filed away. Lesson 3 triggers the real change: it overwrites the whole table with the V2 values, where P002 moves to category='health-snacks', unit_cost=0.68 — the same change you already solved, with two different techniques, in data-modeling and dbt.

Resources

• PyIceberg — API reference, table.append() and table.current_snapshot(), already used in module 1 and reused here against a new table. py.iceberg.apache.org/api. In English.
• Apache Iceberg — official documentation, "Table Spec," "Snapshots" section, the formal definition of what constitutes a data snapshot. iceberg.apache.org/spec. In English.
• data-modeling-for-analytics-guide DESIGN doc — source of DIM_PRODUCT's exact V1 values (P001-P004, with P002 at snacks/0.60) this lesson loads unchanged. src/guides/data-modeling-for-analytics-guide/DISENO.md. In Spanish.
• This guide's DESIGN doc — the "Snapshots and time travel" section (M3), source of the exact V1 → snap_v1 → V2 sequence this module runs. src/guides/lakehouse-and-iceberg-guide/DISENO.md. In Spanish.