Creating your first namespace and table

Description

With the catalog loaded in lesson 4, this lesson creates the two pieces missing for Kiosko's first Iceberg table to truly exist: the kiosko namespace — the logical grouping, equivalent to a database or a schema in SQL — and the kiosko.fact_orders table, with a schema declared explicitly, column by column. By the end of this lesson the table exists, with its own metadata file on disk — but still without a single row inside. Loading the actual data is, precisely, lesson 6's job.

Connection to the module. This is the lesson where the vocabulary from lessons 1 through 3 becomes executable code: the "album index" from the analogy is now a real namespace and table, registered in the catalog you installed in lesson 4.

An analogy: opening the empty album, with its index already printed

Think of the librarian from the previous lesson, already hired and ready. This lesson asks them for two concrete things: first, to open a new section on the shelf — the kiosko namespace, a place where every album for this business is going to live, separate from any other album this same library might hold; second, to reserve a new album within that section, with its cover already printed stating exactly which columns of information every photo added to it is going to have — order_id, store_id, product_id, quantity, unit_price, revenue, order_ts. The album exists, has a cover, has an index — but doesn't have a single photo pasted inside yet. That cover printed in advance, with the columns already declared, is exactly what an Iceberg explicit schema represents.

Worked example: namespace, schema, and table, in real code

Step 1 — Create the kiosko namespace

# create_namespace_and_table.py
import os

from pyiceberg.catalog import load_catalog
from pyiceberg.schema import Schema
from pyiceberg.types import DoubleType, IntegerType, NestedField, StringType, TimestampType

warehouse_path = os.path.abspath("kiosko_warehouse")
catalog_db_path = os.path.abspath("kiosko_catalog.db")

catalog = load_catalog(
    "kiosko",
    type="sql",
    uri=f"sqlite:///{catalog_db_path}",
    warehouse=f"file://{warehouse_path}",
)

catalog.create_namespace("kiosko")
print("Namespaces after create_namespace:", catalog.list_namespaces())

What to expect (verified by running the actual script):

Namespaces after create_namespace: [('kiosko', )]

A namespace is, precisely, a grouping — it doesn't contain a single row of data itself, it only organizes which tables belong together. In a real production catalog, a namespace usually maps to something like a database or a SQL schema; here, kiosko is going to hold, by the end of this guide, five tables: fact_orders, dim_store, dim_product, dim_date, and fact_orders_at_scale.

Step 2 — Declare the schema, column by column

PyIceberg needs an explicit Schema to create a table — it doesn't infer one on its own, although in lesson 6 you're going to see that it can verify that a pyarrow.Table matches this schema before loading it. Each column is declared as a NestedField, with its own field_id — an internal Iceberg numeric identifier, distinct from the column name, which module 4 of this guide is going to use to explain how schema evolution doesn't break old files:

fact_orders_schema = Schema(
    NestedField(field_id=1, name="order_id", field_type=StringType(), required=True),
    NestedField(field_id=2, name="store_id", field_type=StringType(), required=True),
    NestedField(field_id=3, name="product_id", field_type=StringType(), required=True),
    NestedField(field_id=4, name="quantity", field_type=IntegerType(), required=True),
    NestedField(field_id=5, name="unit_price", field_type=DoubleType(), required=True),
    NestedField(field_id=6, name="revenue", field_type=DoubleType(), required=True),
    NestedField(field_id=7, name="order_ts", field_type=TimestampType(), required=True),
)

Notice required=True on all seven columns: it's the explicit declaration that none of fact_orders's columns accept NULL — the same guarantee you already declared, without a second thought, as NOT NULL in data-modeling-for-analytics-guide's CREATE TABLE fact_orders in DuckDB. Iceberg's schema isn't a new idea — it's the same quality control as always, now expressed in this library's own syntax.

Step 3 — Create the table

table = catalog.create_table("kiosko.fact_orders", schema=fact_orders_schema)

print("Table created:", table.name())
print()
print(table.schema())
print()
print("Tables in the kiosko namespace:", catalog.list_tables("kiosko"))

What to expect (verified by running the actual script):

Table created: ('kiosko', 'fact_orders')

table {
  1: order_id: required string
  2: store_id: required string
  3: product_id: required string
  4: quantity: required int
  5: unit_price: required double
  6: revenue: required double
  7: order_ts: required timestamp
}

Tables in the kiosko namespace: [('kiosko', 'fact_orders')]

table.name() returns a two-part tuple — ('kiosko', 'fact_orders') — the same two-level identifier (namespace, table) you're going to use for the rest of this guide as the string "kiosko.fact_orders". table.schema() prints, in the same format you declared, the seven columns with their types and their requiredness — the confirmation that Iceberg registered exactly what you asked for, not one column more or less.

Step 4 — Confirm the table exists, but is empty

print("Current snapshot:", table.current_snapshot())
print("table.scan().to_arrow().num_rows =", table.scan().to_arrow().num_rows)

What to expect (verified by running the actual script):

Current snapshot: None
table.scan().to_arrow().num_rows = 0

This is exactly what the analogy predicted: the album exists, with its cover and its index already declared, but current_snapshot() returns None — there hasn't been a single write yet, so not one snapshot exists — and table.scan().to_arrow() confirms zero rows. The kiosko.fact_orders table is, at this exact moment, a real table, registered in the catalog, with a valid schema — and completely empty. Loading Kiosko's actual forty rows is, precisely, lesson 6's job.

Diagram: what exists on disk after this lesson

flowchart TB
    A["kiosko_catalog.db\n(SQLite)"] -->|"registers"| B["namespace: kiosko"]
    B -->|"contains"| C["table: kiosko.fact_orders\nschema declared, 0 rows"]
    C -->|"points to"| D["kiosko_warehouse/kiosko/fact_orders/\nmetadata/00000-....metadata.json"]
    D -.->|"no snapshots yet\n(lesson 6 creates the first one)"| E["( no data files )"]

On disk, after this lesson, exactly one file exists: kiosko_warehouse/kiosko/fact_orders/metadata/00000-<uuid>.metadata.json — the first metadata file, with the schema you just declared, but with no reference to a snapshot yet. Module 2 of this guide opens that file and explains, field by field, what it contains.

Going deeper: field_id, the identifier that survives a RENAME COLUMN

It's worth pausing on a detail that today looks like a technicality, but that module 4 of this guide turns into the central explanation for why schema evolution is safe: every NestedField has its own field_id, in addition to its name. A Parquet file, internally, doesn't store an Iceberg table's data indexed by the column's name — it stores it indexed by its field_id. This means that if in module 4 you rename country to nation (a hypothetical example, not part of this guide, but useful for intuition), Iceberg doesn't need to rewrite a single existing Parquet file: the field_id stays the same, only the label the catalog presents it with changes. This lesson doesn't go deeper into this — all of module 4 is dedicated to schema evolution — but it's worth knowing, the first time you see a field_id, in fact_orders's schema, that it isn't a decorative detail: it's, precisely, the piece that makes "add or rename a column without rewriting data" a real guarantee and not a marketing promise.

Common mistakes

Trying to create the table before creating the namespace. What happens: someone jumps straight to step 3 of this lesson, without having run catalog.create_namespace("kiosko") first, and catalog.create_table("kiosko.fact_orders", ...) fails with an error stating the namespace doesn't exist. Why it happens: it's easy to assume a two-part identifier like "kiosko.fact_orders" automatically creates both levels, the way some filesystems work with mkdir -p. How to spot it: if create_table() fails with an error mentioning NoSuchNamespaceError or similar, check whether the namespace already exists with catalog.list_namespaces(). How to fix it: always create the namespace before any table that's going to live inside it — the order of this lesson's worked example (namespace first, table second) isn't arbitrary.

Declaring required=False (or leaving it at the default) on columns that should actually be mandatory. What happens: someone copies this lesson's pattern but omits required=True on some column, without realizing NestedField's default is required=False (optional column, accepts NULL). Why it happens: it's easy to overlook a parameter with a default value, especially when the reference example declares it explicitly on all seven columns. How to spot it: if later on you manage to insert rows with NULL in a column that should be mandatory (for example, an empty order_id), check how you declared that column in the Schema. How to fix it: for fact_orders, all seven columns are mandatory by business design — an order with no order_id, no store, or no product doesn't make sense — so all seven must carry required=True, exactly as in the worked example.

Confusing table.current_snapshot() returning None with an error. What happens: someone, seeing Current snapshot: None after step 4, assumes something failed while creating the table. Why it happens: None is usually associated, by habit, with a missing value or an error, rather than with a valid, expected state. How to spot it: if the rest of the script (which did print the schema correctly in step 3) ran without any error, None in current_snapshot() isn't a failure — it's the correct state of any freshly created Iceberg table, before its first write. How to fix it: nothing to fix — current_snapshot() is None is, in fact, exactly the condition that confirms the table exists but is empty, exactly as this lesson's analogy predicted. Lesson 6 creates the first real snapshot.

Exercises

Exercise 1 — Reproduce the four parts yourself. With PyIceberg already installed (lesson 4), run the four parts of this lesson's worked example on your own machine. Confirm you see Namespaces after create_namespace: [('kiosko', )], the seven-column schema printed correctly, and Current snapshot: None.

See solution

If you followed the four steps exactly, your output should be identical to this lesson's — unlike a snapshot_id (which you're going to see for the first time in lesson 6), nothing in this lesson depends on when you run it, so your output should match byte for byte what's shown here. If create_table() fails, check this lesson's first common mistake first — the most frequent cause is not having created the namespace beforehand.

Exercise 2 — Explain why field_id isn't the same as the column's order. Without looking at this lesson's Going deeper section yet, form your own hypothesis: why do you think Iceberg assigns each column an explicit numeric field_id, instead of simply using its position (column 1, column 2, ...) within the schema?

See solution

If Iceberg used only a column's position to identify it within the Parquet files, any operation that changed the columns' order — or that inserted a new column in the middle of the schema, not at the end — would break the correspondence between what an old file has stored and what the current schema expects to find at each position. An explicit field_id, assigned once and never reused or reordered, solves that problem: no matter what position order_id appears in in some future version of the schema, every file — old and new — agrees that "the one with field_id=1" is always the order_id column. This lesson's Going deeper section confirms this intuition, and module 4 uses it to explain why add_column/rename_column/delete_column don't rewrite existing data.

Exercise 3 — Prediction: what would happen if you tried to create kiosko.fact_orders a second time? Without running it yet, predict: if you run catalog.create_table("kiosko.fact_orders", schema=fact_orders_schema) again right after this lesson, without having deleted anything, what do you expect to happen? Justify your answer by thinking about the fact that the catalog already registered that table.

See solution

It should fail, with an error stating the table already exists (TableAlreadyExistsError or similar) — the catalog, exactly as lesson 4's analogy described it, is precisely the registry that knows "this table already exists, with this current metadata," so asking it to create the same table again is an operation the catalog must reject to avoid ambiguity about which of the two versions would be "the current one." If you genuinely needed to recreate the table from scratch during this guide, you'd have to explicitly delete it first with catalog.drop_table("kiosko.fact_orders") — an operation this lesson doesn't use, because there's no need: the table gets created once, and the following lessons only add data to it.

Summary and next step

In this lesson you created the kiosko namespace and the kiosko.fact_orders table, with an explicit seven-column schema — each with its field_id, its type, and its requiredness — inside the local catalog you installed in lesson 4. You confirmed, with table.current_snapshot() is None and table.scan().to_arrow().num_rows == 0, that the table formally exists but doesn't yet have a single row.

Before moving on you should be able to: explain the difference between a namespace and a table; declare a PyIceberg Schema with NestedField, field_id, and correct types; and explain why field_id doesn't depend on a column's position or name.

The album has a cover and an index, but it's still empty. Lesson 6 finally loads Kiosko's actual forty rows for the week — reconstructed as fact_orders.parquet with pyarrow — into this table, with table.append().

Resources

• PyIceberg — API reference, Schema, NestedField, and the available types (StringType, IntegerType, DoubleType, TimestampType, among others). py.iceberg.apache.org/api. In English.
• Apache Iceberg — official documentation, table specification, the field-id section as a column's stable identifier. iceberg.apache.org/docs/latest. In English.
• data-modeling-for-analytics-guide DESIGN doc — source of the original CREATE TABLE fact_orders with its NOT NULL columns, which this lesson reproduces with required=True. src/guides/data-modeling-for-analytics-guide/DISENO.md. In Spanish.
• This guide's DESIGN doc — the full map of the eight modules, including the schema evolution section that revisits field_id in module 4. src/guides/lakehouse-and-iceberg-guide/DISENO.md. In Spanish.