Installing PyIceberg and a local catalog

Description

This lesson leaves theory behind for the rest of the module. It installs PyIceberg for real, on your own machine — no cloud account, no JVM, no Docker — and creates Kiosko's first local catalog: the album index the three previous lessons talked about, now as real code running in your terminal.

Connection to the module. Lessons 1 through 3 built the why: four times a loose Parquet wasn't enough, and the precise distinction between file format and table format. This lesson installs the first concrete piece — the catalog — on top of which lessons 5 and 6 are going to build the namespace and the table. Without a catalog, there's no place a table can register itself as "the current one."

An analogy: hiring the librarian before organizing the first album

Picking up the analogy from the three previous lessons: if Iceberg is the album with an index, and lessons 5 and 6 are going to fill that album with the first photo collection, this lesson installs whoever maintains the index — the librarian who, from this moment on, will always know which version of every album in this library is current. Before there's even a single album, it makes sense that the person (or, in this case, the system) that's going to keep the record exists first. That's, precisely, what an Iceberg catalog is: it doesn't contain a single row of data itself — it contains only a record of which tables exist and where their current metadata is.

Worked example: real install, real catalog

Step 1 — Install PyIceberg with the necessary extras

pip install "pyiceberg[sql-sqlite,pyarrow]"

Notice the two extras in brackets, because neither is optional for what this guide needs: sql-sqlite installs support for a SQLite-backed catalog — the catalog engine you're going to use for almost this entire guide — and pyarrow installs the engine PyIceberg uses to read and write the Parquet data files, which you already know from the previous guides in the ecosystem.

What to expect (verified by running the actual command, in a clean virtual environment; the full list of transitive dependencies is omitted for space — the final summary is shown):

Collecting pyiceberg[pyarrow,sql-sqlite]
  ...
Successfully installed annotated-types-0.8.0 cachetools-6.2.6 certifi-2026.7.22
charset_normalizer-3.5.0 click-8.4.2 fsspec-2026.7.0 idna-3.18 markdown-it-py-4.2.0
mdurl-0.1.2 mmh3-5.2.1 pyarrow-25.0.1 pydantic-2.13.4 pydantic-core-2.46.4
pygments-2.20.0 pyiceberg-0.11.1 pyiceberg-core-0.7.0 pyparsing-3.3.2 pyroaring-1.1.0
python-dateutil-2.9.0.post0 requests-2.34.2 rich-14.3.4 six-1.17.0 sqlalchemy-2.0.52
strictyaml-1.7.3 tenacity-9.1.4 typing-extensions-4.16.0 typing-inspection-0.4.4
urllib3-2.7.0 zstandard-0.25.0

The version this command resolves, verified while writing this lesson, is PyIceberg 0.11.1 — published March 3, 2026, requires Python >=3.10,<4.0, Apache-2.0 license — with pyarrow 25.0.1 and sqlalchemy 2.0.52 as the two dependencies that make it possible, respectively, to read/write Parquet and talk to the SQLite catalog. All the code and all the output in the rest of this guide is verified against this exact version.

Confirm the install:

python3 -c "import pyiceberg; print('pyiceberg', pyiceberg.__version__)"

What to expect (verified by running the actual command):

pyiceberg 0.11.1

Step 2 — Load your first local catalog

With PyIceberg installed, load_catalog() is the library's central function: you tell it what type of catalog you want, where its registry database lives, and where in the filesystem it should write the data files for any new table.

# create_catalog.py
import os

from pyiceberg.catalog import load_catalog

warehouse_path = os.path.abspath("kiosko_warehouse")
catalog_db_path = os.path.abspath("kiosko_catalog.db")
os.makedirs(warehouse_path, exist_ok=True)

catalog = load_catalog(
    "kiosko",
    type="sql",
    uri=f"sqlite:///{catalog_db_path}",
    warehouse=f"file://{warehouse_path}",
)

print("Catalog loaded:", catalog.name)
print("Type:", type(catalog).__name__)
print("Existing namespaces:", catalog.list_namespaces())

What to expect (verified by running the actual script, in an empty directory):

Catalog loaded: kiosko
Type: SqlCatalog
Existing namespaces: []

catalog.list_namespaces() returns an empty list — that makes sense, because this catalog was just created, and you haven't yet asked it to register any namespace (that's, precisely, lesson 5's job). But notice what's already happened on disk, without you having created a single table yet:

ls -la kiosko_catalog.db kiosko_warehouse/

What to expect:

-rw-r--r--  1 user  staff  20480 <date> kiosko_catalog.db

kiosko_warehouse:
total 0
drwxr-xr-x  2 user  staff  64 <date> .
drwxr-xr-x  N user  staff N*32 <date> ..

kiosko_catalog.db already exists, with 20 KB of internal SQLite tables PyIceberg created to keep its own registry — you're going to inspect exactly what that database stores in module 2; kiosko_warehouse/ exists as a directory, but it's completely empty — there's still no data or metadata file inside it, because no table exists. The catalog is ready; the album still doesn't have a single page.

Diagram: what just got installed

flowchart LR
    A["pip install\npyiceberg[sql-sqlite,pyarrow]"] --> B["load_catalog('kiosko', type='sql', ...)"]
    B --> C["kiosko_catalog.db\n(SQLite -- registry of namespaces and tables)"]
    B --> D["kiosko_warehouse/\n(filesystem -- empty, no tables yet)"]
    C -.->|"lesson 5"| E["catalog.create_namespace('kiosko')"]
    D -.->|"lesson 5-6"| F["catalog.create_table(...)\ntable.append(...)"]

Going deeper: why a SQL catalog, and not another option

PyIceberg supports several catalog types — sql (the one you just used, backed by SQLite, Postgres, or MySQL), rest (the standard protocol spoken by managed catalogs like AWS Glue Catalog or Polaris, named without being implemented in module 7), hadoop (based purely on the filesystem, with no database at all). This guide chooses sql with SQLite specifically because it satisfies, with no friction, three conditions the rest of the guide needs: it runs completely locally, with no separate server to spin up; it requires no account or credential; and it supports optimistic concurrency control — the guarantee that two simultaneous writes to the same table can't corrupt it — with the same transactional guarantees as any real SQL database, something a purely file-based catalog (hadoop) can't offer with the same solidity. Module 7 of this guide names the production catalogs — REST, AWS Glue Catalog, Unity Catalog, Polaris — and explains which guarantee each one solves, without implementing them: this whole guide uses, from start to finish, 100% local catalogs.

Common mistakes

Installing pyiceberg without the extras, and discovering the error only when creating a table. What happens: someone runs pip install pyiceberg plain, without [sql-sqlite,pyarrow], and import pyiceberg works with no error at all — the problem only shows up in lesson 5, when trying load_catalog(type="sql", ...), with an import error about a missing SQLite module. Why it happens: PyIceberg is deliberately designed to be modular — not everyone needs SQLite or pyarrow — so the base package installs without failing, even though it's missing pieces you're going to need later. How to spot it: if load_catalog(type="sql", ...) fails with a ModuleNotFoundError mentioning sqlalchemy or something SQL-related, check how you installed PyIceberg. How to fix it: reinstall with this lesson's exact extras: pip install "pyiceberg[sql-sqlite,pyarrow]" — the double quotes aren't decorative, they keep the shell from interpreting the brackets as a file-expansion pattern.

Using a relative path for warehouse, and having tables "disappear" when the working directory changes. What happens: someone writes warehouse="file://./kiosko_warehouse" (a relative path) instead of an absolute one, and in a later run, from a different folder, PyIceberg reports that the kiosko.fact_orders table doesn't exist. Why it happens: a relative path is resolved against the working directory at the moment the script runs — if you run the script once from ~/kiosko/ and again from ~/kiosko/scripts/, the relative path points to two physically different places on disk, even though the catalog says the same thing. How to spot it: if catalog.list_tables("kiosko") returns an empty list after you already created tables in an earlier run, suspect an inconsistent relative path first, before assuming something got corrupted. How to fix it: this lesson's worked example uses os.path.abspath() exactly to prevent this error — always resolve warehouse_path and catalog_db_path to absolute paths before passing them to load_catalog(), no matter which directory you run the script from.

Forgetting to create the warehouse directory before loading the catalog. What happens: someone runs load_catalog() pointing to a warehouse_path that doesn't yet exist as a folder on disk, and the catalog gets created with no error — PyIceberg doesn't validate this upfront — but the first real write (in lesson 6) fails with a filesystem error. How to spot it: if table.append() fails with an error related to a path that doesn't exist, check whether the warehouse_path directory was actually created before loading the catalog. How to fix it: this lesson's worked example includes os.makedirs(warehouse_path, exist_ok=True) before load_catalog(), exactly to guarantee the directory exists from the start — a pattern worth keeping in any new script for the rest of this guide.

Exercises

Exercise 1 — Reproduce the full install yourself. On your own machine, with Python >=3.10 available, run pip install "pyiceberg[sql-sqlite,pyarrow]", confirm the version with python3 -c "import pyiceberg; print(pyiceberg.__version__)", and then run this lesson's create_catalog.py script. Confirm you see Existing namespaces: [] and that kiosko_warehouse/ shows up as an empty directory.

See solution

If your Python version satisfies >=3.10,<4.0, you should see a successful install with pyiceberg-0.11.1 (or a newer version if you install this later in time — PyIceberg publishes releases regularly) in the Successfully installed line, and the script should print exactly Catalog loaded: kiosko, Type: SqlCatalog, Existing namespaces: []. If import pyiceberg fails, first check your Python version with python3 --version — it's the most common cause of a failure at this step.

Exercise 2 — Prediction: what does kiosko_catalog.db contain at this point? Without opening the file yet — module 2 does that in depth — predict: if you inspected kiosko_catalog.db with any SQLite client after running only this lesson's create_catalog.py (without having created any namespace or table yet), would you expect to find tables with Kiosko data inside? Justify your answer with what you learned about what a catalog is.

See solution

No — at this point kiosko_catalog.db contains only the internal tables PyIceberg needs to function as a catalog (the SqlCatalog's own schema, designed to register namespaces and pointers to tables), but no row of Kiosko's business data yet, because this lesson's worked example stopped right before creating the kiosko namespace (catalog.list_namespaces() returned an empty list). The catalog, at this point, is exactly like a freshly printed index for an album that doesn't exist yet: the structure is ready, but there's no real entry pointing to anything.

Exercise 3 — Explain the difference between catalog_db_path and warehouse_path. In 2-3 sentences, explain what each of the two paths in this lesson's worked example stores, and why they're two different things instead of one.

See solution

catalog_db_path is the location of the SQLite database that acts as the catalog: the registry of which namespaces and tables exist, and which metadata file each table points to right now — it's, in this lesson's analogy, the librarian and their card catalog. warehouse_path is the location in the filesystem where each table's actual files live — both the Parquet data files and the JSON/Avro metadata files module 2 is going to inspect — it's, in the same analogy, the physical shelf where the albums themselves live. They're two different things because they play different roles: the catalog knows what exists and where to find it; the warehouse contains what exists. Nothing stops a real deployment from having one live in a cloud-managed database and the other in object storage like S3 — this guide keeps both local, for simplicity and zero cost.

Summary and next step

In this lesson you installed PyIceberg for real — version 0.11.1 at the time this guide was written, with the sql-sqlite and pyarrow extras — and created your first local catalog, kiosko, backed by SQLite, with an empty warehouse in the filesystem. You confirmed, with catalog.list_namespaces() returning an empty list, that the catalog exists but doesn't yet have any registered namespace.

Before moving on you should be able to: explain the difference between catalog_db_path and warehouse_path; reproduce the install and the catalog load on your own machine; and name why this guide chose a sql/SQLite catalog instead of hadoop or rest.

The catalog exists, but it's empty. Lesson 5 creates the first namespace — kiosko — and the first table — kiosko.fact_orders — with an explicit schema, inside that catalog.

Resources

• PyIceberg — official documentation (quickstart), the exact install with extras and setting up a local SqlCatalog with SQLite. py.iceberg.apache.org. In English.
• PyIceberg — PyPI, current version 0.11.1, published March 3, 2026, Python requirements and Apache-2.0 license. pypi.org/project/pyiceberg. In English.
• PyIceberg — API reference, load_catalog() and the supported catalog types (sql, rest, hadoop, glue). py.iceberg.apache.org/api. In English.
• This guide's DESIGN doc — the "What actually runs / gets verified" section, source of the exact SqlCatalog configuration this lesson uses. src/guides/lakehouse-and-iceberg-guide/DISENO.md. In Spanish.