Writing your first DataFrameModel

Description

This lesson writes, for the first time in this guide, a real Pandera schema: a Python class with a name and a type per column, run against the S04 DataFrame lesson 4 left ready. The result is going to be surprising, and that surprise is exactly this lesson's point: a schema that only declares types, with no additional rule, lets S04's entire file through with no warning at all — the same deceptive green checkmark module 1 named, now reproduced with the new tool.

Connection to the module. This lesson builds the framework lessons 6 and 7 are going to fill in with real rules — nullable=False, unique=True, gt=0. Without first seeing how little a types-only schema does, the value of those specific rules would be much harder to appreciate.

An analogy: the guest list with only names, no conditions at all

This module's lesson 1 compared a guard who remembers rules from memory against a fixed sign on the door with written entry conditions. This lesson adds an important nuance to that analogy: imagine a sign on the door that simply says, "Guest list: first and last name." It says nothing about minimum age, or whether an invitation is required, or whether each name can only appear once on the list. It's, technically, a sign — it satisfies the form of a written rule —, but anyone who writes their first and last name on a piece of paper gets in, with no other condition. An event with only that sign could end up with the same person signed in three times, or with a minor inside, and the sign wouldn't have failed in any sense — it simply never promised to check those things.

A Pandera DataFrameModel that only declares types — order_id: str, quantity: int, with no additional Field — is exactly that minimal sign. It confirms each column has the correct data type — a string, an integer — but says nothing about whether those values repeat, whether they can be empty, or whether they respect any range. This lesson builds that minimal sign first, on purpose, so the difference against lessons 6 and 7's real rules is completely clear.

Worked example: the simplest possible class, first clean, then against S04

Step 1 — the class syntax, compared to something you already know

OrdersSchema gets declared as a Python class, with syntax already familiar to you from kiosko.py: the Order class, a @dataclass with one type per attribute. A Pandera DataFrameModel uses exactly the same idea — type annotations per attribute —, but instead of describing a single order, it describes an entire DataFrame column:

# first_schema.py
import pandera.polars as pa
import polars as pl

class OrdersSchemaMinimal(pa.DataFrameModel):
    order_id: str
    quantity: int

Two lines inside the class, each with the same shape you already saw in Order: column_name: type. The underlying difference is that Order describes a single Python row, while OrdersSchemaMinimal describes an entire DataFrame column — every order_id value, across S04's twelve rows, has to be a str; every quantity value, an int.

Step 2 — test it against clean data, by hand

Before touching S04's real file, confirm the class works with a minimal, hand-built example:

# first_schema.py -- continued
clean_rows = pl.DataFrame({
    "order_id": ["ORD-T1", "ORD-T2"],
    "quantity": [3, 1],
})

validated = OrdersSchemaMinimal.validate(clean_rows)
print(f"Validated rows: {validated.height}")
print(validated)

What to expect.

Validated rows: 2
shape: (2, 2)
┌──────────┬──────────┐
│ order_id ┆ quantity │
│ ---      ┆ ---      │
│ str      ┆ i64      │
╞══════════╪══════════╡
│ ORD-T1   ┆ 3        │
│ ORD-T2   ┆ 1        │
└──────────┴──────────┘

OrdersSchemaMinimal.validate() threw no exception, and returned the same DataFrame it received — two rows, unchanged. This is Pandera's baseline behavior when everything passes: validate() returns the validated DataFrame, so you can keep chaining work on top of it, instead of just returning True.

Step 3 — the same class, now against S04's real file

With kiosko.duckdb and the orders_s04 table lesson 4 built:

# first_schema.py -- continued
import duckdb

con = duckdb.connect("kiosko.duckdb")
df = con.sql("SELECT * FROM orders_s04").pl()

validated_s04 = OrdersSchemaMinimal.validate(df)
print(f"\nValidated S04 rows: {validated_s04.height}")
print("Pandera found no problem.")

What to expect.

Validated S04 rows: 12
Pandera found no problem.

Stop on this result, because it's exactly the warning that opens this lesson. All twelve rows of orders_2026-08-14.csv pass, with no exception at all — including ORD-9503, with an empty unit_price (this class doesn't even declare that column); including ORD-9507, with quantity=-1 (-1 is, without any doubt, a valid int); including ORD-9502's second appearance (nothing in this class asks whether an order_id repeats). OrdersSchemaMinimal fulfilled exactly what it declared — every order_id is a str, every quantity is an int — and nothing more. It's, with Pandera involved, the same lie of the green checkmark module 1's lesson 2 named: a tool honestly answering the question it was asked, when the question that needed asking was a different one.

Diagram: what a types-only schema asks, and what it doesn't

flowchart TD
    A["OrdersSchemaMinimal:\norder_id: str\nquantity: int"] --> B{"For each row:\nis it str? is it int?"}
    B -->|"ORD-9503, empty unit_price"| C["PASSES\n(unit_price isn't even declared)"]
    B -->|"ORD-9507, quantity=-1"| D["PASSES\n(-1 IS a valid int)"]
    B -->|"ORD-9502 x2"| E["PASSES\n(nothing checks uniqueness)"]
    B -->|"the 6 genuinely clean rows"| F["PASSES"]

    C --> G["Result: 12/12,\nno exception at all"]
    D --> G
    E --> G
    F --> G

The diagram shows the four problem rows from S04's incident — completeness, validity, both appearances of the duplicate pair — all converging on the same result: PASSES. Not because Pandera has any bug, but because OrdersSchemaMinimal, as written in this lesson, never asked them the right question.

Going deeper: two defaults worth knowing by heart

Before moving on to lesson 6, it's worth fixing two Pandera default behaviors this lesson already used, without naming them yet:

nullable is False by default — but it only protects a column that's declared. A subtle detail: if you had declared unit_price: float (with no Field) in OrdersSchemaMinimal, that column WOULD have failed against S04, because Pandera's default for nullable is False — any declared column rejects nulls unless you explicitly say otherwise with Field(nullable=True). OrdersSchemaMinimal's problem in this lesson isn't that nullable is misconfigured — it's that unit_price doesn't even appear in the class. A column that isn't declared doesn't get checked at all; it doesn't matter how strict Pandera's defaults are for the columns you did declare.

unique is False by default, and you have to ask for it explicitly. Unlike nullable, which protects by default, uniqueness is never assumed — you have to write Field(unique=True) on purpose for Pandera to check it. This makes sense: most columns in a real DataFrame aren't unique (store_id repeats across every row from the same store, for instance), so requiring uniqueness by default would break most real schemas for no reason. order_id, on the other hand, does need that condition — and lesson 6 adds it explicitly.

And a third piece, which you already used without knowing it: when you ran OrdersSchemaMinimal.validate(df) against S04, the result kept the DataFrame's six original columns (order_id, store_id, product_id, quantity, unit_price, order_ts), not just the two you declared. This is because, by default, Pandera isn't "strict" about extra columns — strict=False is the default —: a DataFrame can have columns the schema doesn't mention, and Pandera lets them through without comment. If you wanted to reject any undeclared column, you'd have to add class Config: strict = True inside the class — something this guide doesn't need, because every lesson from module 2 onward builds OrdersSchema bit by bit, adding new columns without meaning to say "this is everything that exists."

Common mistakes

Thinking that declaring the correct type already "validates" the column. What happens: someone writes unit_price: float in a schema, with no Field, and assumes that already covers completeness because "nulls aren't floats." Why it happens: it seems reasonable that None wouldn't fit into the float type. How to spot it: revisit this lesson's exact result — unit_price with an empty value in the CSV became, in lesson 4, a NULL of type DOUBLE/Float64 — Polars represents that null as a valid value inside a Float64 column, not as a type error. How to fix it: nullable=False is a separate condition from the type, not an automatic consequence of it — lesson 6 adds it explicitly for completeness.

Being surprised that Pandera "said nothing" about S04. What happens: someone runs OrdersSchemaMinimal.validate(df) against S04, sees Validated S04 rows: 12 with no error, and concludes Pandera "doesn't work" or "has a bug," because they already know — from module 1 — that the file has broken rows. Why it happens: after three whole lessons talking about Pandera as the tool that solves module 1's problem, a clean result feels contradictory. How to spot it: if your reaction is "this shouldn't happen," re-read exactly what OrdersSchemaMinimal declares — two columns, type only, no business rule at all. How to fix it: Pandera isn't failing here — it's doing exactly what it was asked. The "problem" isn't the tool's, it's the still-incomplete schema's; lessons 6 and 7 complete it.

Writing all the rules at once, without first seeing the minimal schema work. What happens: someone, in a hurry to reach the final result, copies lesson 7's complete OrdersSchema directly — with unique, nullable, ge, gt — and skips this lesson entirely. Why it happens: it seems more efficient to write the final version in one go. How to spot it: if you never saw, with your own eyes, a types-only schema pass S04 with no warning, you're missing this lesson's central pedagogical piece — the exact reason each Field in the following lessons exists. How to fix it: build the schema in the same order this guide does — types first, rules after — even if you already know the final result; that order is what lets you see, with evidence, what each new piece adds.

Exercises

Exercise 1 — Add store_id: str to OrdersSchemaMinimal and confirm the result doesn't change. Extend the class with a third column, store_id: str (with no Field), and run the validation against S04 again. Does the number of validated rows change?

class OrdersSchemaV2(pa.DataFrameModel):
    order_id: str
    quantity: int
    store_id: str

validated_v2 = OrdersSchemaV2.validate(df)
print(f"Validated rows: {validated_v2.height}")

Validated rows: 12

Exercise 2 — Predict what would happen if you declared unit_price: float with no Field, without running it yet. Based on this lesson's Going deeper section, before executing anything, predict: if you add unit_price: float (with no Field) to OrdersSchemaMinimal and run it against S04, does it pass or fail? Why?

Exercise 3 — Verify Exercise 2 by actually running the code. Run Exercise 2's schema against S04 and confirm your prediction with the exact error message.

class OrdersSchemaV3(pa.DataFrameModel):
    order_id: str
    quantity: int
    unit_price: float

try:
    OrdersSchemaV3.validate(df)
except pa.errors.SchemaError as exc:
    print(f"SchemaError: {exc}")

SchemaError: non-nullable column 'unit_price' contains null values

Summary and next step

In this lesson you wrote your first Pandera DataFrameModel, with the same type-annotation syntax you already knew from kiosko.py. You tested it first against hand-built clean data, then against S04's real file — and confirmed, with executed evidence, that a types-only schema lets all twelve rows through with no warning at all, even though you already know six of them have real problems. You also fixed two of Pandera's default behaviors — nullable=False, unique=False — that the following lessons are going to use constantly.

Before moving on you should be able to: explain why OrdersSchemaMinimal passes S04 with no errors, even though the file has broken rows; name this lesson's two defaults (nullable, unique) and say which one protects without being asked and which doesn't; and predict what would happen if you declared a column with an incorrect type for some real value.

You have the framework. Lesson 6 fills OrdersSchema in with the first two real rules — nullable=False for completeness, unique=True for uniqueness — and confirms, with executed evidence, that this time they do catch something.

Resources

• Pandera — official documentation, DataFrameModel section (type annotation syntax, Field defaults). pandera.readthedocs.io. In English.
• data-engineering-foundations-guide, module 4 — the source of the Order class (@dataclass), the type annotation syntax this lesson compares against DataFrameModel. src/guides/data-engineering-foundations-guide/workbook/module-04-modeling-your-first-tables/es/. In Spanish.
• This guide's DESIGN — the exact OrdersSchema syntax this module builds bit by bit. src/guides/data-reliability-and-governance-guide/DISENO.md. In Spanish.