Module 4: Data Contracts As Versioned Artifacts
Writing orders_contract.yaml
Description
This lesson builds, section by section, the real file the rest of the module uses without re-explaining it: contracts/orders_contract.yaml. Every field has a concrete reason behind it — it isn't a generic template copied from somewhere, it's the direct translation of the rules S04 already demonstrated it needed in modules 1 and 2, plus the informal SLA module 1's lesson 5 left pending.
Connection to the module. Lesson 2 defined a contract's three components. This lesson writes them, one by one, as real YAML. Lesson 4 parses this same file with pydantic, and lesson 5 turns it back into an executable schema.
Before writing: where does each value come from?
No field in this file gets invented in this lesson. Each one already exists, in another form, somewhere earlier in this guide:
| Contract field | Where it comes from |
|---|---|
order_id: unique, not null | OrdersSchema.order_id, module 2 |
unit_price: not null, >= 0 | OrdersSchema.unit_price, module 2 |
quantity: not null, > 0 | OrdersSchema.quantity, module 2 |
sla.freshness_hours: 24 | The informal agreement from module 1's lesson 5 |
sla.row_count: between 5 and 20 | The expected size of a new store's first file (12 real rows, with margin) |
on_violation: quarantine | A preview of module 7 — quarantine, not total rejection like foundations M7 |
Writing a contract, in this sense, is never an exercise in imagination — it's the work of distilling, into one place, rules already proven necessary with real evidence.
Worked example: the file, section by section
Section 1 — metadata: who, what, and at what version
contract_version: "1.0.0"
dataset: orders_s04
owner: kiosko-data-team
description: >
Data contract for the sales files sent by any Kiosko store,
including S04. Declares the minimum schema, the delivery SLA,
and what to do if a file violates it.contract_version follows semantic versioning format (MAJOR.MINOR.PATCH) — this module's lesson 6 precisely explains which kind of contract change corresponds to each number. dataset unambiguously identifies which table this contract applies to — the same name, orders_s04, modules 2 and 3 already use for the kiosko.duckdb table. owner declares who at Kiosko is responsible for this contract — information no Pandera schema needs, but that any real data governance system does need, to know who to notify if something fails. description, with the YAML > operator (which joins several lines into a single paragraph, replacing line breaks with spaces), is the prose explanation someone with no access to the code can read.
Section 2 — schema: the direct translation of OrdersSchema
schema:
- name: order_id
type: string
nullable: false
unique: true
- name: unit_price
type: float
nullable: false
minimum: 0
- name: quantity
type: integer
nullable: false
exclusive_minimum: 0Compare this, field by field, against module 2's OrdersSchema: order_id: str = pa.Field(unique=True) becomes type: string, unique: true; unit_price: float = pa.Field(nullable=False, ge=0) becomes type: float, nullable: false, minimum: 0; quantity: int = pa.Field(gt=0) becomes type: integer, exclusive_minimum: 0. The distinction between minimum (equivalent to ge, "greater than or equal to") and exclusive_minimum (equivalent to gt, "strictly greater than") matters, and this lesson keeps it on purpose: unit_price=0 would be valid (a free product, in theory), but quantity=0 makes no business sense — nobody buys zero units of anything.
Section 3 — SLA: punctuality and volume
sla:
freshness_hours: 24
row_count:
min: 5
max: 20freshness_hours: 24 is, literally, the number module 1's lesson 5 mentioned in prose and never wrote anywhere executable — now it is. row_count.min: 5 and row_count.max: 20 declare the row range Kiosko considers reasonable for a new store's first day: neither suspiciously few (fewer than 5 would suggest a truncated file), nor suspiciously many (more than 20 would suggest a whole-file duplicate retransmission, not just a single row like ORD-9502). S04's real file has 12 rows — within range, with plenty of margin in both directions.
Section 4 — violation policy
on_violation: quarantineA single value, but with a decision behind it: quarantine, not reject. Foundations M7 already showed, with evidence (rows_loaded=0, the entire file rejected), what happens when the policy is "all or nothing" — even ORD-9501, S04's file's first row and perfectly valid, would have been lost along with the broken ones. quarantine declares Kiosko's intent to never repeat that: separate the good from the bad, not discard the entire file over six rows.
What to expect. Save all four sections together as contracts/orders_contract.yaml, and confirm it by reading the raw file, with no parsing yet beyond plain YAML:
# preview_contract.py
import pprint
import yaml
with open("orders_contract.yaml") as f:
raw = yaml.safe_load(f)
print(f"Type of 'raw': {type(raw)}")
print(f"Top-level keys: {list(raw.keys())}\n")
pprint.pprint(raw)Running python3 preview_contract.py, the output is exactly this:
Type of 'raw': <class 'dict'>
Top-level keys: ['contract_version', 'dataset', 'owner', 'description', 'schema', 'sla', 'on_violation']
{'contract_version': '1.0.0',
'dataset': 'orders_s04',
'description': 'Data contract for the sales files sent by any Kiosko '
'store, including S04. Declares the minimum schema, the '
'delivery SLA, and what to do if a file violates it.\n',
'on_violation': 'quarantine',
'owner': 'kiosko-data-team',
'schema': [{'name': 'order_id',
'nullable': False,
'type': 'string',
'unique': True},
{'minimum': 0,
'name': 'unit_price',
'nullable': False,
'type': 'float'},
{'exclusive_minimum': 0,
'name': 'quantity',
'nullable': False,
'type': 'integer'}],
'sla': {'freshness_hours': 24, 'row_count': {'max': 20, 'min': 5}}}
yaml.safe_load() — PyYAML's recommended function over plain yaml.load(), because it doesn't execute any arbitrary Python type embedded in the YAML, only safe data types (dict, list, str, int, float, bool) — converts the entire file into a nested Python dictionary, ready to inspect. Notice something important: at this point, raw is a dict with no guaranteed structure at all — if someone accidentally deleted the sla key, this same script would still run with no error, because yaml.safe_load() knows nothing about which fields are required. That guarantee is, precisely, what lesson 4 adds with pydantic.
Diagram: the complete file, from the reason to the YAML line
flowchart TD
A["OrdersSchema (M2):\norder_id, unit_price, quantity"] --> D["schema: [...]"]
B["Informal SLA (M1L5):\n24 hours"] --> E["sla:\n freshness_hours: 24"]
C["The foundations M7 lesson:\ntotal rejection = bad"] --> F["on_violation: quarantine"]
D --> G["orders_contract.yaml"]
E --> G
F --> G
Common mistakes
Confusing minimum with exclusive_minimum. What happens: someone writes minimum: 0 for quantity, instead of exclusive_minimum: 0, not realizing that would allow quantity=0 as valid. Why it happens: the two names look similar, and the difference — inclusive versus exclusive — is subtle if you don't think carefully about each concrete case. How to spot it: ask yourself, for every numeric column, whether the boundary value itself (0 in this case) makes business sense. unit_price=0 could, in theory, represent a free promotional product — it makes sense to include it. quantity=0 doesn't represent any real sale — it makes no sense to include it. How to fix it: review every numeric bound in the contract, explicitly asking "does the boundary value itself count as valid?", and choose minimum (inclusive) or exclusive_minimum (exclusive) accordingly — never out of habit or by copying the previous field without thinking.
Using yaml.load() instead of yaml.safe_load(). What happens: someone, copying an old example from the internet, uses yaml.load(f) with no Loader argument, or with Loader=yaml.Loader (the full loader, not the safe one). Why it happens: in older PyYAML versions, yaml.load() with no specified loader worked with no warning, so old examples still circulate with that form. How to spot it: modern PyYAML throws an explicit Warning if you use yaml.load() with no loader; and, more importantly, yaml.Loader (unlike yaml.SafeLoader, which is what safe_load() uses) can execute arbitrary Python code if the YAML file contains it — a real security risk if the file comes from a source you don't fully control. How to fix it: always use yaml.safe_load() to read any config file or contract, no exceptions — it's the function this lesson uses, and the only correct one within this guide.
Forgetting the > operator in description and ending up with a paragraph full of literal line breaks. What happens: someone writes a multi-line description with no > operator at the start, and when the YAML gets parsed, description ends up being a string with literal \n characters inside, instead of a continuous paragraph. Why it happens: YAML has several ways of writing multi-line text (|, >, single quotes with continuation), and without knowing them ahead of time it's easy to write the block the "obvious" way that doesn't produce the expected result. How to spot it: print repr(contract["description"]) — if you see several \ns in the middle of the text instead of a clean paragraph, the operator used wasn't the right one for your case. How to fix it: > (folded block scalar) joins consecutive lines with a space, ideal for prose meant to read as a paragraph — exactly what this lesson uses; | (literal block scalar) preserves line breaks exactly as written, useful for text where formatting matters (like a sample error message), but not for a prose description.
Exercises
Exercise 1 — Break the YAML on purpose with incorrect indentation, and observe the error. Change the indentation of unique: true in the schema's first column, removing two spaces (so it ends up at the same level as - name: order_id instead of nested). Run preview_contract.py again.
See solution
With broken indentation, yaml.safe_load() throws a syntax error, typically something like:
yaml.scanner.ScannerError: mapping values are not allowed here
(The exact message varies depending on how badly the indentation ends up after the change.) This confirms something important about YAML as a format: unlike JSON, which uses explicit braces and brackets to mark structure, YAML uses indentation as part of its syntax — a spacing mistake isn't a cosmetic detail, it's a real syntax error that prevents the file from being read at all. It's worth mentioning: this is exactly the kind of error this module's lesson 4, with pydantic, still can't prevent — pydantic validates the contract's semantic structure (does it have the correct fields, with the correct types?), but it can only act after yaml.safe_load() has already managed to read the file with no syntax errors.
Exercise 2 — Add an sla.max_null_percentage: 5 field and decide whether it makes sense in this contract. Without implementing it in any code yet (that would come in a future module), add that field to the YAML and, in 2-3 sentences, argue whether it should live inside sla or somewhere else in the contract.
See solution
sla:
freshness_hours: 24
row_count:
min: 5
max: 20
max_null_percentage: 5max_null_percentage doesn't fit perfectly inside sla, even though at first glance it might seem reasonable — an SLA, as this module defines it, describes the whole file's punctuality and volume, not the content quality of specific columns. An acceptable null percentage is, rather, a property of each column, much closer to the schema component (where nullable already lives) than to the sla component. This exercise has no single correct answer, but the right reasoning is noticing that a contract's three components have conceptual boundaries, and not every new field automatically fits into the first one that comes to mind.
Exercise 3 — Rewrite the complete schema section replacing lists with a dictionary indexed by column name. This contract's schema uses a list of objects (schema: [{name: ..., type: ...}, ...]). Rewrite it as a dictionary where the key is the column name (schema: {order_id: {type: ..., ...}, ...}), and in 1-2 sentences, argue which of the two forms you'd prefer for this contract.
See solution
schema:
order_id:
type: string
nullable: false
unique: true
unit_price:
type: float
nullable: false
minimum: 0
quantity:
type: integer
nullable: false
exclusive_minimum: 0Both forms are valid YAML and hold exactly the same information. The list form (the one this lesson uses) has a concrete advantage for this module: it preserves column order explicitly and predictably when iterating, and generalizes better if the contract ever needed to allow two columns with related rules but equal names in different contexts (uncommon, but possible). The dictionary form has the advantage of making access to a specific column by name more direct (schema["order_id"] instead of searching for it in a list). For this contract, with only three columns, the difference is mostly stylistic — lesson 4 builds ColumnContract to work with the list form, the one this guide chooses.
Summary and next step
In this lesson you built orders_contract.yaml, section by section, with the exact reason behind every field: the schema, translated directly from module 2's OrdersSchema; the SLA, the number module 1's lesson 5 left pending; and the violation policy, a conscious decision to never repeat foundations M7's total rejection. You confirmed, with yaml.safe_load() actually run, that the file reads with no error and produces exactly the expected dictionary structure.
Before moving on you should be able to: explain where each of the contract's six concrete values comes from (three column rules, two SLA numbers, one policy); and reproduce the complete file from scratch, section by section.
You have the file, read as a Python dictionary with no guarantee yet about its structure. Lesson 4 adds that guarantee: it parses this same file with pydantic, and demonstrates what happens when someone accidentally breaks it.
Resources
- PyYAML — official documentation (
yaml.safe_loadversusyaml.load, and the|/>block scalar syntax for multi-line text). pyyaml.org/wiki/PyYAMLDocumentation. In English. - YAML — the format's official specification (the complete syntax, including indentation as part of the grammar). yaml.org/spec/1.2.2. In English.
- Module 2, lesson 5 onward, of this same guide — the exact source of
OrdersSchema's three rules this lesson translates into YAML.src/guides/data-reliability-and-governance-guide/workbook/module-02-declarative-data-quality-tests-with-pandera/es/05-writing-your-first-dataframeschema.md. In Spanish. - Module 1, lesson 5, of this same guide — the exact source of the informal 24-hour SLA.
src/guides/data-reliability-and-governance-guide/workbook/module-01-when-green-does-not-mean-correct/es/05-meet-s04-kioskos-fourth-store.md. In Spanish. - This guide's DESIGN.
src/guides/data-reliability-and-governance-guide/DISENO.md. In Spanish.