What, precisely, a data contract is

Description

This module's lesson 1 showed the problem — quality rules scattered across Python code and loose prose — and promised a solution: an artifact that brings together schema + SLA + violation policy. This lesson stops on that definition and makes it precise, component by component, because the rest of the module uses it without re-explaining it. It isn't an abstract definition: each of the three components already has, in this guide, a concrete piece of Kiosko that represents it.

Connection to the module. This lesson gives the idea lesson 1 presented a name and an exact structure. Lesson 3 takes this same definition and writes it, section by section, as S04's real YAML file.

The three components, one by one

Schema. What shape the data should have: which columns exist, what type each one is, which can be null, which must be unique, and what value range is valid. You already built this, letter for letter, in module 2: OrdersSchema is S04's schema — unique order_id, non-null and non-negative unit_price, positive quantity. A contract doesn't reinvent this; it inherits it.

SLA (Service Level Agreement). How punctually and with what volume the data should arrive. It isn't a property of any individual row — no column in orders_s04 says "I arrived on time" —, it's a property of the whole file, compared against the clock: did it arrive within the agreed window? Is the row count within expectations, neither suspiciously few nor suspiciously many? Module 1's lesson 5 already mentioned S04's informal 24-hour SLA, but never wrote it anywhere executable. That is, precisely, the component it was missing.

Violation policy (on_violation). What happens when something in the schema or the SLA isn't met. This is the piece none of the previous three modules' tools needed to declare, because none of them yet acted on their own results — validate_orders() split rows into valid and rejected and that's where its job ended; OrdersSchema.validate() threw an exception and the script caught it to print a report. A contract goes one step further: it declares, ahead of time, what should happen — quarantine, reject the whole file, only alert without blocking anything. This guide's module 7 builds the real mechanism (quarantine()); this module only declares the intent.

All three together, and only all three together, form a complete contract. Any one of the three, on its own, is a useful but incomplete piece.

An analogy: the fixed menu a supplier signs with the restaurant

Go back to lesson 1's analogy. A fixed menu between a restaurant and its fish supplier has, precisely, the same three-part structure. Schema: which species get delivered, in what weight range, fresh or frozen — the product's exact shape. SLA: how often the order arrives, before what time of day, at what minimum and maximum quantity. Violation policy: if a batch arrives with weight out of range, does it get returned entirely? Accepted at a discount? Does the contract get canceled if it happens three times in a month? A document that only said "send me fresh fish" — with no other two parts — wouldn't protect the restaurant from anything: it wouldn't know when to expect the order, or what to do if it doesn't arrive one day.

Worked example: classifying three drafts — which one is a complete contract?

This lesson's code builds three versions of a possible specification for orders_s04, and automatically checks which of the three have all three components:

# contract_anatomy.py
import yaml

SCHEMA_KEYS = {"schema"}
SLA_KEYS = {"sla"}
POLICY_KEYS = {"on_violation"}


def classify_contract(doc: dict) -> dict:
    return {
        "has_schema": bool(SCHEMA_KEYS & doc.keys()),
        "has_sla": bool(SLA_KEYS & doc.keys()),
        "has_violation_policy": bool(POLICY_KEYS & doc.keys()),
    }


schema_only_doc = {"dataset": "orders_s04", "schema": [{"name": "order_id", "type": "string"}]}
schema_and_sla_doc = {**schema_only_doc, "sla": {"freshness_hours": 24}}

with open("orders_contract.yaml") as f:
    full_contract_doc = yaml.safe_load(f)

candidates = [
    ("Schema only (what OrdersSchema already was)", schema_only_doc),
    ("Schema + SLA, no violation policy", schema_and_sla_doc),
    ("Complete orders_contract.yaml", full_contract_doc),
]

for label, doc in candidates:
    result = classify_contract(doc)
    is_full_contract = all(result.values())
    verdict = "IS a complete data contract" if is_full_contract else "is NOT a complete data contract (missing at least one piece)"
    print(f"{label}:")
    print(f"  {result}")
    print(f"  -> {verdict}\n")

What to expect. Running python3 contract_anatomy.py (with lesson 3's orders_contract.yaml in the same folder — here the final file is used, even though this lesson doesn't build it step by step yet), the output is exactly this:

Schema only (what OrdersSchema already was):
  {'has_schema': True, 'has_sla': False, 'has_violation_policy': False}
  -> is NOT a complete data contract (missing at least one piece)

Schema + SLA, no violation policy:
  {'has_schema': True, 'has_sla': True, 'has_violation_policy': False}
  -> is NOT a complete data contract (missing at least one piece)

Complete orders_contract.yaml:
  {'has_schema': True, 'has_sla': True, 'has_violation_policy': True}
  -> IS a complete data contract

The first draft is, precisely, what you already had at the end of module 2: a schema, nothing more. OrdersSchema, as it's written today, would occupy exactly that row of the table — useful, executable, but not a contract. The second draft adds the SLA, and still lacks the third piece. Only the third — the real file lesson 3 builds — has all three at once, and that, and only that, is why classify_contract() marks it as a complete contract.

Diagram: the three questions a complete contract answers

flowchart LR
    A["What shape should\nthe data have?"] --> D["Schema\n(order_id, unit_price,\nquantity...)"]
    B["How punctually and\nwith what volume should\nit arrive?"] --> E["SLA\n(freshness_hours,\nrow_count)"]
    C["What happens if it\ndoesn't comply?"] --> F["Violation policy\n(on_violation)"]
    D --> G["Complete\ndata contract"]
    E --> G
    F --> G

Going deeper: a contract doesn't replace the schema, it includes it

A clarification is worth making that prevents a common mistake: a contract isn't an alternative to OrdersSchema — it's an artifact that includes the equivalent of OrdersSchema's schema, plus two things OrdersSchema never had by design. Pandera, as a tool, never set out to solve "how punctually should this file arrive?" or "what do we do if it doesn't comply?" — those questions are outside a declarative DataFrame schema's scope, no matter how many Fields you add to it. This distinction is the same one module 3 already drew about consistency: it isn't that Pandera is badly designed, it's that a single-table schema, by construction, can only answer questions about that table's shape — never about time or about consequences.

Going deeper: why the three components live in ONE file, not three

Someone could argue, with some logic, that it would be tidier to keep three separate files — orders_schema.yaml, orders_sla.yaml, orders_violation_policy.yaml — instead of bundling the three components into a single orders_contract.yaml. This guide deliberately chooses one file, and it's worth explaining why, because the reason isn't just aesthetic.

A contract describes, in the most literal sense, an agreement between two parties about the same dataset — the same way a real lease agreement doesn't get signed as three separate documents (one for the rooms, another for the price, another for the consequences of not paying), a data contract gains precision by declaring its three components together, with a single version covering all three at once. If the schema and the SLA lived in separate files, with independent version numbers, an awkward question would come up every time someone needed to cite "S04's contract": which schema version, combined with which SLA version? That version combinatorics — three files, each with its own history — is exactly the kind of silent desync this lesson already identified as the original problem a contract solves. A single file, with a single contract_version, eliminates that question by construction: anyone citing orders_contract.yaml v1.0.0 is unambiguously citing all three components together, exactly as they stood at that precise moment. This module's lesson 6 revisits this same idea when showing precisely what it means for that single version number to go up.

Common mistakes

Calling any YAML file that describes columns a "contract." What happens: someone writes a YAML with column names and types, and calls it "X's data contract," with no SLA or violation policy at all. Why it happens: YAML is the format you associate with contracts after this lesson, and it's easy to confuse format with content. How to spot it: mentally apply classify_contract() — if your file only has the key equivalent to schema, it's a schema in YAML, not a contract. How to fix it: require the presence of all three components before calling any artifact a "contract"; a schema with no SLA or policy is, at most, a draft.

Thinking the SLA only applies to "when" the file arrives. What happens: someone reduces the SLA concept solely to punctuality (freshness), ignoring that it also covers volume. Why it happens: freshness is the SLA part that's easiest to explain with a concrete deadline. How to spot it: revisit lesson 3's contract — sla.row_count has a minimum and a maximum, with no relation to time at all. How to fix it: a data SLA typically covers two independent questions: did it arrive on time? and did it arrive with a reasonable volume? A file that arrives punctually but with 200 rows when 12 were expected also violates its SLA, even though punctuality is perfect.

Confusing the violation policy with the mechanism that enforces it. What happens: someone, seeing on_violation: quarantine in lesson 3's contract, expects this module to also implement the quarantine() function that separates good rows from bad ones. Why it happens: the field is right there, declared, since the contract's first version — it seems natural for the same module to also execute it. How to spot it: revisit what this module actually builds — it parses the contract, generates a Pandera schema. No lesson implements quarantine(). How to fix it: on_violation is, in this module, declarative metadata — a text value stating what the correct policy would be. This guide's module 7 ("the incident and data governance") is the one that builds the real mechanism that reads that value and acts on it.

Exercises

Exercise 1 — Build a fourth draft with SLA and policy, but no schema. Using this lesson's classify_contract(), build a dictionary with sla and on_violation, but without the schema key, and confirm the verdict.

sla_and_policy_doc = {"dataset": "orders_s04", "sla": {"freshness_hours": 24}, "on_violation": "quarantine"}
result = classify_contract(sla_and_policy_doc)
print(result)
print("IS a complete data contract" if all(result.values()) else "is NOT a complete data contract")

{'has_schema': False, 'has_sla': True, 'has_violation_policy': True}
is NOT a complete data contract

Exercise 2 — Argue whether module 1 lesson 5's informal 24-hour SLA, by itself (with no written schema or policy), would qualify as "a contract." In 2-3 sentences, using classify_contract() as a conceptual reference, explain why that informal agreement — even though real and already mentioned in this guide — wouldn't qualify as a complete contract.

Exercise 3 — Explain, in your own words, why "schema" and "contract" aren't synonyms, using the restaurant menu example. Without repeating the analogy exactly as written in this lesson, write your own version of why a menu that only said "fresh fish" wouldn't protect the restaurant the same way a complete one would.

Summary and next step

In this lesson you gave "data contract" a precise, three-component definition — schema, SLA, violation policy — and confirmed, with classify_contract() actually run, that a document needs all three to qualify as a complete contract, not just the easiest one to write (the schema). You also saw that a contract doesn't replace OrdersSchema — it includes it, and adds two questions Pandera, by design, never set out to answer.

Before moving on you should be able to: name a contract's three components without hesitation; and explain why a YAML that only has columns and types isn't, yet, a data contract.

Lesson 3 builds S04's real contract, section by section, with the exact reason behind every field.

Resources

• PayPal — official data-contract-template repository (the reference structure of a complete contract — schema, quality, SLA, security — this lesson simplifies down to this guide's three central components). github.com/paypal/data-contract-template. In English.
• Andrew Jones (GoCardless) — "Improving Data Quality with Data Contracts" (the original article that already distinguished schema from service expectations). medium.com/gocardless-tech/improving-data-quality-with-data-contracts-238041e35698. In English.
• PyYAML — official documentation (yaml.safe_load, the function this lesson uses to read the contract). pyyaml.org/wiki/PyYAMLDocumentation. In English.
• Module 1, lesson 5, of this same guide — the exact source of the informal 24-hour SLA this lesson uses as an example of an incomplete component. src/guides/data-reliability-and-governance-guide/workbook/module-01-when-green-does-not-mean-correct/es/05-meet-s04-kioskos-fourth-store.md. In Spanish.
• This guide's DESIGN. src/guides/data-reliability-and-governance-guide/DISENO.md. In Spanish.