Should S04 be allowed to write without a contract?

Description

Everything this module has built so far is post-hoc validation: S04's data already exists, it's already in kiosko.duckdb, and the contract gets used to check it after it arrived. This lesson asks a different question, one of governance, not code: if a contract can decide with precision which data is valid, shouldn't it also be able to decide whether a dataset can even start writing, before a single file arrives? This question doesn't have one single correct answer — but it has an answer with real history behind it, and this lesson tells that story with evidence and with code.

Connection to the module. Lessons 4 and 5 used the contract to check data that had already arrived. This lesson uses the same contract, already parsed and registered, for a different question: should S04 have been able to write its first file at all, before this module existed?

An analogy: the lease, signed before you move in

Think of two ways to rent an apartment. The first: you move in first — you bring your things, you start living there — and only afterward, if something breaks or there's a disagreement with the landlord, does anyone check which rules applied. The second: you sign the lease before you get the keys — with the usage rules, the deposit, and the consequences of breaking something, all written and agreed on beforehand, before a single box enters the apartment. The first way isn't, necessarily, irresponsible — plenty of people live like this, informally, with a relative or a friend. But when something breaks, the question "was this allowed?" has no clear answer, because no rule was ever written down before the situation was already underway.

S04 "moved in" — started selling, started sending files — before any contract existed. Foundations M7 was the first collision with that reality: S04 was already selling, DIM_STORE didn't know about it yet. And even though S04 has been registered since lesson 5 of module 1, up until the start of this module, nobody had signed any contract before its first real file arrived. This module came later — reviewing what was already at the door, not deciding whether it should come in.

Two governance models: post-hoc validation versus admission control

It's worth naming both models precisely, because each has real advantages and costs — neither is simply "better" than the other.

Post-hoc validation (what modules 1 through 3 built, and what this module has kept building until now): data arrives first, with no admission check at all, and gets reviewed afterward with the tools you already know. The advantage: it never blocks anyone — a new data producer, like S04 on its first day, can start sending information without waiting for any approval process. The cost: any problem gets discovered after the data is already inside the system, potentially already used by a report or a dashboard before anyone notices the error.

Admission control: before a dataset can even write a single row, the system asks "does a registered contract exist for this?" — and if it doesn't, the write gets blocked, no matter how good the data itself is. The advantage: no data with unknown rules ever enters the system. The cost: a legitimate, new data producer, like S04 sending its first file, would stay blocked until someone — a human — takes the time to write a contract first, even if its data is perfect.

Worked example: applying admission control, retroactively, to S04's real timeline

# admission_demo.py
import yaml
from contract import DataContract

CONTRACT_REGISTRY: dict[str, DataContract] = {}


def register_contract(contract: DataContract) -> None:
    CONTRACT_REGISTRY[contract.dataset] = contract


def admission_check(dataset: str) -> tuple[bool, str]:
    """Decide whether a dataset can write, BEFORE any file arrives."""
    if dataset not in CONTRACT_REGISTRY:
        return False, f"BLOCKED -- no registered contract exists for '{dataset}'"
    contract = CONTRACT_REGISTRY[dataset]
    return True, f"ALLOWED -- contract '{contract.contract_version}' registered for '{dataset}'"


print("=== 2026-08-14: S04 sends orders_2026-08-14.csv (in the real timeline) ===")
allowed, reason = admission_check("orders_s04")
print(f"admission_check('orders_s04') -> allowed={allowed}")
print(f"  {reason}")

print("\n=== Module 4: the contract is written and registered ===")
with open("orders_contract.yaml") as f:
    raw = yaml.safe_load(f)
contract = DataContract.model_validate(raw)
register_contract(contract)
print(f"register_contract(dataset='{contract.dataset}', version='{contract.contract_version}')")

print("\n=== S04's NEXT file (hypothetical, future) ===")
allowed, reason = admission_check("orders_s04")
print(f"admission_check('orders_s04') -> allowed={allowed}")
print(f"  {reason}")

What to expect. Running python3 admission_demo.py, with lesson 3's orders_contract.yaml in the same folder, the output is exactly this:

=== 2026-08-14: S04 sends orders_2026-08-14.csv (in the real timeline) ===
admission_check('orders_s04') -> allowed=False
  BLOCKED -- no registered contract exists for 'orders_s04'

=== Module 4: the contract is written and registered ===
register_contract(dataset='orders_s04', version='1.0.0')

=== S04's NEXT file (hypothetical, future) ===
admission_check('orders_s04') -> allowed=True
  ALLOWED -- contract '1.0.0' registered for 'orders_s04'

Read this result carefully, because it is, with executed evidence, the uncomfortable answer to this lesson's title question: applied retroactively, a strict admission control would have blocked S04's real file from 2026-08-14 — the same file that modules 1 through 3 of this guide diagnosed in depth. CONTRACT_REGISTRY was empty for orders_s04 at that point in the timeline, because the contract wasn't written until this module, several steps after the incident. Only after register_contract() — the moment equivalent to "this module already exists" — would S04's next hypothetical file pass the check.

Diagram: the same timeline, with and without admission control

flowchart TD
    A["2026-08-14: S04 sends\norders_2026-08-14.csv"] --> B{"With post-hoc\nvalidation (real)"}
    A --> C{"With admission\ncontrol (hypothetical)"}
    B --> D["The file is accepted.\nModules 1-3 diagnose it\nafterward, row by row."]
    C --> E["BLOCKED:\nno contract exists\nfor orders_s04 yet"]
    E --> F["S04 cannot sell\nuntil someone\nwrites the contract first"]

Going deeper: why this guide didn't choose strict admission control

It's worth being honest about the decision this guide already made, without presenting it as the only correct option. Modules 1 through 3 — and a good part of this module 4 — built a post-hoc validation system, not admission control: S04 was able to send its file, and Kiosko reviewed it afterward, with increasingly sophisticated tools. That choice has a concrete reason, the same one this lesson's contrast already showed: strict admission control, applied from S04's first day, would have blocked it completely — not a single real sale from the new Mexico City store could have been registered until someone, a human, took the time to write orders_contract.yaml. For a business that needs to start operating, that cost can be unacceptable.

The industry's real answer — the same one documented by GoCardless and the rest of this guide's sources — is almost never "all or nothing" between the two models. It's more common to start with post-hoc validation for new data producers (as Kiosko actually did with S04), and raise the rigor over time: first a contract that only alerts if something doesn't comply (on_violation: alert, blocking nothing), then one that quarantines problematic rows (quarantine, this guide's choice since lesson 3), and only for critical, mature datasets, real admission control that blocks writes with no contract. The on_violation field you already wrote in lesson 3 is exactly the mechanism that allows that gradation — it isn't a binary decision of "block everything" versus "block nothing," it's a spectrum of increasing rigor.

Common mistakes

Concluding that post-hoc validation "was wrong" because admission control would have avoided it. What happens: someone, seeing the worked example's result (BLOCKED), concludes that Kiosko should have used admission control from the start, and that everything built in modules 1 through 3 was, in retrospect, a bad decision. Why it happens: it's tempting to judge a past decision with information you only have afterward. How to spot it: if your conclusion is "they should have blocked S04 from day one," you're not considering the real cost of that alternative — a new store, with no sales registered at all, while someone writes a document. How to fix it: this lesson's Going deeper section says it precisely — both models have real costs, and the correct choice depends on context (how critical the dataset is, how mature the data producer is). Post-hoc validation with a fast response (like the one this guide already built) is, for a case like a store that just opened, a defensible decision, not a mistake.

Thinking admission_check(), as written, is already a production governance system. What happens: someone, excited by the worked example, assumes CONTRACT_REGISTRY and admission_check() are already pieces ready to block real writes in a Kiosko pipeline. Why it happens: the code really runs, with a clear result — it feels like a finished piece. How to spot it: ask yourself who would call admission_check() in a real system, and at exactly which point of the data flow — this lesson never connects it to any real orders_s04 loading step. How to fix it: this example is a conceptual simulation, meant to reason about the governance question with runnable code, not a piece of infrastructure. Connecting real admission control to the exact moment a file arrives — before any INSERT or COPY into a table — is orchestration work, exactly the kind of scheduled task airflow-and-declarative-orchestration-guide teaches you to build.

Assuming "no contract" means "no rules at all." What happens: someone argues that, before this module, S04's data had no rules applying to it, because no formal contract existed. Why it happens: the absence of an artifact called "contract" gets confused with the total absence of expectations. How to spot it: review modules 1 through 3 — validate_orders(), OrdersSchema, validate_referential_integrity() — all of these already existed and ran against S04's data before this module, even though none of them were packaged as a versioned contract. How to fix it: the correct distinction isn't "rules versus no rules" — it's "rules scattered across code, with no declared SLA or policy" versus "rules unified in a versioned artifact with all three components complete." Modules 1 through 3 of this guide never left S04 with no checks at all; this module formalizes those checks, it doesn't invent them out of nowhere.

Exercises

Exercise 1 — Simulate a "graduated" admission control, with three levels instead of two. Modify admission_check() so that, instead of returning only True/False, it returns one of three levels: "blocked" (no contract), "allowed_with_alert" (contract exists, but on_violation is "alert"), "allowed_with_quarantine" (contract exists, on_violation is "quarantine" or "reject").

def admission_check_graduated(dataset: str) -> str:
    if dataset not in CONTRACT_REGISTRY:
        return "blocked"
    contract = CONTRACT_REGISTRY[dataset]
    if contract.on_violation == "alert":
        return "allowed_with_alert"
    return "allowed_with_quarantine"


print(admission_check_graduated("orders_s04"))

allowed_with_quarantine

Exercise 2 — Argue whether S02 Kiosko Norte (a store that has already been selling since the start of this guide) should pass admission_check() today. S02 never had an incident like S04's in foundations M7 — it was always in DIM_STORE from the ecosystem's start. In 2-3 sentences, should admission_check("orders_s02") return True or False in this guide's current state, and why?

Exercise 3 — Write, in 3-4 sentences, your own governance recommendation for Kiosko. Based on everything you saw in this lesson, write a concrete recommendation: should Kiosko implement strict admission control for all its datasets? Only for some? Never? Justify your answer with at least one cost argument and one risk argument.

Summary and next step

In this lesson you faced a governance question, not a code one: if a contract can decide which data is valid, why shouldn't it also decide whether a dataset can even write at all? You simulated, with executed code, admission control applied retroactively to S04's real timeline, and confirmed something uncomfortable but honest: it would have blocked the very file this entire guide used as its case study. You named the two governance models — post-hoc validation versus admission control —, with their real costs, and saw why the industry almost never chooses one of them absolutely.

Before moving on you should be able to: explain the difference between post-hoc validation and admission control, with an example of your own; and argue, with at least one cost and one risk, why Kiosko chose the model that modules 1 through 4 already built.

You have the complete, versioned contract, and now it's also been thought through as a possible entry gate, not just a post-hoc filter. Lesson 8, this module's closing project, assembles everything — parsing, schema generation, comparison against module 2 — into a single final script.

Resources

• Andrew Jones (GoCardless) — "Data Contracts at GoCardless — 6 Months On" (the real experience of introducing contracts gradually, without blocking producing teams from day one). medium.com/gocardless-tech/data-contracts-at-gocardless-6-months-on-bbf24a37206e. In English.
• data-engineering-foundations-guide, module 7 — the exact source of S04's total rejection (ValueError: unknown store_id: S04), this guide's first collision with the governance problem this lesson explores. src/guides/data-engineering-foundations-guide/workbook/module-07-partitioning-and-orchestration/en/06-handling-a-failed-step-without-losing-the-run.md. In English.
• airflow-and-declarative-orchestration-guide — the guide that teaches you to connect real admission control to the exact moment a file arrives, as a DAG's scheduled task. src/guides/airflow-and-declarative-orchestration-guide/DISENO.md. In Spanish.
• Module 1, lesson 5, of this same guide — the exact source of S04's timeline that this lesson re-examines through the lens of admission control. src/guides/data-reliability-and-governance-guide/workbook/module-01-when-green-does-not-mean-correct/en/05-meet-s04-kioskos-fourth-store.md. In English.
• This guide's DESIGN. src/guides/data-reliability-and-governance-guide/DISENO.md. In Spanish.