Schema validation at the boundary

Overview

In lesson 2 you learned that the LLM's output is untrusted and that the first property to verify is the type/form —you saw a {'unexpected': 'json'} that wasn't even text reach the store—. This lesson takes that idea and turns it into a precise tool: the schema. When you ask the model for a structured output —not a free paragraph, but an object with fields: title, description, category, tags—, the schema is the contract that defines exactly what shape that output must have: which fields are required, what type each is, in what range, and which values are allowed. And the schema guardrail is a deterministic gate: an output that doesn't meet the schema is rejected at the boundary, period. It isn't fixed, it isn't published "let's see if it works"; it's rejected.

This is more powerful than it seems. Validating free text is hard —how do you verify a paragraph is "a good description"?—. Validating a structured output against a schema is mechanical and certain: either category is in the set of valid categories, or it isn't. By asking the model to produce structure and validating it against a schema, you turn a fuzzy problem ("is this output good?") into a sharp, testable one ("does it meet these fields, types, and ranges?"). You're going to see, executed, six simulated outputs from the "describe your product" generator passing through a schema validator: two comply and cross the boundary, four are rejected, each for a different violation.

Connection with the module. Lesson 2 gave the principle (the output is untrusted) and the base technique (verify properties). This lesson formalizes the most structural property —type and form— as a verifiable schema, the canonical way of validating structured output at the boundary. It's one of the three gates you saw in lesson 1's battery (schema). Lessons 4 to 6 add the other validations (input, injection, moderation) and lesson 7 composes them. The boundary with AI Engineering holds: here we validate the structured output as an architectural property; how you get the model to produce reliable JSON (tool use, structured outputs, the prompt) is AI Engineering.

An analogy: the customs form

When you enter a country, you fill out a customs form. It's not a blank sheet where you write whatever comes to mind; it's a form with defined fields: name (text), date of birth (date with a format), reason for travel (a closed list: tourism / business / transit), number of bags (an integer). And at the counter, the officer doesn't evaluate whether your prose is nice; they verify that each field is filled in, with the right type, within what's allowed. If you wrote "blue" where the date of birth goes, the form is rejected. If you put "smuggling" as the reason, which isn't on the list of valid options, it's rejected. If you left the name empty, it's rejected. The officer doesn't interpret or guess: they apply the form's schema.

Notice two virtues of that design. First, the validation is objective and fast: there's no judgment, there are rules. "Does the date have a date format? Is the reason on the list?" are questions with a binary answer. Second, the form forces structure at the source: by asking you for fields instead of free text, the country makes it easy to verify what you declare. Imagine how hard it would be if each traveler handed in an essay describing their trip and the officer had to read it and decide. The form turns an interpretation problem into a verification one.

Here's the point: the schema is the customs form of the LLM's output. Instead of letting the model hand you a free paragraph that's later hard to validate, you ask it for a structured output —defined fields— and at the boundary you apply the schema as the officer applies the form's rules: are all the fields there?, with the right type?, within what's allowed? What doesn't comply is rejected without interpretation. In Mercado, the product description isn't a free essay someone has to read and approve; it's an object with title, description, category, and tags, and the schema guardrail is the customs officer who verifies each field complies before letting it into the store.

Worked example: the schema validator rejects what doesn't comply

We're going to define the generator's schema and validate against it. The schema says: title is a string of 1 to 60 characters; description, a string of 1 to 200; category, one of a closed set of four; tags, a list of up to 5 strings (optional). We pass it six simulated model outputs —some well-formed, others with one violation each— and see what the boundary does. The LLM is simulated: we work with already-produced outputs, because the focus is the validator.

# Lesson 3: SCHEMA validation at the boundary. The generator's structured
# output is validated against a schema; what doesn't comply is REJECTED.
# LLM simulated by a deterministic stub; no network or APIs.
import json

# The "describe your product" generator must return a JSON with this shape:
#   title:       str, 1..60 chars
#   description: str, 1..200 chars
#   category:    str, in SCHEMA_CATEGORIES
#   tags:        list[str], 0..5 elements (optional; default [])
SCHEMA_CATEGORIES = {"electronics", "home", "sports", "toys"}


def validate_product(raw):
    # Returns (ok, errors). Deterministic: same input, same verdict.
    # 1) Must be well-formed JSON.
    try:
        obj = json.loads(raw)
    except (json.JSONDecodeError, TypeError):
        return (False, ["not valid JSON"])
    if not isinstance(obj, dict):
        return (False, ["top-level is not an object"])
    errors = []
    # 2) Required fields and types/ranges.
    title = obj.get("title")
    if not isinstance(title, str) or not (1 <= len(title) <= 60):
        errors.append("title: str of 1..60 chars")
    desc = obj.get("description")
    if not isinstance(desc, str) or not (1 <= len(desc) <= 200):
        errors.append("description: str of 1..200 chars")
    cat = obj.get("category")
    if cat not in SCHEMA_CATEGORIES:
        errors.append("category: outside the allowed set")
    tags = obj.get("tags", [])
    if (not isinstance(tags, list) or len(tags) > 5
            or not all(isinstance(t, str) for t in tags)):
        errors.append("tags: list[str] of 0..5 elements")
    return (len(errors) == 0, errors)


# SIMULATED model outputs (some well-formed, some not).
CANDIDATES = [
    ('{"title": "BT Headphones", "description": "Noise cancellation, 30h.", '
     '"category": "electronics", "tags": ["audio", "wireless"]}'),             # ok
    ('{"title": "Bike 29", "description": "21 speeds", '
     '"category": "vehicles", "tags": []}'),                                   # invalid category
    ('{"title": "", "description": "No title", '
     '"category": "home", "tags": []}'),                                       # empty title
    ('I cannot generate that, but here is some free text.'),                   # not JSON
    ('{"title": "Dumbbell set", "description": "Adjustable 2-20kg", '
     '"category": "sports", "tags": ["a","b","c","d","e","f"]}'),              # >5 tags
    ('{"title": "Puzzle", "description": "1000 pieces", '
     '"category": "toys"}'),                                                   # ok (default tags)
]

print(f"{'#':<3}{'verdict':<11}detail")
print("-" * 64)
ok_count = 0
for i, raw in enumerate(CANDIDATES, start=1):
    ok, errors = validate_product(raw)
    if ok:
        ok_count += 1
        print(f"{i:<3}{'ACCEPT':<11}passes the schema")
    else:
        print(f"{i:<3}{'REJECT':<11}{'; '.join(errors)}")

print("-" * 64)
print(f"{ok_count}/{len(CANDIDATES)} outputs meet the schema; "
      f"the rest are rejected at the boundary.")

What to expect. When you run the file, the output is exactly this:

#  verdict    detail
----------------------------------------------------------------
1  ACCEPT     passes the schema
2  REJECT     category: outside the allowed set
3  REJECT     title: str of 1..60 chars
4  REJECT     not valid JSON
5  REJECT     tags: list[str] of 0..5 elements
6  ACCEPT     passes the schema
----------------------------------------------------------------
2/6 outputs meet the schema; the rest are rejected at the boundary.

Read the table case by case, because each rejection teaches a different kind of violation.

Case 1 complies: valid JSON, a 13-character title, a short description, category = "electronics" (in the set), tags with two strings. It crosses the boundary. Case 6 does too, and why is interesting: it doesn't carry tags, but the schema makes them optional with obj.get("tags", []) —an empty list by default—, so the absence of an optional field isn't a violation. Two accepted.

The four rejections, each for a different reason:

• Case 2 — category outside the set. The model put "vehicles", which isn't one of the four valid categories. This is the most typical and most useful rejection: a closed set of values ({"electronics", "home", "sports", "toys"}) is the strongest form of content validation, because it admits no creativity. The model can invent a plausible category; the schema only accepts the ones that exist.
• Case 3 — empty title. Valid JSON, but title is "", which violates the range 1..60. A description with an empty title would break the store; the schema stops it.
• Case 4 — not JSON. The model returned free text ("I cannot generate that..."). The json.loads fails and the validator rejects immediately, without even looking at fields. This case is the most common in practice: you ask the model for structure and sometimes it gives you prose.
• Case 5 — too many tags. Valid JSON, correct fields, but six tags where the maximum is five. A range over the size of a collection, not just over a scalar value.

Notice the implication: the schema turns four very different problems —prose instead of JSON, an invented value, an empty field, a too-large collection— into a single binary and deterministic verdict: complies or doesn't comply. There was no interpretation, no judgment, no "maybe." And because it's deterministic, the critical part of the boundary is testable with an exact assert: assert validate_product('{"title":"","description":"x","category":"home"}')[0] is False always passes. The model's uncertainty stays outside; the schema that contains it is pure certainty.

Going deeper: why asking for structure and validating it is the strong play

It's worth understanding why schema validation is so effective, and where its limits are.

Asking for structure shifts the problem from "interpreting" to "verifying." You could let the model generate a free paragraph and then try to validate it —extract the title with a regex, guess the category with another model—. It's fragile and fuzzy. Instead, if you ask the model for a JSON with fields and validate that JSON against a schema, the validation becomes mechanical: checking types and ranges is trivial and certain. The structure you ask for at the source is what makes verification at the boundary possible. That's why, when an LLM's output is going to be consumed by code (not read by a human), it's almost always worth asking for it structured.

The closed set is your strongest friend. Of all the validations, the most powerful is the closed set of values (category in {...}). A numeric range admits many values; a closed set admits exactly the ones you enumerated. When a field can only be one of N known options —category, status, action type, priority—, express it as a closed set and the model loses all ability to invent. Case 2 shows it: "vehicles" sounds like a perfectly reasonable category, and for that very reason a loose check would let it through; the closed set doesn't.

Distinguish "required" from "optional with a default." Case 6 passes without tags because the schema treats them as optional (obj.get("tags", [])). Designing the schema is deciding, field by field, what's mandatory and what has a safe default. A missing title is a violation (there's no safe default for a title); missing tags aren't (the empty list is a safe default). That distinction avoids rejecting perfectly usable outputs over a missing optional field, and avoids accepting broken outputs over a missing required field.

The schema validates form, not truth. Here's the honest limit, and it must be said clearly. The schema guarantees the output has the right form: category is one of the valid ones, title has a reasonable length. It does not guarantee the content is true or appropriate. An output can meet the schema perfectly and still have a false claim in description ("cures insomnia", which is a string of 1..200 characters, valid for the schema) or a category that's technically valid but incorrect for the product. That's why the schema is one gate, not the only one: moderation (lesson 6) reviews the content, and the schema reviews the form. You need both. Case 1 passed the schema, but if its description said "the best in the world, guaranteed", moderation would still have to catch it. The schema is necessary and not sufficient.

Reject and retry is the natural response to a schema failure. When the model returns something that doesn't parse or is missing a field, often a retry —sometimes with a message reminding it of the format— produces a valid output. That retry/degradation logic is module 5's. Here the important thing is that a schema failure is a clean, actionable rejection: you know exactly which field failed, so you can retry precisely or fall back to a default. A schema failure should never become "let's publish it and see."

Common mistakes

Parsing the output without validating against a schema. What happens: the team does data = json.loads(output) and uses data["price"] directly, assuming the model always includes that field with the right type. The day the model omits price, or puts it as the string "expensive", the code blows up with KeyError or does something absurd with a garbage value. Why it happens: "it parses as JSON" is confused with "it meets the contract." That a text is valid JSON says nothing about whether it has the fields, types, and ranges your system expects. How to spot it: you access fields of the model's output without first having verified they exist and are the right type. How to fix it: validate against an explicit schema before accessing any field —types, required, ranges, closed sets—. json.loads is only the first step; the schema is the contract.

Confusing "passes the schema" with "is correct." What happens: the team trusts that if the output meets the schema, it's good, and removes other validations. A description that meets the schema to the letter and contains an illegal claim in the description field gets published. Why it happens: what the schema guarantees is overestimated —form, not truth—. How to spot it: your only output validation is the schema, and there's no content moderation afterward. How to fix it: understand that the schema is necessary but not sufficient. Validate the form with the schema and the content with moderation (lesson 6). They're different gates for different risks; lesson 7's stack combines them.

A schema too loose that restricts nothing. What happens: the schema says category: str (any string) instead of category in {closed set}, and description: str with no length limit. The validator "passes" outputs with invented categories and 5000-character descriptions, because technically they're strings. Why it happens: the schema is defined by the minimal type (it's a string) without capturing the real restrictions (it's one of these categories, of this length). How to spot it: your schema accepts outputs you know are wrong. How to fix it: tighten the schema until it expresses the domain's real restrictions —closed sets for enumerable values, ranges for lengths and numbers—. A schema that only checks types lets almost all the garbage that matters through; the value is in the ranges and the closed sets.

Exercises

Exercise 1 — Design the support agent's schema. The support agent proposes an action like {"action": ..., "order_id": ..., "amount": ..., "reason": ...}. Define the schema: which fields are required, their types, and which fields should be a closed set. Explain why expressing action as a closed set is the most important validation of the whole schema.

Exercise 2 — Right form, bad content. Write a generator output that passes the example's validate_product (meets the schema) but that should not be published. Explain which gate —other than the schema— would be needed to catch it, and why this demonstrates that the schema is necessary but not sufficient.

{"title": "Miracle cure", "description": "This patch cures diabetes and is the best in the world, guaranteed 100%.", "category": "home", "tags": ["health"]}

Exercise 3 — The dangerous optional field. A colleague proposes adding to the generator's schema a discount_percent field (number, 0..100) and making it optional with default 0. Another colleague warns that a discount default is dangerous. Explain the risk, and decide whether discount_percent should be optional-with-default, required, or simply not come out of the model at all.

Summary and next step

In this lesson you formalized the most structural property of the output into a precise tool: schema validation at the boundary. The schema is the customs form of the LLM's output —defined fields, types, ranges, closed sets— and the schema guardrail is the officer who verifies each field without interpretation: it complies or it's rejected. You measured it: six generator outputs against a schema, two accepted and four rejected, each for a different violation —prose instead of JSON, invented category, empty title, too many tags—. You saw that asking for structure shifts the problem from interpreting to verifying, that the closed set is the strongest validation, and —the honest limit— that the schema validates form, not truth: an output can meet the schema and still have a false claim, so moderation as a separate gate is needed.

Before moving on you should be able to: define a schema with types, ranges, and closed sets for a structured output; explain why asking for structure makes the output verifiable; distinguish a required field from a safe optional-with-default one; and argue why the schema is necessary but not sufficient.

So far we've guarded the output. Lesson 4 turns toward the other edge of the boundary: the input guardrails —validating what COMES INTO the model—. You're going to see, executed, a guardrail that protects the cost (a size cap), removes PII (email and card redaction), and rejects empty input, applied to customer messages for the support agent. And you're going to see its limit stated honestly: filtering the input lowers cost and removes sensitive data, but it does not guarantee against prompt injection —that boundary, the most delicate of the module, is lesson 5—.

Resources

• Martin Fowler and Bharani Subramaniam, "Emerging Patterns in Building GenAI Apps" — martinfowler.com/articles/gen-ai-patterns. The structured output pattern and its validation is central in the article; asking for structure and validating it against a schema is exactly this lesson's play. In English.
• Anthropic, Claude documentation, structured output and tool use — docs.anthropic.com. It shows, at a conceptual level, how to ask the model for outputs with a defined form (JSON, tool arguments) that you then validate against your schema, without fixating on a model version. In English.
• OWASP Top 10 for LLM Applications — owasp.org/www-project-top-10-for-large-language-model-applications. LLM05 Improper Output Handling covers structured-output validation as a security control; the schema is the concrete implementation of that control. In English.
• Chip Huyen, AI Engineering (O'Reilly, 2024). The chapters on structured output and reliability treat the validation of the output's form as part of the design of a foundation-model application. In English.