5. Output Formatting and Parsing

Capsule overview

In production, the LLM's output has to be parseable by code. An output that's "almost JSON" or that comes wrapped in extra text breaks your pipeline. In this capsule you'll learn the four main formats (JSON, XML, Markdown, CSV), techniques to maximize format consistency, robust extraction with regex when the output drifts, and how to handle malformed outputs with retry logic.

The focus is production: you need your code to work 99.9% of the time, not 80%. For that, the parsing system has to handle every variation LLMs can generate, from pure JSON to JSON wrapped in markdown with an introductory sentence.

Why it matters: If your classifier returns "TECHNICAL" 95% of the time but "The ticket is of type TECHNICAL." the other 5%, you have a production bug that shows up intermittently — the worst kind of bug. This capsule gives you the tools to eliminate it.

The 4 Main Formats

JSON — The standard for APIs

JSON is the most used format when the output is going to be processed by code. Easy to parse, extensible, and familiar to every LLM.

from openai import OpenAI
import json

client = OpenAI()

# Basic extraction with JSON
def extract_entities(text: str) -> dict:
    """
    Extracts people, organizations and locations from the text.
    Uses JSON mode to guarantee valid JSON.
    """
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": """
Extract entities from the text.
Response format (exact JSON):
{"people": ["name1", ...], "organizations": ["org1", ...], "locations": ["location1", ...]}
If there are no entities of some type, use an empty list. Only JSON, nothing else.
"""
            },
            {"role": "user", "content": text}
        ],
        temperature=0,
        response_format={"type": "json_object"}  # Guarantees valid JSON (OpenAI)
    )
    return json.loads(response.choices[0].message.content)

# Test
texts = [
    "Maria Garcia from Google visited the Microsoft office in Madrid.",
    "Apple's CEO met with the president of France in Paris.",
    "There are no people or places mentioned in this text."
]

for t in texts:
    result = extract_entities(t)
    print(f"Input: {t[:60]}")
    print(f"Output: {result}\n")

Output:

Input: Maria Garcia from Google visited the Microsoft office in Mad
Output: {'people': ['Maria Garcia'], 'organizations': ['Google', 'Microsoft'], 'locations': ['Madrid']}

Input: Apple's CEO met with the president of France in Paris.
Output: {'people': ["Apple's CEO", 'president of France'], 'organizations': ['Apple'], 'locations': ['Paris']}

Input: There are no people or places mentioned in this text.
Output: {'people': [], 'organizations': [], 'locations': []}

XML — For nested structure and Claude

XML is especially effective with Claude, which is optimized to follow instructions with tags. It's also useful for documents with a hierarchical structure.

from anthropic import Anthropic
import xml.etree.ElementTree as ET

ant_client = Anthropic()

def extract_xml_analysis(text: str) -> dict:
    """Extracts a structured analysis in XML format (optimal for Claude)."""
    response = ant_client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=500,
        system="""
Analyze the text and respond ONLY with XML in this exact format:
<analysis>
  <sentiment>POSITIVE|NEGATIVE|NEUTRAL</sentiment>
  <topics>
    <topic>topic1</topic>
    <topic>topic2</topic>
  </topics>
  <summary>one-sentence summary</summary>
</analysis>
No text before or after the XML.
""",
        messages=[{"role": "user", "content": text}]
    )
    
    raw_xml = response.content[0].text.strip()
    
    # Parse XML
    root = ET.fromstring(raw_xml)
    return {
        "sentiment": root.find("sentiment").text,
        "topics": [t.text for t in root.findall("topics/topic")],
        "summary": root.find("summary").text
    }

# Test
text = "The new iPhone has an incredible camera but the battery barely lasts. Fans are divided."
try:
    result = extract_xml_analysis(text)
    print(f"Analysis: {result}")
except ET.ParseError as e:
    print(f"Error parsing XML: {e}")

Markdown — For human-readable output

def generate_markdown_report(data: dict) -> str:
    """Generates a Markdown report for display."""
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": """
Generate an executive report in Markdown with exactly this structure:
## Executive Summary
[2-3 sentences]

## Key Metrics
| Metric | Value | Trend |
|--------|-------|-------|
[data rows]

## Recommendations
1. [First recommendation]
2. [Second recommendation]
3. [Third recommendation]

Markdown only. No additional text.
"""
            },
            {"role": "user", "content": str(data)}
        ],
        temperature=0.2
    )
    return response.choices[0].message.content

# Test
sales_data = {
    "Q1": 1200000, "Q2": 1500000, "Q3": 1100000, "Q4": 1800000,
    "top_product": "Enterprise Plan", "retention_rate": "87%"
}
print(generate_markdown_report(sales_data))

CSV — For tabular data

import csv
import io

def extract_csv_data(text: str) -> list[dict]:
    """Extracts structured data in CSV format."""
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": """
Extract the products mentioned in the text.
Exact CSV format (with header):
name,price,available

Rules:
- One row per product
- price: number only, no currency symbol (e.g. 29.99)
- available: true or false
- If there is no price: leave it empty (two commas in a row)
CSV only. No additional text.
"""
            },
            {"role": "user", "content": text}
        ],
        temperature=0
    )
    
    raw_csv = response.choices[0].message.content.strip()
    reader = csv.DictReader(io.StringIO(raw_csv))
    return list(reader)

products_text = "We have the Pro laptop for $1299 (in stock) and the basic mouse for $25 (sold out)."
products = extract_csv_data(products_text)
print(f"Extracted products: {products}")

Techniques to Guarantee Format Consistency

Technique 1: Prominent explicit instruction

The format instruction has to be impossible to ignore:

# ❌ Buried instruction
prompt = """
You are a data extractor. Analyze the text and find the relevant data.
Make sure you're precise. The response format must be JSON with the keys: name, email.
The text may contain several types of information. Only extract name and email.
Text: [...]
"""

# ✅ Prominent instruction, at the end
prompt = """
Extract name and email from the text.

REQUIRED FORMAT (only this, nothing else):
{"name": "string or null", "email": "string or null"}

Text: [...]
"""

Technique 2: A concrete output example

def extract_contact(text: str) -> dict:
    """Extracts a contact with an explicit output example."""
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": """
Extract name and email from the text.

Example of a correct output: {"name": "John Garcia", "email": "john@mail.com"}
Example if the email is missing: {"name": "Anna Perez", "email": null}
Example if there's nothing: {"name": null, "email": null}

Respond with the JSON ONLY. No text before or after.
"""
            },
            {"role": "user", "content": text}
        ],
        temperature=0,
        response_format={"type": "json_object"}
    )
    return json.loads(response.choices[0].message.content)

# Test with edge cases
cases = [
    "Contact Michael at michael@company.com",
    "There's only a name here: Robert",
    "This message has no contact data"
]

for c in cases:
    print(f"'{c}' → {extract_contact(c)}")

Technique 3: JSON Mode (OpenAI) and JSON Schema

OpenAI offers two levels of guarantee for JSON:

# Level 1: JSON Mode — guarantees valid JSON, no fixed schema
response_json_mode = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[...],
    response_format={"type": "json_object"},  # Valid JSON guaranteed
    temperature=0
)

# Level 2: Structured Outputs — JSON that conforms to a specific JSON Schema
from pydantic import BaseModel

class ContactOutput(BaseModel):
    name: str | None
    email: str | None

# With OpenAI's parse() (requires pydantic)
response_structured = client.beta.chat.completions.parse(
    model="gpt-4o-mini",
    messages=[...],
    response_format=ContactOutput  # Guaranteed schema, validated by Pydantic
)
contact = response_structured.choices[0].message.parsed
print(f"name={contact.name}, email={contact.email}")

Technique 4: Delimiter tags

Use tags to mark exactly where the output starts and ends:

SYSTEM = """
Classify the sentiment of the text.

Respond with the category between tags:
<sentiment>POSITIVE|NEGATIVE|NEUTRAL</sentiment>

That's all. No text before or after the tags.
"""

def extract_from_tags(raw: str, tag: str) -> str | None:
    """Extracts the content between XML tags."""
    import re
    match = re.search(f'<{tag}>(.*?)</{tag}>', raw, re.DOTALL)
    return match.group(1).strip() if match else None

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "The product is incredible, I recommend it."}
    ],
    temperature=0
)
sentiment = extract_from_tags(response.choices[0].message.content, "sentiment")
print(f"Sentiment: {sentiment}")  # POSITIVE

Robust Parsing: Handling Malformed Outputs

LLMs can return JSON wrapped in markdown, with text before or after, or with small syntax errors. Your parser has to handle all of it:

import re
import json

def robust_json_parse(raw: str) -> dict | None:
    """
    Multi-strategy JSON parser.
    Handles: pure JSON, ```json...```, text+JSON, trailing commas.
    Returns None if it can't parse.
    """
    raw = raw.strip()
    
    # Strategy 1: pure JSON
    try:
        return json.loads(raw)
    except json.JSONDecodeError:
        pass
    
    # Strategy 2: extract from a markdown block ```json...``` or ```...```
    match = re.search(r'```(?:json)?\s*\n?(.*?)\n?```', raw, re.DOTALL)
    if match:
        try:
            return json.loads(match.group(1).strip())
        except json.JSONDecodeError:
            pass
    
    # Strategy 3: find the first balanced { ... }
    start = raw.find('{')
    if start == -1:
        return None
    
    depth = 0
    for i, char in enumerate(raw[start:], start):
        if char == '{':
            depth += 1
        elif char == '}':
            depth -= 1
            if depth == 0:
                candidate = raw[start:i+1]
                # Clean trailing commas (a common JSON error)
                candidate = re.sub(r',\s*([}\]])', r'\1', candidate)
                try:
                    return json.loads(candidate)
                except json.JSONDecodeError:
                    break
    
    return None

# Test with real problematic outputs
problematic_outputs = [
    '{"name": "John", "email": "john@mail.com"}',  # Perfect
    '```json\n{"name": "Anna"}\n```',               # Markdown
    'Here is the result:\n{"name": "Peter"}',       # With text
    '{"name": "Charles",}',                          # Trailing comma
    '```\n{"name": "Louis"}\n```',                   # No "json" label
]

for output in problematic_outputs:
    result = robust_json_parse(output)
    status = "✅" if result else "❌"
    print(f"{status} '{output[:50]}' → {result}")

Output:

✅ '{"name": "John", "email": "john@mail.com"}' → {'name': 'John', 'email': 'john@mail.com'}
✅ '```json\n{"name": "Anna"}\n```' → {'name': 'Anna'}
✅ 'Here is the result:\n{"name": "Peter"}' → {'name': 'Peter'}
✅ '{"name": "Charles",}' → {'name': 'Charles'}
✅ '```\n{"name": "Louis"}\n```' → {'name': 'Louis'}

Retry Logic with Feedback

When the first attempt fails, the retry includes specific feedback about what went wrong:

def parse_with_retry(
    raw: str,
    expected_schema: str,
    max_retries: int = 2
) -> dict:
    """
    Tries to parse the output, retrying with feedback if it fails.
    
    Args:
        raw: The model's output
        expected_schema: Description of the expected format, for the feedback
        max_retries: Number of additional attempts
    """
    # First attempt: direct parsing
    result = robust_json_parse(raw)
    if result is not None:
        return result
    
    # Retry with feedback
    for attempt in range(max_retries):
        print(f"  Retry {attempt + 1}: output was not valid JSON")
        
        fix_prompt = f"""
The previous output was not valid JSON.

The output I received:
{raw[:500]}

I need exactly this format:
{expected_schema}

Respond ONLY with valid JSON. No additional text.
"""
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": fix_prompt}],
            temperature=0,
            response_format={"type": "json_object"}  # Forces JSON on the retry
        )
        raw = response.choices[0].message.content
        result = robust_json_parse(raw)
        if result is not None:
            print(f"  ✅ Retry {attempt + 1} succeeded")
            return result
    
    raise ValueError(f"Could not parse after {max_retries} retries. Last output: {raw[:200]}")

# Usage
SCHEMA = '{"sentiment": "POSITIVE|NEGATIVE|NEUTRAL", "confidence": 0.0-1.0}'

# Simulate an initial malformed output
initial_raw = "The sentiment is positive with high confidence."  # Not JSON
try:
    result = parse_with_retry(initial_raw, SCHEMA)
    print(f"Final result: {result}")
except ValueError as e:
    print(f"Error: {e}")

Validation with Pydantic

After parsing the JSON, validate that it matches the expected schema:

from pydantic import BaseModel, Field, ValidationError
from typing import Literal

class SentimentAnalysis(BaseModel):
    sentiment: Literal["POSITIVE", "NEGATIVE", "NEUTRAL"]
    confidence: float = Field(ge=0.0, le=1.0)
    main_aspect: str | None = None

def classify_with_validation(text: str) -> SentimentAnalysis:
    """Classifies and validates with Pydantic."""
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": """
Analyze the sentiment.
JSON: {"sentiment": "POSITIVE|NEGATIVE|NEUTRAL", "confidence": 0.0-1.0, "main_aspect": "string or null"}
JSON only.
"""
            },
            {"role": "user", "content": text}
        ],
        temperature=0,
        response_format={"type": "json_object"}
    )
    
    data = json.loads(response.choices[0].message.content)
    
    try:
        return SentimentAnalysis(**data)
    except ValidationError as e:
        # Sanitize out-of-range values before failing
        if "confidence" in data:
            data["confidence"] = max(0.0, min(1.0, float(data.get("confidence", 0.5))))
        return SentimentAnalysis(**data)

# Test
texts = [
    "The product is excellent, highly recommended",
    "Terrible experience, never again",
    "Fine, neither good nor bad"
]

for t in texts:
    result = classify_with_validation(t)
    print(f"'{t[:40]}' → {result.model_dump()}")

Format Comparison

Connection to the Project

In the Few-Shot Classification System (capsule 08) the classification output always goes through:

1. robust_json_parse() to pull the JSON out of the LLM's output
2. SentimentAnalysis (or an equivalent schema) to validate with Pydantic
3. parse_with_retry() if the first attempt fails
4. Sanitization of out-of-range values before failing with an error

This pipeline guarantees the system doesn't break on unexpected outputs.

Troubleshooting

Problem 1: JSON with a trailing comma

Cause: The model generates {"a": 1, "b": 2,} — that's not standard JSON.

Fix:

# Clean it before json.loads
raw_clean = re.sub(r',\s*([}\]])', r'\1', raw)
json.loads(raw_clean)

Problem 2: Truncated output

Cause: max_tokens too low for the complete JSON.

Fix:

# Estimate the tokens you need: 1 token ≈ 4 chars of JSON
# If the expected output is ~500 chars, set max_tokens=150+
max_tokens = len(json.dumps(expected_schema)) // 3  # With buffer

Problem 3: Strings with unescaped quotes

Cause: "text with "quotes" inside" — invalid JSON.

Fix: Use JSON mode in OpenAI (it guarantees valid JSON) or Structured Outputs to avoid the problem entirely. If you already have the output, a retry with feedback usually fixes it.

Problem 4: The model adds "Sure, here's the JSON:" first

Cause: The format instruction isn't strict enough.

Fix:

# Add to the system prompt:
"Your response must start DIRECTLY with { and end with }."
"No greetings, no explanations, no text before or after the JSON."

Problem 5: Malformed XML (unclosed tags)

Cause: The model generates <sentiment>POSITIVE with no closing tag.

Fix: Use BeautifulSoup with a lenient parser for irregular XML:

from bs4 import BeautifulSoup
soup = BeautifulSoup(raw_xml, "xml")  # More tolerant parser than ElementTree
sentiment = soup.find("sentiment").text

Exercises

Exercise 1: Multi-format parser (Easy)

Implement a function that automatically detects whether the output is JSON, XML, or a plain string, and parses it accordingly.

def auto_parse(raw: str) -> dict | str:
    """
    Detects the format and parses automatically.
    Returns: dict (for JSON/XML) or str (for plain text)
    """
    raw = raw.strip()
    
    # Try JSON
    json_result = robust_json_parse(raw)
    if json_result:
        return json_result
    
    # Try XML
    if raw.startswith("<") or "</" in raw:
        try:
            root = ET.fromstring(raw)
            # Convert simple XML to a dict
            return {child.tag: child.text for child in root}
        except ET.ParseError:
            pass
    
    # Plain text
    return raw

# Test
outputs = [
    '{"key": "value"}',
    '<result><value>42</value></result>',
    'POSITIVE'
]
for o in outputs:
    print(f"'{o}' → {auto_parse(o)} (type: {type(auto_parse(o)).__name__})")

Exercise 2: Validate a complex schema (Medium)

Create a Pydantic schema for the Prompt Analyzer's output (with classification, components, suggestions, quality) and a function that parses and validates it.

from pydantic import BaseModel, Field
from typing import Literal

class ClassificationOutput(BaseModel):
    technique: Literal["zero-shot", "few-shot", "chain-of-thought", "mixed"]
    confidence: float = Field(ge=0.0, le=1.0)

class ComponentsOutput(BaseModel):
    instruction: Literal["present", "absent", "implicit"]
    context: Literal["present", "absent", "implicit"]
    output_format: Literal["present", "absent", "implicit"]

class PromptAnalysisOutput(BaseModel):
    classification: ClassificationOutput
    components: ComponentsOutput
    suggestions: list[str] = Field(min_length=1, max_length=6)
    score: int = Field(ge=0, le=100)

def parse_prompt_analysis(raw: str) -> PromptAnalysisOutput | None:
    data = robust_json_parse(raw)
    if not data:
        return None
    try:
        return PromptAnalysisOutput(**data)
    except ValidationError as e:
        print(f"Validation failed: {e}")
        return None

Exercise 3: Retry with specific feedback (Hard)

Implement an improved version of parse_with_retry where the feedback message states exactly which part of the schema failed (missing field, wrong type, out-of-range value).

def parse_with_detailed_retry(
    raw: str, 
    schema_cls: type[BaseModel],
    max_retries: int = 2
) -> BaseModel:
    """Retry with detailed feedback about the validation error."""
    
    for attempt in range(max_retries + 1):
        data = robust_json_parse(raw)
        if data:
            try:
                return schema_cls(**data)
            except ValidationError as e:
                # Pull the specific errors out of Pydantic
                errors = []
                for error in e.errors():
                    field = ".".join(str(l) for l in error["loc"])
                    error_type = error["type"]
                    message = error["msg"]
                    errors.append(f"  - Field '{field}': {error_type} — {message}")
                
                error_detail = "\n".join(errors)
                
                if attempt < max_retries:
                    fix_prompt = f"""
The previous JSON has validation errors:
{error_detail}

The invalid JSON I received:
{raw[:300]}

Expected schema: {schema_cls.model_json_schema()}

Fix it and return ONLY the valid JSON.
"""
                    response = client.chat.completions.create(
                        model="gpt-4o-mini",
                        messages=[{"role": "user", "content": fix_prompt}],
                        temperature=0,
                        response_format={"type": "json_object"}
                    )
                    raw = response.choices[0].message.content
        else:
            if attempt < max_retries:
                fix_prompt = f"The output is not valid JSON: '{raw[:100]}'. Return JSON only."
                response = client.chat.completions.create(
                    model="gpt-4o-mini",
                    messages=[{"role": "user", "content": fix_prompt}],
                    temperature=0,
                    response_format={"type": "json_object"}
                )
                raw = response.choices[0].message.content
    
    raise ValueError(f"Parsing failed after {max_retries} retries")

Summary

In this capsule you learned:

• Formats: JSON for APIs/code, XML for Claude/hierarchy, Markdown for humans, CSV for tables, exact string for classification
• Consistency techniques: Prominent instruction at the end, output example, JSON mode (OpenAI), Structured Outputs, XML delimiter tags
• Robust parser: Multi-strategy: pure JSON → markdown → first {...} → clean trailing commas
• Retry logic: Direct first attempt, retry with specific feedback if it fails, JSON mode forced on the retry
• Pydantic validation: Parse + validate the schema + sanitize out-of-range values + specific error feedback

Next capsule: Boundary testing — what happens when you get empty, adversarial, or extremely long inputs, and how to make your prompts defensive.

Further resources

1. OpenAI JSON Mode — Documentation for response_format and Structured Outputs with JSON Schema
2. OpenAI Structured Outputs (Pydantic) — How to use client.beta.chat.completions.parse() with Pydantic models
3. Pydantic v2 Validators — Advanced validators for complex schemas
4. Anthropic Structured Outputs — Anthropic's native JSON Schema (beta)
5. Python json module — Complete documentation with error handling
6. Python re module — Regular expression reference for the multi-strategy parser