4. Config Management

Description

The configuration of an AI app is more complex than that of a traditional web app: temperature, model, max_tokens, retry parameters, system prompts, budget limits, and API keys. With pydantic-settings, all that configuration lives in one place, with types, validation, and automatic reading from environment variables. This capsule implements a complete configuration system.

The problem of scattered configuration

# Before: config scattered across multiple places
# models.py
MODEL = "gpt-4o-mini"
TEMPERATURE = 0.7

# utils.py
MAX_TOKENS = int(os.getenv("MAX_TOKENS", "500"))
RETRY_ATTEMPTS = 3

# guardrails/pipeline.py
MAX_INPUT_TOKENS = 4000
INJECTION_THRESHOLD = 0.8

# main.py
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
    raise ValueError("OPENAI_API_KEY not set")

# To change MAX_TOKENS in staging:
# → Search in the 4 files
# → There's no type validation
# → If you write MAKS_TOKENS in .env, no error — it silently uses the default

# After: everything in a centralized Settings
# settings = get_settings()
# settings.model → "gpt-4o-mini"
# settings.temperature → 0.7
# settings.max_tokens → 500
# If MAKS_TOKENS is in .env, pydantic ignores it (it's not in the model)
# If MAX_TOKENS="abc", pydantic raises ValidationError at startup

pydantic-settings: type-safe configuration

pip install pydantic-settings python-dotenv

# src/config.py
from pydantic import Field, SecretStr, field_validator, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
from typing import Literal, Optional
from functools import lru_cache

class Settings(BaseSettings):
    """
    Centralized configuration for the AI app.
    
    Reads automatically from:
    1. Environment variables (highest priority)
    2. .env.{ENVIRONMENT} if it exists
    3. .env (fallback)
    4. Defaults in the model (lowest priority)
    
    Example .env:
        OPENAI_API_KEY=sk-...
        MODEL=gpt-4o-mini
        TEMPERATURE=0.0
        MAX_TOKENS=500
    """
    
    model_config = SettingsConfigDict(
        env_file=".env",
        env_file_encoding="utf-8",
        case_sensitive=False,    # MODEL == model == Model
        extra="ignore"           # Ignore undefined variables (no crash)
    )
    
    # ─── App Settings ────────────────────────────────────────────
    environment: Literal["development", "staging", "production", "testing"] = "development"
    app_name: str = "sentiment-analyzer"
    app_version: str = "0.1.0"
    debug: bool = False
    
    # ─── OpenAI Settings ─────────────────────────────────────────
    # SecretStr: doesn't appear in repr, logs, or str()
    openai_api_key: SecretStr = Field(default=SecretStr(""))
    
    # ─── LLM Parameters ──────────────────────────────────────────
    model: str = "gpt-4o-mini"
    temperature: float = 0.0
    max_tokens: int = 500
    seed: Optional[int] = None
    
    # ─── Retry Settings ──────────────────────────────────────────
    retry_attempts: int = 3
    retry_delay_seconds: float = 1.0
    retry_backoff_multiplier: float = 2.0
    
    # ─── Guardrail Settings ───────────────────────────────────────
    max_input_chars: int = 20_000       # Input character limit
    max_input_tokens: int = 4_000       # Token limit (with tiktoken)
    enable_injection_check: bool = True
    enable_pii_redaction: bool = True
    enable_content_filter: bool = True
    
    # ─── Logging Settings ─────────────────────────────────────────
    log_level: str = "INFO"
    log_to_file: bool = False
    log_file_path: str = "logs/app.json"
    
    # ─── Cost Settings ────────────────────────────────────────────
    max_cost_per_request_usd: float = 0.10
    daily_budget_usd: float = 10.0
    
    # ─── Mock Settings ────────────────────────────────────────────
    use_mock_llm: bool = False
    mock_response: str = '{"sentiment": "positive", "score": 0.8, "confidence": 0.9}'

Validators: declarative validation

# src/config.py (continued)

class Settings(BaseSettings):
    # ... (fields from the previous block) ...
    
    @field_validator("temperature")
    @classmethod
    def validate_temperature(cls, v: float) -> float:
        if not 0.0 <= v <= 2.0:
            raise ValueError(f"temperature must be in [0.0, 2.0], got {v}")
        return v
    
    @field_validator("max_tokens")
    @classmethod
    def validate_max_tokens(cls, v: int) -> int:
        if not 1 <= v <= 128_000:
            raise ValueError(f"max_tokens must be in [1, 128000], got {v}")
        return v
    
    @field_validator("log_level")
    @classmethod
    def validate_log_level(cls, v: str) -> str:
        allowed = {"DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"}
        v_upper = v.upper()
        if v_upper not in allowed:
            raise ValueError(f"log_level must be one of {allowed}")
        return v_upper
    
    @field_validator("model")
    @classmethod
    def validate_model(cls, v: str) -> str:
        # Don't validate against a strict list — OpenAI adds models frequently
        # Just validate that it's not empty
        if not v.strip():
            raise ValueError("model cannot be empty")
        return v.strip()
    
    @model_validator(mode="after")
    def validate_production_requirements(self) -> "Settings":
        """Validations that depend on multiple fields."""
        if self.environment == "production":
            if self.use_mock_llm:
                raise ValueError(
                    "use_mock_llm=True is not allowed in production. "
                    "Set USE_MOCK_LLM=false"
                )
            if not self.openai_api_key.get_secret_value():
                raise ValueError(
                    "openai_api_key is required in production. "
                    "Set OPENAI_API_KEY=sk-..."
                )
            if self.log_level == "DEBUG":
                raise ValueError(
                    "log_level=DEBUG is not allowed in production "
                    "(it can expose sensitive information)"
                )
        return self
    
    # ─── Properties ───────────────────────────────────────────────
    
    @property
    def is_production(self) -> bool:
        return self.environment == "production"
    
    @property
    def is_development(self) -> bool:
        return self.environment == "development"
    
    @property
    def is_testing(self) -> bool:
        return self.environment == "testing"
    
    def create_openai_client(self):
        """Factory to create the OpenAI client with the configured API key."""
        from openai import OpenAI
        api_key = self.openai_api_key.get_secret_value()
        if not api_key:
            raise ValueError("OPENAI_API_KEY not configured")
        return OpenAI(api_key=api_key)
    
    def get_llm_params(self) -> dict:
        """Parameters for the LLM call as a dict."""
        params = {
            "model": self.model,
            "temperature": self.temperature,
            "max_tokens": self.max_tokens,
        }
        if self.seed is not None:
            params["seed"] = self.seed
        return params


@lru_cache()
def get_settings() -> Settings:
    """
    Load and cache the configuration.
    
    Why lru_cache():
    - pydantic-settings reads .env on every instantiation (I/O)
    - In production, the config doesn't change between requests
    - lru_cache() reads the .env only once at startup
    
    For tests: use the override_settings() fixture
    """
    return Settings()

Loading per environment with multiple .env files

# Pattern: .env as the base, .env.{environment} as the override

import os
from pathlib import Path
from pydantic_settings import BaseSettings, SettingsConfigDict

def _get_env_files() -> list[str]:
    """
    Return the .env files to load in priority order (last = highest).
    
    Order:
    1. .env (base, lowest priority)
    2. .env.{environment} (environment-specific override)
    3. System environment variables (highest priority, always)
    """
    env = os.getenv("ENVIRONMENT", "development")
    files = [".env"]
    
    env_specific = f".env.{env}"
    if Path(env_specific).exists():
        files.append(env_specific)
    
    return files

class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=_get_env_files(),
        env_file_encoding="utf-8",
        extra="ignore"
    )
    
    environment: str = "development"
    # ... rest of the fields ...

# File structure:
# .env              → Common variables (no secrets)
# .env.development  → Dev overrides (use_mock_llm=true, debug=true)
# .env.staging      → Staging overrides (budget caps, real LLM)
# .env.production   → Prod overrides (secrets in Vault or env vars, not here)
# .env.testing      → Test overrides (mock=true, log_level=WARNING)
# .env.local        → Personal local override (in .gitignore)

Example .env files

# .env (base — committed, no secrets)
APP_NAME=sentiment-analyzer
APP_VERSION=0.1.0
MODEL=gpt-4o-mini
MAX_TOKENS=500
RETRY_ATTEMPTS=3
LOG_LEVEL=INFO

# .env.development (dev overrides)
ENVIRONMENT=development
TEMPERATURE=0.0
DEBUG=true
USE_MOCK_LLM=true
LOG_LEVEL=DEBUG
DAILY_BUDGET_USD=1.0

# .env.staging (staging overrides)
ENVIRONMENT=staging
USE_MOCK_LLM=false
LOG_LEVEL=INFO
DAILY_BUDGET_USD=10.0
ENABLE_INJECTION_CHECK=true

# .env.example (documentation — committed, no real values)
# Copy this to .env and fill in the values

# REQUIRED in production:
OPENAI_API_KEY=

# Optional (has reasonable defaults):
ENVIRONMENT=development
MODEL=gpt-4o-mini
TEMPERATURE=0.0
MAX_TOKENS=500
LOG_LEVEL=INFO
USE_MOCK_LLM=false
DAILY_BUDGET_USD=10.0

Using SecretStr for secrets

# Why SecretStr instead of str for api_key?

from pydantic import SecretStr

# With str:
settings = Settings(openai_api_key="sk-abcdef123456")
print(settings)  # openai_api_key='sk-abcdef123456' ← IN LOGS!
print(repr(settings.openai_api_key))  # 'sk-abcdef123456' ← EXPOSED
log.info("settings loaded", settings=settings.model_dump())  # ← API KEY IN LOGS!

# With SecretStr:
settings = Settings(openai_api_key=SecretStr("sk-abcdef123456"))
print(settings)  # openai_api_key=SecretStr('**********') ← MASKED
print(repr(settings.openai_api_key))  # SecretStr('**********') ← MASKED

# To use the value:
api_key = settings.openai_api_key.get_secret_value()  # Explicit
client = OpenAI(api_key=api_key)

# ❌ This does expose the value:
log.info("api_key", key=settings.openai_api_key.get_secret_value())
# ✅ Only extract where needed, don't log it

Testing with injected configuration

# tests/conftest.py

import pytest
from unittest.mock import patch
from src.config import Settings, get_settings

@pytest.fixture
def test_settings() -> Settings:
    """Settings for tests: always mock, never calls the real LLM."""
    return Settings(
        environment="testing",
        openai_api_key="sk-test-key-not-real",
        use_mock_llm=True,
        log_level="WARNING",  # Less noise in tests
        model="gpt-4o-mini",
        temperature=0.0,
        max_tokens=500
    )

@pytest.fixture
def override_settings(test_settings: Settings):
    """
    Override of get_settings() for all tests.
    Avoids reading the real .env during tests.
    """
    with patch("src.config.get_settings", return_value=test_settings):
        # Clear the lru_cache cache
        get_settings.cache_clear()
        yield test_settings
    get_settings.cache_clear()

# Usage in tests:
def test_something_with_config(override_settings):
    settings = get_settings()  # Returns test_settings, doesn't read .env
    assert settings.use_mock_llm is True
    assert settings.environment == "testing"

# Alternative: inline override with monkeypatch
def test_with_monkeypatch(monkeypatch):
    monkeypatch.setenv("MODEL", "gpt-4o")
    monkeypatch.setenv("TEMPERATURE", "0.5")
    get_settings.cache_clear()
    settings = Settings()  # Reads the env vars from monkeypatch
    assert settings.model == "gpt-4o"
    assert settings.temperature == 0.5

Exercises

Exercise 1: Add fields to Settings

Add the following fields to your Settings with reasonable default values:

1. max_input_length: int — maximum characters of the user input
2. enable_guardrails: bool — enable or disable the guardrails
3. fallback_model: str — alternative model if the primary one fails

class Settings(BaseSettings):
    # ...
    max_input_length: int = 10_000      # 10K chars is reasonable for analysis
    enable_guardrails: bool = True      # Always on by default
    fallback_model: str = "gpt-4o-mini" # Same model as fallback (cheaper)
    
    @field_validator("max_input_length")
    @classmethod
    def validate_max_input(cls, v: int) -> int:
        if v < 10:
            raise ValueError("max_input_length must be >= 10")
        if v > 100_000:
            raise ValueError("max_input_length > 100K may be very expensive")
        return v

Exercise 2: Detect configuration errors at startup

What's the problem with this code?

@app.post("/analyze")
async def analyze(body: AnalyzeRequest):
    api_key = os.getenv("OPENAI_API_KEY")  # Reads on every request
    if not api_key:
        raise HTTPException(500, "API key not configured")
    client = OpenAI(api_key=api_key)
    ...

# With Settings, the error is at startup:
settings = get_settings()  # ← ValidationError here if api_key is missing in prod
# The app never starts → the error is visible immediately

Exercise 3: Per-environment configuration

Write the .env.development and .env.production files for an app where:

• In dev: use mock LLM, DEBUG logging, $1/day budget
• In prod: real LLM, INFO logging, $50/day budget

# .env.development
ENVIRONMENT=development
USE_MOCK_LLM=true
LOG_LEVEL=DEBUG
DAILY_BUDGET_USD=1.0
TEMPERATURE=0.0
ENABLE_INJECTION_CHECK=false  # Less restrictive in dev to ease debugging

# .env.production
ENVIRONMENT=production
USE_MOCK_LLM=false
LOG_LEVEL=INFO
DAILY_BUDGET_USD=50.0
TEMPERATURE=0.0
ENABLE_INJECTION_CHECK=true
ENABLE_PII_REDACTION=true
ENABLE_CONTENT_FILTER=true
# OPENAI_API_KEY is set as an environment variable on the server, NOT in .env

Summary

• pydantic-settings centralizes all configuration in a Python model with types, defaults, and validation
• SecretStr for secrets: they don't appear in repr, logs, or automatic dumps
• model_validator at startup: detects incorrect configuration before receiving the first request
• Multiple .env files: .env as the base + .env.{environment} as the override
• lru_cache() in get_settings(): the config is read only once at startup, not on every request
• Tests: override_settings fixture to inject test config without reading the real .env

Additional resources

1. pydantic-settings Documentation — Complete official documentation
2. SecretStr in Pydantic — Types for secrets
3. 12-Factor App — Config — The philosophy behind config management
4. python-dotenv — For loading .env in Python
5. Pydantic Validators — field_validator and model_validator