3. Multi-environment Config Management

Overview

In this capsule you'll build a robust configuration system that handles three environments (development/LocalStack, staging/AWS, production/AWS) with typing, validation, and secrets separation. In the previous capsule you abstracted the environment. Now you'll give that abstraction structure: Pydantic Settings to type and validate, per-environment .env files, validation rules that prevent invalid configurations, and a secrets management pattern that doesn't expose credentials in configuration files.

Context: Environment variables for 2 environments seem simple. For 3+ environments (dev, staging, prod) with different credentials, endpoints, buckets, feature flags, and secrets, config management needs structure. "Add a .env" isn't a pattern — it's the first step. The complete pattern includes typing (not reading strings that should be ints), validation (not accepting an ENVIRONMENT=produccion if the valid value is production), coherent defaults, and secrets separation (not storing AWS keys in git).

Why Config Management Matters

The cost of poorly managed config

Real scenario — what happens without config management:

1. Developer changes ENVIRONMENT to "staging" to test
2. Forgets to change it back to "local"
3. Runs tests → they write data to the staging S3
4. Another developer reads that data as "real" in staging
5. It goes to production with test data → corruption

Another scenario:
1. Lambda in production has S3_BUCKET=ai-assets-staging
2. Typo in the config → reads staging data, not production
3. Users receive responses with test data
4. Nobody understands why — the code "is fine"

Config management prevents these scenarios with validation at load time, not at runtime when it's already too late.

What a config system needs for production

Pydantic Settings: Typed Config

The base settings class

"""config/settings.py — Typed settings with Pydantic."""

from pydantic_settings import BaseSettings
from pydantic import field_validator, model_validator
from typing import Optional
from enum import Enum


class EnvironmentName(str, Enum):
    LOCAL = "local"
    STAGING = "staging"
    PRODUCTION = "production"


class Settings(BaseSettings):
    """Complete application configuration.

    Loads values from:
    1. System environment variables
    2. .env file (if it exists)
    3. Defaults defined here
    """

    # --- Environment identity ---
    environment: EnvironmentName = EnvironmentName.LOCAL
    app_name: str = "ai-migration-app"
    app_version: str = "1.0.0"
    debug: bool = False

    # --- AWS / Cloud ---
    aws_region: str = "us-east-1"
    aws_endpoint_url: Optional[str] = None
    aws_access_key_id: Optional[str] = None
    aws_secret_access_key: Optional[str] = None

    # --- S3 ---
    s3_bucket: str = "ai-assets-local"
    s3_prefix_prompts: str = "prompts/"
    s3_prefix_documents: str = "documents/"
    s3_prefix_responses: str = "responses/"

    # --- Lambda ---
    lambda_function_name: str = "ai-processor-local"
    lambda_timeout: int = 120
    lambda_memory: int = 768

    # --- AI / LLM ---
    openai_api_key: Optional[str] = None
    openai_model: str = "gpt-4o-mini"
    openai_max_tokens: int = 1000
    openai_temperature: float = 0.3

    # --- Feature Flags ---
    feature_sagemaker_enabled: bool = False
    feature_advanced_logging: bool = False
    feature_cost_tracking: bool = False

    # --- Operational ---
    log_level: str = "INFO"
    request_timeout: int = 30
    max_retries: int = 3
    circuit_breaker_threshold: int = 5

    @field_validator("environment", mode="before")
    @classmethod
    def normalize_environment(cls, v):
        if isinstance(v, str):
            return v.lower().strip()
        return v

    @field_validator("log_level", mode="before")
    @classmethod
    def normalize_log_level(cls, v):
        if isinstance(v, str):
            v = v.upper().strip()
            valid = {"DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"}
            if v not in valid:
                raise ValueError(f"log_level must be one of {valid}, received: '{v}'")
        return v

    @model_validator(mode="after")
    def validate_environment_config(self):
        """Validates that the config is coherent for the environment."""
        if self.environment == EnvironmentName.LOCAL:
            if not self.aws_endpoint_url:
                self.aws_endpoint_url = "http://localhost:4566"
            if not self.aws_access_key_id:
                self.aws_access_key_id = "test"
                self.aws_secret_access_key = "test"

        if self.environment == EnvironmentName.PRODUCTION:
            if self.debug:
                raise ValueError(
                    "debug=True is not allowed in production"
                )
            if self.aws_endpoint_url:
                raise ValueError(
                    "aws_endpoint_url must not be set in production "
                    "(it could redirect traffic to LocalStack)"
                )

        return self

    @property
    def is_local(self) -> bool:
        return self.environment == EnvironmentName.LOCAL

    @property
    def is_aws(self) -> bool:
        return self.environment in (EnvironmentName.STAGING, EnvironmentName.PRODUCTION)

    @property
    def is_production(self) -> bool:
        return self.environment == EnvironmentName.PRODUCTION

    class Config:
        env_file = ".env"
        env_file_encoding = "utf-8"
        use_enum_values = True

Loading settings with a specific .env file

"""config/loader.py — Loads settings according to the environment."""

import os
from config.settings import Settings


def get_settings(env_name: str | None = None) -> Settings:
    """Loads settings from the .env of the given environment.

    Priority:
    1. System environment variables (always win)
    2. .env.{environment} file
    3. Defaults of the Settings class
    """
    env = env_name or os.environ.get("ENVIRONMENT", "local")
    env_file = f".env.{env}"

    if os.path.exists(env_file):
        return Settings(_env_file=env_file)

    return Settings()


# Singleton for use in the application
_settings: Settings | None = None


def get_cached_settings() -> Settings:
    """Returns cached settings (singleton)."""
    global _settings
    if _settings is None:
        _settings = get_settings()
    return _settings

Per-Environment .env Files

.env.local — Development with LocalStack

# .env.local — Local development with LocalStack
ENVIRONMENT=local
APP_NAME=ai-migration-app
DEBUG=true

# AWS / LocalStack
AWS_REGION=us-east-1
AWS_ENDPOINT_URL=http://localhost:4566
AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test

# S3
S3_BUCKET=ai-assets-local

# Lambda
LAMBDA_FUNCTION_NAME=ai-processor-local
LAMBDA_TIMEOUT=120
LAMBDA_MEMORY=768

# AI (use your real key to invoke OpenAI even locally)
OPENAI_API_KEY=sk-your-dev-key-here
OPENAI_MODEL=gpt-4o-mini
OPENAI_MAX_TOKENS=500
OPENAI_TEMPERATURE=0.3

# Feature flags
FEATURE_SAGEMAKER_ENABLED=false
FEATURE_ADVANCED_LOGGING=false
FEATURE_COST_TRACKING=false

# Operational
LOG_LEVEL=DEBUG
REQUEST_TIMEOUT=30
MAX_RETRIES=3
CIRCUIT_BREAKER_THRESHOLD=10

.env.staging — AWS Staging

# .env.staging — AWS Staging
ENVIRONMENT=staging
APP_NAME=ai-migration-app
DEBUG=true

# AWS (no endpoint_url → uses real AWS)
AWS_REGION=us-east-1
# Credentials via IAM role — do NOT put keys here

# S3
S3_BUCKET=ai-assets-staging-123456789012

# Lambda
LAMBDA_FUNCTION_NAME=ai-processor-staging
LAMBDA_TIMEOUT=120
LAMBDA_MEMORY=768

# AI
OPENAI_MODEL=gpt-4o-mini
OPENAI_MAX_TOKENS=800
OPENAI_TEMPERATURE=0.3
# OPENAI_API_KEY via secrets manager or system env var

# Feature flags
FEATURE_SAGEMAKER_ENABLED=true
FEATURE_ADVANCED_LOGGING=true
FEATURE_COST_TRACKING=true

# Operational
LOG_LEVEL=INFO
REQUEST_TIMEOUT=30
MAX_RETRIES=3
CIRCUIT_BREAKER_THRESHOLD=5

.env.production — AWS Production

# .env.production — AWS Production
ENVIRONMENT=production
APP_NAME=ai-migration-app
DEBUG=false

# AWS
AWS_REGION=us-east-1
# Credentials via IAM role exclusively

# S3
S3_BUCKET=ai-assets-prod-123456789012

# Lambda
LAMBDA_FUNCTION_NAME=ai-processor-prod
LAMBDA_TIMEOUT=60
LAMBDA_MEMORY=1024

# AI
OPENAI_MODEL=gpt-4o-mini
OPENAI_MAX_TOKENS=1000
OPENAI_TEMPERATURE=0.2
# OPENAI_API_KEY via AWS Secrets Manager

# Feature flags
FEATURE_SAGEMAKER_ENABLED=true
FEATURE_ADVANCED_LOGGING=true
FEATURE_COST_TRACKING=true

# Operational
LOG_LEVEL=WARNING
REQUEST_TIMEOUT=15
MAX_RETRIES=5
CIRCUIT_BREAKER_THRESHOLD=3

.gitignore — Never commit secrets

# .gitignore
.env
.env.local
.env.staging
.env.production

# But DO commit the templates:
# .env.example → template without real values

.env.example — Template for new developers

# .env.example — Copy as .env.{environment} and fill in the values
ENVIRONMENT=local
AWS_REGION=us-east-1
AWS_ENDPOINT_URL=http://localhost:4566
AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test
S3_BUCKET=ai-assets-local
OPENAI_API_KEY=sk-your-key-here
LOG_LEVEL=DEBUG

Per-Environment Config Validation

A validator that prevents costly mistakes

"""config/validators.py — Additional configuration validations."""

from config.settings import Settings, EnvironmentName


class ConfigValidationError(Exception):
    """Configuration validation error."""
    pass


def validate_config(settings: Settings) -> list[str]:
    """Validates the configuration and returns warnings.

    Raises ConfigValidationError for critical problems.
    """
    warnings = []

    # Production: must have secrets via IAM, not in .env
    if settings.is_production:
        if settings.aws_access_key_id and settings.aws_access_key_id != "test":
            raise ConfigValidationError(
                "Production must not have AWS_ACCESS_KEY_ID in config. "
                "Use IAM roles."
            )

    # Staging: verify it doesn't point to LocalStack
    if settings.environment == EnvironmentName.STAGING:
        if settings.aws_endpoint_url and "localhost" in settings.aws_endpoint_url:
            raise ConfigValidationError(
                "Staging points to localhost — is this intentional? "
                "Staging should use real AWS."
            )

    # OpenAI key: warning if missing in environments that need it
    if not settings.openai_api_key and settings.is_aws:
        warnings.append(
            "OPENAI_API_KEY not configured. AI inference will fail."
        )

    # Feature flags: warning if SageMaker enabled in local
    if settings.feature_sagemaker_enabled and settings.is_local:
        warnings.append(
            "FEATURE_SAGEMAKER_ENABLED=true in local. "
            "LocalStack has limited SageMaker support."
        )

    # Timeout: warning if too low in production
    if settings.is_production and settings.request_timeout < 10:
        warnings.append(
            f"REQUEST_TIMEOUT={settings.request_timeout}s is low for production. "
            f"Consider 15-30s."
        )

    # Lambda memory: warning if low for AI workloads
    if settings.lambda_memory < 512:
        warnings.append(
            f"LAMBDA_MEMORY={settings.lambda_memory}MB may be insufficient "
            f"for AI workloads. Recommended: 768MB+."
        )

    return warnings


def validate_and_report(settings: Settings):
    """Validates config and prints a report."""
    print(f"Validating configuration for: {settings.environment}")

    try:
        warnings = validate_config(settings)
    except ConfigValidationError as e:
        print(f"❌ CRITICAL ERROR: {e}")
        raise

    if warnings:
        for w in warnings:
            print(f"⚠️  {w}")
    else:
        print("✅ Valid configuration with no warnings")

Startup validation — fail fast

"""app_startup.py — Validation at application startup."""

import sys
from config.loader import get_settings
from config.validators import validate_and_report, ConfigValidationError


def startup():
    """Loads and validates configuration at startup."""
    try:
        settings = get_settings()
    except Exception as e:
        print(f"❌ Error loading configuration: {e}")
        sys.exit(1)

    try:
        validate_and_report(settings)
    except ConfigValidationError as e:
        print(f"❌ Invalid configuration. Cannot start.")
        sys.exit(1)

    print(f"\n{'='*50}")
    print(f"App: {settings.app_name} v{settings.app_version}")
    print(f"Environment: {settings.environment}")
    print(f"Region: {settings.aws_region}")
    print(f"Bucket: {settings.s3_bucket}")
    print(f"Debug: {settings.debug}")
    print(f"{'='*50}\n")

    return settings


if __name__ == "__main__":
    startup()

Secrets Management

The problem: secrets in configuration files

The golden rule: if a compromised value causes harm, it's a secret.

Secrets:
├── AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY → access to your AWS account
├── OPENAI_API_KEY → consumption of your OpenAI credit
└── DATABASE_PASSWORD → access to your data

NOT secrets:
├── AWS_REGION → it's public (us-east-1)
├── S3_BUCKET → the name doesn't grant access (IAM does)
├── LOG_LEVEL → no harm if exposed
└── LAMBDA_TIMEOUT → operational configuration

Pattern for per-environment secrets

"""config/secrets.py — Per-environment secrets management."""

import os
import json
from config.settings import Settings


class SecretsProvider:
    """Provides secrets according to the environment."""

    def __init__(self, settings: Settings):
        self.settings = settings

    def get_openai_key(self) -> str:
        """Gets the OpenAI API key."""
        # Priority 1: direct environment variable
        key = os.environ.get("OPENAI_API_KEY")
        if key:
            return key

        # Priority 2: on AWS, use Secrets Manager
        if self.settings.is_aws:
            return self._get_from_secrets_manager("openai-api-key")

        # Priority 3: read from .env (already loaded in settings)
        if self.settings.openai_api_key:
            return self.settings.openai_api_key

        raise ValueError("OPENAI_API_KEY not configured in any source")

    def _get_from_secrets_manager(self, secret_name: str) -> str:
        """Reads a secret from AWS Secrets Manager."""
        import boto3

        kwargs = {"region_name": self.settings.aws_region}
        if self.settings.aws_endpoint_url:
            kwargs["endpoint_url"] = self.settings.aws_endpoint_url

        client = boto3.client("secretsmanager", **kwargs)
        response = client.get_secret_value(SecretId=secret_name)

        secret_string = response["SecretString"]
        try:
            secret_dict = json.loads(secret_string)
            return secret_dict.get("api_key", secret_string)
        except json.JSONDecodeError:
            return secret_string

Create a secret in LocalStack (for testing)

import boto3

sm = boto3.client(
    "secretsmanager",
    endpoint_url="http://localhost:4566",
    aws_access_key_id="test",
    aws_secret_access_key="test",
    region_name="us-east-1",
)

sm.create_secret(
    Name="openai-api-key",
    SecretString=json.dumps({"api_key": "sk-test-key-for-localstack"}),
)

response = sm.get_secret_value(SecretId="openai-api-key")
print(f"Secret: {response['SecretString']}")

Config Comparison Tool

A tool to compare configs between environments

"""config/compare.py — Compares configuration between environments."""

from config.settings import Settings, EnvironmentName


def compare_configs(env_a: str, env_b: str) -> dict:
    """Compares the configuration of two environments."""
    settings_a = Settings(environment=env_a, _env_file=f".env.{env_a}")
    settings_b = Settings(environment=env_b, _env_file=f".env.{env_b}")

    fields_a = settings_a.model_dump()
    fields_b = settings_b.model_dump()

    differences = {}
    for key in fields_a:
        val_a = fields_a[key]
        val_b = fields_b[key]
        if val_a != val_b:
            differences[key] = {"env_a": val_a, "env_b": val_b}

    return differences


def print_config_diff(env_a: str = "local", env_b: str = "staging"):
    """Prints the configuration differences between two environments."""
    diffs = compare_configs(env_a, env_b)

    print(f"\n{'='*60}")
    print(f"CONFIG DIFF: {env_a} vs {env_b}")
    print(f"{'='*60}")

    if not diffs:
        print("No differences (same .env?)")
        return

    # Separate secrets from normal config
    secret_keys = {"aws_access_key_id", "aws_secret_access_key", "openai_api_key"}

    for key, vals in sorted(diffs.items()):
        if key in secret_keys:
            print(f"\n  🔒 {key}:")
            print(f"     {env_a}: {'*****' if vals['env_a'] else 'Not set'}")
            print(f"     {env_b}: {'*****' if vals['env_b'] else 'Not set'}")
        else:
            print(f"\n  📋 {key}:")
            print(f"     {env_a}: {vals['env_a']}")
            print(f"     {env_b}: {vals['env_b']}")

    print(f"\n  Total differences: {len(diffs)}")
    print(f"{'='*60}")

Example output:

============================================================
CONFIG DIFF: local vs staging
============================================================

  📋 environment:
     local: local
     staging: staging

  📋 debug:
     local: True
     staging: True

  📋 aws_endpoint_url:
     local: http://localhost:4566
     staging: None

  🔒 aws_access_key_id:
     local: *****
     staging: Not set

  📋 s3_bucket:
     local: ai-assets-local
     staging: ai-assets-staging-123456789012

  📋 feature_sagemaker_enabled:
     local: False
     staging: True

  📋 log_level:
     local: DEBUG
     staging: INFO

  Total differences: 12
============================================================

Troubleshooting

Problem 1: Pydantic ValidationError at startup

A value in .env doesn't match the expected type.

# .env has:
# LAMBDA_TIMEOUT=abc  ← should be int

# Error:
# ValidationError: 1 validation error for Settings
# lambda_timeout
#   Input should be a valid integer [type=int_parsing, ...]

# Solution: check the types in .env
# LAMBDA_TIMEOUT=120  ← correct

Problem 2: Settings ignores the .env file

Pydantic Settings prioritizes system environment variables over the .env.

# If you have this in the shell:
export ENVIRONMENT=production

# And this in .env.local:
ENVIRONMENT=local

# Pydantic uses "production" (system env var wins)

# Solution: unset the system variable
unset ENVIRONMENT
# Or be explicit in the loader:
# Settings(_env_file=".env.local")

Problem 3: Production config has debug=True

The validate_environment_config validator should prevent it, but if it's bypassed:

# The Settings class has:
# @model_validator(mode="after")
# def validate_environment_config(self):
#     if self.environment == "production" and self.debug:
#         raise ValueError("debug=True is not allowed in production")

# This prevents starting with invalid config.
# If the error persists, check that .env.production doesn't have DEBUG=true.

Problem 4: Staging secrets accessible in local

This happens when the developer has AWS credentials in ~/.aws/credentials and doesn't set AWS_ENDPOINT_URL:

# The client without endpoint_url → talks to real AWS
# Even if ENVIRONMENT=local

# Solution: the Settings validator sets endpoint_url automatically in local:
# if self.environment == "local" and not self.aws_endpoint_url:
#     self.aws_endpoint_url = "http://localhost:4566"

Problem 5: .env.example is out of date

Add a script that verifies that .env.example has the same keys as the Settings class:

from config.settings import Settings

settings_keys = set(Settings.model_fields.keys())

with open(".env.example") as f:
    example_keys = set()
    for line in f:
        line = line.strip()
        if line and not line.startswith("#") and "=" in line:
            key = line.split("=")[0].strip().lower()
            example_keys.add(key)

missing = settings_keys - example_keys
extra = example_keys - settings_keys

if missing:
    print(f"⚠️ Keys in Settings but not in .env.example: {missing}")
if extra:
    print(f"⚠️ Keys in .env.example but not in Settings: {extra}")
if not missing and not extra:
    print("✅ .env.example in sync with Settings")

Practical Exercises

Exercise 1: Settings with custom validation

Extend the Settings class with a max_document_size_mb field (int, default 10) and a validator that rejects values greater than 50 in production and greater than 100 in other environments.

from pydantic_settings import BaseSettings
from pydantic import field_validator, model_validator
from typing import Optional
from enum import Enum


class EnvironmentName(str, Enum):
    LOCAL = "local"
    STAGING = "staging"
    PRODUCTION = "production"


class Settings(BaseSettings):
    environment: EnvironmentName = EnvironmentName.LOCAL
    aws_region: str = "us-east-1"
    aws_endpoint_url: Optional[str] = None
    s3_bucket: str = "ai-assets-local"
    max_document_size_mb: int = 10

    @model_validator(mode="after")
    def validate_document_size(self):
        if self.environment == EnvironmentName.PRODUCTION:
            if self.max_document_size_mb > 50:
                raise ValueError(
                    f"max_document_size_mb={self.max_document_size_mb} "
                    f"exceeds the 50MB limit for production"
                )
        else:
            if self.max_document_size_mb > 100:
                raise ValueError(
                    f"max_document_size_mb={self.max_document_size_mb} "
                    f"exceeds the 100MB limit"
                )
        return self

    class Config:
        env_file = ".env"
        use_enum_values = True


# Test: production with 60MB → fails
try:
    s = Settings(environment="production", max_document_size_mb=60)
except ValueError as e:
    print(f"✅ Correct validation: {e}")

# Test: local with 80MB → OK
s = Settings(environment="local", max_document_size_mb=80)
print(f"✅ Local with 80MB: OK")

# Test: local with 150MB → fails
try:
    s = Settings(environment="local", max_document_size_mb=150)
except ValueError as e:
    print(f"✅ Correct validation: {e}")

Exercise 2: Config loader with fallback chain

Implement a config loader that tries to load in this order: .env.{env}.local (local overrides) → .env.{env} → .env → defaults. The first file that exists is used.

import os
from pydantic_settings import BaseSettings
from typing import Optional


class Settings(BaseSettings):
    environment: str = "local"
    aws_region: str = "us-east-1"
    aws_endpoint_url: Optional[str] = None
    s3_bucket: str = "ai-assets-local"
    debug: bool = False

    class Config:
        env_file = ".env"
        use_enum_values = True


def load_settings_with_fallback(env_name: str | None = None) -> Settings:
    """Loads settings with a fallback chain of .env files."""
    env = env_name or os.environ.get("ENVIRONMENT", "local")

    candidates = [
        f".env.{env}.local",  # Developer's local overrides
        f".env.{env}",        # Environment config
        ".env",               # General fallback
    ]

    env_file_used = None
    for candidate in candidates:
        if os.path.exists(candidate):
            env_file_used = candidate
            break

    if env_file_used:
        print(f"📁 Config loaded from: {env_file_used}")
        return Settings(_env_file=env_file_used)
    else:
        print(f"📁 Config loaded from: defaults (no .env found)")
        return Settings()


settings = load_settings_with_fallback()
print(f"Environment: {settings.environment}")
print(f"Bucket: {settings.s3_bucket}")
print(f"Debug: {settings.debug}")

Exercise 3: Config export/import

Create functions to export the current configuration (without secrets) to a JSON file, and import configuration from a JSON to compare or restore.

import json
from datetime import datetime
from config.settings import Settings


SECRET_FIELDS = {"aws_access_key_id", "aws_secret_access_key", "openai_api_key"}


def export_config(settings: Settings, filepath: str) -> str:
    """Exports the current configuration to JSON (without secrets)."""
    config_dict = settings.model_dump()

    for field in SECRET_FIELDS:
        if field in config_dict and config_dict[field]:
            config_dict[field] = "***REDACTED***"

    export = {
        "exported_at": datetime.utcnow().isoformat(),
        "environment": settings.environment,
        "config": config_dict,
    }

    with open(filepath, "w") as f:
        json.dump(export, f, indent=2, default=str)

    print(f"✅ Config exported to: {filepath}")
    print(f"   Environment: {settings.environment}")
    print(f"   Fields: {len(config_dict)}")
    print(f"   Redacted secrets: {len(SECRET_FIELDS)}")
    return filepath


def import_and_compare(filepath: str, current: Settings) -> dict:
    """Imports config from JSON and compares it with the current one."""
    with open(filepath) as f:
        imported = json.load(f)

    imported_config = imported["config"]
    current_config = current.model_dump()

    differences = {}
    for key in current_config:
        imported_val = imported_config.get(key)
        current_val = current_config[key]

        if key in SECRET_FIELDS:
            continue

        if imported_val != current_val:
            differences[key] = {
                "imported": imported_val,
                "current": current_val,
            }

    print(f"\n📊 Comparison: {filepath} vs current config")
    print(f"   Imported from: {imported.get('exported_at', 'unknown')}")
    if differences:
        for key, vals in differences.items():
            print(f"   ≠ {key}: {vals['imported']} → {vals['current']}")
    else:
        print("   ✅ No differences (excluding secrets)")

    return differences


settings = Settings()
export_config(settings, "config-backup.json")
diffs = import_and_compare("config-backup.json", settings)

Exercise 4: Environment-aware logging config

Create a function that configures Python logging according to the environment: DEBUG with a detailed format in local, INFO with a JSON format in staging, WARNING with a compact JSON format in production.

import logging
import json
from datetime import datetime
from config.settings import Settings, EnvironmentName


class JSONFormatter(logging.Formatter):
    """Formatter that produces JSON for cloud environments."""

    def __init__(self, compact: bool = False):
        super().__init__()
        self.compact = compact

    def format(self, record):
        log_data = {
            "timestamp": datetime.utcnow().isoformat(),
            "level": record.levelname,
            "message": record.getMessage(),
            "module": record.module,
        }
        if not self.compact:
            log_data["function"] = record.funcName
            log_data["line"] = record.lineno
        if record.exc_info and record.exc_info[0]:
            log_data["exception"] = self.formatException(record.exc_info)
        return json.dumps(log_data)


def configure_logging(settings: Settings) -> logging.Logger:
    """Configures logging according to the environment."""
    logger = logging.getLogger(settings.app_name)
    logger.handlers.clear()
    logger.setLevel(getattr(logging, settings.log_level))

    handler = logging.StreamHandler()

    if settings.is_local:
        formatter = logging.Formatter(
            "%(asctime)s [%(levelname)s] %(name)s.%(funcName)s:%(lineno)d — %(message)s",
            datefmt="%H:%M:%S",
        )
    elif settings.is_production:
        formatter = JSONFormatter(compact=True)
    else:
        formatter = JSONFormatter(compact=False)

    handler.setFormatter(formatter)
    logger.addHandler(handler)

    logger.info(f"Logging configured: level={settings.log_level}, "
                f"env={settings.environment}")
    return logger


# Demo
for env in ["local", "staging", "production"]:
    print(f"\n--- Environment: {env} ---")
    try:
        s = Settings(
            environment=env,
            log_level="DEBUG" if env == "local" else "INFO" if env == "staging" else "WARNING",
            debug=(env == "local"),
        )
        logger = configure_logging(s)
        logger.debug("Debug message")
        logger.info("Info message")
        logger.warning("Warning message")
    except Exception as e:
        print(f"  Config error: {e}")

Summary

• Config management is not "add a .env." It's typing with Pydantic, per-environment validation, secrets separation, and environment-specific .env files.
• Pydantic Settings is your source of truth. The class defines what config exists, what type it has, what its default is, and what validations apply. The class IS the documentation.
• Three .env files: .env.local (LocalStack), .env.staging (AWS staging), .env.production (AWS prod). Never in git. Yes to a .env.example as a template.
• Validators that protect: debug=True in production → error. endpoint_url in production → error. SageMaker enabled in local → warning. Failing fast is better than failing at runtime.
• Secrets go separately. In local, in .env (outside git). On AWS, via IAM roles or Secrets Manager. Never hardcoded in code.
• In the next capsule, you'll use this configuration to inject boto3 clients with dependency injection.

Additional Resources

1. Pydantic Settings Management — Official documentation
2. The Twelve-Factor App — Config — Configuration principles
3. python-dotenv — Loading .env files
4. AWS Secrets Manager — Secrets management on AWS
5. Pydantic Validators — Custom validation in Pydantic
6. OWASP — Secrets Management — Secrets best practices