4. Deployment Checklist de Seguridad

Descripción

Un sistema AI que pasa todos los tests en staging puede ser vulnerable en producción si el deploy no se ejecuta correctamente. Una API key hardcodeada que pasó el code review, un rate limiter que no se habilitó, un PII scanner que solo corre en dev — estos son errores de deploy, no errores de código. El deployment checklist de seguridad es la última línea de defensa.

Los deploy checklists para sistemas AI son diferentes a los de aplicaciones web tradicionales. Además de los clásicos (HTTPS, CORS, secrets), necesitas verificar que las defensas de AI estén activas: ¿el system prompt está hardened? ¿El injection detector está en el pipeline de producción? ¿El PII scanner procesa outputs reales? Estas verificaciones no existen en checklists genéricos.

Esta cápsula te guía por la creación de un checklist de 50+ items organizado por categoría, con verificaciones automatizadas donde sea posible y verificaciones manuales donde sea necesario. Al final tendrás un DeploymentChecklist ejecutable que genera un reporte pass/fail y se integra en tu pipeline CI/CD como gate pre-deploy.

Checklist por categoría

Secrets & Credentials

API Security

Input/Output Security

PII Protection

Monitoring & Observability

Infrastructure

LLM-Specific

DeploymentChecklist class

El DeploymentChecklist modela cada item como verificación pass/fail. Las automáticas se ejecutan programáticamente; las manuales requieren confirmación humana.

from pydantic import BaseModel, Field, computed_field
from enum import Enum
from datetime import datetime
from typing import Optional, Callable

class CheckType(str, Enum):
    AUTOMATIC = "automatic"
    MANUAL = "manual"

class CheckStatus(str, Enum):
    PENDING = "pending"
    PASS = "pass"
    FAIL = "fail"
    SKIP = "skip"

class CheckCategory(str, Enum):
    SECRETS = "secrets"
    API_SECURITY = "api_security"
    INPUT_OUTPUT = "input_output"
    PII = "pii"
    MONITORING = "monitoring"
    INFRASTRUCTURE = "infrastructure"
    LLM_SPECIFIC = "llm_specific"

class ChecklistItem(BaseModel):
    id: str
    category: CheckCategory
    description: str
    check_type: CheckType
    status: CheckStatus = CheckStatus.PENDING
    verification_method: str
    notes: str = ""
    verified_by: str = ""
    verified_at: Optional[datetime] = None
    environments: list[str] = Field(default_factory=lambda: ["staging", "production"])

class ChecklistReport(BaseModel):
    project_name: str
    environment: str
    run_date: datetime
    total_items: int
    passed: int
    failed: int
    pending: int
    pass_rate: float
    deploy_approved: bool
    failed_items: list[dict] = Field(default_factory=list)
    summary: str = ""

class DeploymentChecklist(BaseModel):
    """Checklist con verificación automática/manual y gate pre-deploy."""
    project_name: str
    environment: str = "production"
    items: list[ChecklistItem] = Field(default_factory=list)
    auto_checks: dict[str, Callable[[], bool]] = Field(default_factory=dict, exclude=True)

    class Config:
        arbitrary_types_allowed = True

    def add_item(self, item: ChecklistItem) -> None:
        self.items.append(item)

    def register_auto_check(self, item_id: str, check_fn: Callable[[], bool]) -> None:
        self.auto_checks[item_id] = check_fn

    def run_automatic_checks(self) -> int:
        """Ejecuta verificaciones automáticas registradas."""
        executed = 0
        for item in self.items:
            if item.check_type != CheckType.AUTOMATIC or item.id not in self.auto_checks:
                continue
            if self.environment not in item.environments:
                item.status = CheckStatus.SKIP
                continue
            try:
                result = self.auto_checks[item.id]()
                item.status = CheckStatus.PASS if result else CheckStatus.FAIL
                item.verified_at = datetime.now()
                item.verified_by = "automated"
                executed += 1
            except Exception as e:
                item.status = CheckStatus.FAIL
                item.notes = f"Check error: {e}"
                executed += 1
        return executed

    def mark_manual(self, item_id: str, passed: bool, verified_by: str, notes: str = "") -> None:
        for item in self.items:
            if item.id == item_id:
                item.status = CheckStatus.PASS if passed else CheckStatus.FAIL
                item.verified_by = verified_by
                item.verified_at = datetime.now()
                item.notes = notes

    def generate_report(self) -> ChecklistReport:
        applicable = [i for i in self.items if self.environment in i.environments]
        total = len(applicable)
        passed = sum(1 for i in applicable if i.status == CheckStatus.PASS)
        failed = sum(1 for i in applicable if i.status == CheckStatus.FAIL)
        pending = sum(1 for i in applicable if i.status == CheckStatus.PENDING)
        pass_rate = (passed / total * 100) if total > 0 else 0
        # Deploy solo aprobado sin failures ni pending
        deploy_ok = (failed == 0 and pending == 0)
        failed_items = [{"id": i.id, "description": i.description, "notes": i.notes} for i in applicable if i.status == CheckStatus.FAIL]

        summary = (f"✅ DEPLOY APPROVED — all {total} checks passed" if deploy_ok
                   else f"❌ DEPLOY BLOCKED — {failed} failed" if failed > 0
                   else f"⏳ DEPLOY PENDING — {pending} not verified")

        return ChecklistReport(
            project_name=self.project_name, environment=self.environment,
            run_date=datetime.now(), total_items=total, passed=passed,
            failed=failed, pending=pending, pass_rate=pass_rate,
            deploy_approved=deploy_ok, failed_items=failed_items, summary=summary,
        )

    def print_report(self) -> None:
        r = self.generate_report()
        print(f"{'='*55}")
        print(f"Deployment Checklist: {r.project_name} ({r.environment})")
        print(f"{'='*55}")
        print(f"Total: {r.total_items} | Pass: {r.passed} | Fail: {r.failed} | Pending: {r.pending}")
        print(f"Pass rate: {r.pass_rate:.0f}%\n{r.summary}")
        if r.failed_items:
            print("\nFailed:")
            for item in r.failed_items:
                print(f"  ❌ {item['id']}: {item['description']}")

Verificación automatizada

Las verificaciones automáticas eliminan error humano en los checks más críticos.

import os
import subprocess
from pathlib import Path
import re

class AutomatedSecurityChecks:
    """Checks automatizados de configuración de seguridad."""

    def __init__(self, project_root: str = "."):
        self.root = Path(project_root)

    def check_no_env_committed(self) -> bool:
        """Verifica que .env no está trackeado por git."""
        gitignore = self.root / ".gitignore"
        if not gitignore.exists():
            return False
        content = gitignore.read_text()
        env_ignored = any(
            line.strip() in (".env", ".env*", "*.env")
            for line in content.splitlines() if not line.strip().startswith("#")
        )
        if not env_ignored:
            return False
        try:
            result = subprocess.run(
                ["git", "ls-files", ".env"], capture_output=True, text=True, cwd=self.root,
            )
            return len(result.stdout.strip()) == 0
        except FileNotFoundError:
            return True

    def check_no_secrets_in_code(self) -> bool:
        """Busca secrets hardcodeados. Retorna True si no encuentra."""
        patterns = [
            r"sk-[a-zA-Z0-9]{20,}",
            r"AKIA[0-9A-Z]{16}",
            r"(?i)password\s*=\s*['\"][^'\"]+['\"]",
        ]
        skip_dirs = {"venv", "node_modules", ".git", "__pycache__"}
        for py_file in self.root.rglob("*.py"):
            if any(d in py_file.parts for d in skip_dirs):
                continue
            try:
                content = py_file.read_text(errors="ignore")
                for p in patterns:
                    if re.search(p, content):
                        print(f"  ⚠ Secret in: {py_file.relative_to(self.root)}")
                        return False
            except (PermissionError, OSError):
                continue
        return True

    def check_rate_limiting(self) -> bool:
        """Verifica que existe configuración de rate limiting."""
        indicators = ["rate_limit", "ratelimit", "throttle", "RATE_LIMIT"]
        for f in self.root.rglob("*.py"):
            if any(d in f.parts for d in ("venv", ".git")):
                continue
            try:
                content = f.read_text(errors="ignore")
                if any(ind in content for ind in indicators):
                    return True
            except (PermissionError, OSError):
                continue
        return False

    def check_pii_scanner_active(self) -> bool:
        """Verifica que el PII scanner está en el pipeline."""
        indicators = ["PIIScanner", "pii_scan", "presidio", "PostLLMPIIScanner"]
        for f in self.root.rglob("*.py"):
            if any(d in f.parts for d in ("venv", ".git", "test")):
                continue
            try:
                if any(ind in f.read_text(errors="ignore") for ind in indicators):
                    return True
            except (PermissionError, OSError):
                continue
        return False

    def run_all(self) -> dict[str, bool]:
        checks = {
            "no_env_committed": self.check_no_env_committed,
            "no_secrets_in_code": self.check_no_secrets_in_code,
            "rate_limiting": self.check_rate_limiting,
            "pii_scanner_active": self.check_pii_scanner_active,
        }
        results = {}
        for name, fn in checks.items():
            try:
                results[name] = fn()
            except Exception as e:
                print(f"  ⚠ '{name}' error: {e}")
                results[name] = False
        return results

checker = AutomatedSecurityChecks(".")
print("=== Automated Security Checks ===\n")
for name, passed in checker.run_all().items():
    print(f"{'✅' if passed else '❌'} {name}")

Verificación manual

Algunos items requieren juicio humano. Se documentan con criterios claros para el revisor.

from pydantic import BaseModel, Field
from datetime import datetime

class ManualVerification(BaseModel):
    item_id: str
    description: str
    reviewer: str
    review_date: datetime
    passed: bool
    evidence: str
    notes: str = ""

class ManualCheckRegistry(BaseModel):
    """Registro de verificaciones manuales con accountability."""
    verifications: list[ManualVerification] = Field(default_factory=list)

    def record(self, item_id: str, description: str, reviewer: str, passed: bool, evidence: str, notes: str = "") -> None:
        self.verifications.append(ManualVerification(
            item_id=item_id, description=description, reviewer=reviewer,
            review_date=datetime.now(), passed=passed, evidence=evidence, notes=notes,
        ))

    def pending(self, all_ids: list[str]) -> list[str]:
        verified = {v.item_id for v in self.verifications}
        return [mid for mid in all_ids if mid not in verified]

    def summary(self) -> str:
        lines = ["=== Manual Verifications ===", ""]
        for v in self.verifications:
            icon = "✅" if v.passed else "❌"
            lines.append(f"{icon} {v.item_id}: {v.description}")
            lines.append(f"   By {v.reviewer} on {v.review_date.strftime('%Y-%m-%d')}")
            lines.append(f"   Evidence: {v.evidence}")
            if v.notes:
                lines.append(f"   Notes: {v.notes}")
        return "\n".join(lines)

registry = ManualCheckRegistry()
registry.record("IO-02", "System prompt hardened", "Ana García", True, "Prompt v2.3 reviewed")
registry.record("M-07", "Incident response runbook", "Diana Ruiz", False, "Missing weekend escalation contacts", "Update by 2026-03-20")
print(registry.summary())

Pre-launch vs Post-launch checks

from pydantic import BaseModel
from enum import Enum

class CheckPhase(str, Enum):
    PRE_LAUNCH = "pre_launch"
    POST_LAUNCH = "post_launch"
    BOTH = "both"

class PhaseCheck(BaseModel):
    id: str
    description: str
    phase: CheckPhase
    blocks_deploy: bool

checks = [
    PhaseCheck(id="PRE-01", description="Adversarial test suite passes", phase=CheckPhase.PRE_LAUNCH, blocks_deploy=True),
    PhaseCheck(id="PRE-02", description="No secrets in source code", phase=CheckPhase.PRE_LAUNCH, blocks_deploy=True),
    PhaseCheck(id="PRE-03", description="PII scanner tests pass", phase=CheckPhase.PRE_LAUNCH, blocks_deploy=True),
    PhaseCheck(id="POST-01", description="No PII in production logs", phase=CheckPhase.POST_LAUNCH, blocks_deploy=False),
    PhaseCheck(id="POST-02", description="Rate limiting works under load", phase=CheckPhase.POST_LAUNCH, blocks_deploy=False),
    PhaseCheck(id="POST-03", description="Injection alerts fire correctly", phase=CheckPhase.POST_LAUNCH, blocks_deploy=False),
    PhaseCheck(id="BOTH-01", description="Health check returns 200", phase=CheckPhase.BOTH, blocks_deploy=True),
    PhaseCheck(id="BOTH-02", description="Error rate < 5%", phase=CheckPhase.BOTH, blocks_deploy=True),
]

for phase in CheckPhase:
    phase_checks = [c for c in checks if c.phase == phase]
    print(f"\n[{phase.value.upper()}]")
    for c in phase_checks:
        tag = "🚫 BLOCKER" if c.blocks_deploy else "📋 Advisory"
        print(f"  {c.id}: {c.description} — {tag}")

Checklist por entorno

No todos los checks aplican a todos los entornos.

from pydantic import BaseModel, Field

class EnvironmentConfig(BaseModel):
    name: str
    required_checks: list[str]
    optional_checks: list[str]
    strict_mode: bool

def build_env_configs() -> dict[str, EnvironmentConfig]:
    return {
        "development": EnvironmentConfig(
            name="Development", strict_mode=False,
            required_checks=["S-01", "IO-01", "P-01"],
            optional_checks=["A-03", "M-01"],
        ),
        "staging": EnvironmentConfig(
            name="Staging", strict_mode=True,
            required_checks=["S-01", "S-02", "S-06", "A-01", "A-02", "A-03", "A-04", "IO-01", "IO-03", "P-01", "P-02", "P-03", "L-01", "L-07"],
            optional_checks=["M-02", "M-04", "I-03"],
        ),
        "production": EnvironmentConfig(
            name="Production", strict_mode=True,
            required_checks=[
                "S-01", "S-02", "S-03", "S-04", "S-05", "S-06", "S-07",
                "A-01", "A-02", "A-03", "A-04", "A-05", "A-06", "A-07",
                "IO-01", "IO-02", "IO-03", "IO-04", "IO-05", "IO-06", "IO-07",
                "P-01", "P-02", "P-03", "P-04", "P-05", "P-06", "P-07",
                "M-01", "M-02", "M-03", "M-04", "M-05", "M-06", "M-07",
                "I-01", "I-02", "I-03", "I-04", "I-05", "I-06", "I-07",
                "L-01", "L-02", "L-03", "L-04", "L-05", "L-06", "L-07", "L-08",
            ],
            optional_checks=[],
        ),
    }

for name, cfg in build_env_configs().items():
    print(f"{cfg.name}: {len(cfg.required_checks)} required, {len(cfg.optional_checks)} optional, strict={'ON' if cfg.strict_mode else 'OFF'}")

# Output esperado:
# Development: 3 required, 2 optional, strict=OFF
# Staging: 14 required, 3 optional, strict=ON
# Production: 57 required, 0 optional, strict=ON

Integración con CI/CD

El checklist se ejecuta automáticamente en GitHub Actions como gate pre-deploy.

# .github/workflows/security-checklist.yml
name: Security Deployment Checklist

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  security-checklist:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install dependencies
        run: pip install pip-audit pydantic

      - name: Check for secrets in code
        run: |
          if grep -rn "sk-[a-zA-Z0-9]\{20,\}" --include="*.py" .; then
            echo "::error::Hardcoded API keys found"
            exit 1
          fi

      - name: Check .env not tracked
        run: |
          if git ls-files .env | grep -q ".env"; then
            echo "::error::.env tracked by git"
            exit 1
          fi

      - name: Dependency vulnerability scan
        run: pip-audit || true

      - name: Run adversarial test suite
        run: python -m pytest tests/security/ -v --tb=short

      - name: Run deployment checklist
        run: python scripts/run_deploy_checklist.py --env production

      - name: Upload report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: security-checklist-report
          path: checklist-report.json

El script que el workflow ejecuta:

import sys
import json
from datetime import datetime
from pathlib import Path

def run_deploy_checklist(environment: str = "production") -> bool:
    """Ejecuta el checklist y retorna True si pasa."""
    print(f"Running checklist for: {environment}")
    results: dict[str, bool] = {}
    failures: list[str] = []

    # .gitignore incluye .env
    gitignore = Path(".gitignore")
    check = gitignore.exists() and ".env" in gitignore.read_text()
    results["env_in_gitignore"] = check
    if not check:
        failures.append("S-01: .env not in .gitignore")

    # Security tests existen
    check = len(list(Path(".").rglob("tests/security/*.py"))) > 0
    results["security_tests_exist"] = check
    if not check:
        failures.append("L-07: No security tests found")

    # System prompt versionado
    prompt_files = list(Path(".").rglob("*system_prompt*")) + list(Path(".").rglob("prompts/*.txt"))
    check = len(prompt_files) > 0
    results["system_prompt_versioned"] = check
    if not check:
        failures.append("L-05: No system prompt file found")

    # Guardar reporte
    report = {
        "environment": environment,
        "timestamp": datetime.now().isoformat(),
        "results": results,
        "failures": failures,
        "passed": len(failures) == 0,
    }
    with open("checklist-report.json", "w") as f:
        json.dump(report, f, indent=2)

    for name, ok in results.items():
        print(f"{'✅' if ok else '❌'} {name}")

    if failures:
        print(f"\n❌ BLOCKED — Fix: {'; '.join(failures)}")
        return False
    print("\n✅ DEPLOY APPROVED")
    return True

if __name__ == "__main__":
    env = "production"
    if "--env" in sys.argv:
        idx = sys.argv.index("--env")
        env = sys.argv[idx + 1] if idx + 1 < len(sys.argv) else "production"
    sys.exit(0 if run_deploy_checklist(env) else 1)

Troubleshooting

1. El checklist automático pasa en local pero falla en CI

Problema: Los checks dependen de archivos o configuraciones que existen localmente pero no en CI.

Solución: Asegura que todo lo necesario para los checks esté en el repositorio o se genere en el pipeline. Usa variables de entorno de CI para configuraciones del entorno. Para checks que requieren servicios externos (vault, cloud), usa mocks en CI.

2. Falsos positivos en el secret scanner

Problema: El scanner detecta strings que parecen API keys pero son test fixtures o hashes legítimos.

Solución: Mantén un .secret-scan-ignore con excepciones documentadas. Refina los regex para prefijos específicos del proveedor. Nunca deshabilites el scanner completo por falsos positivos.

3. Los checks manuales se saltan por presión de tiempo

Problema: El equipo marca checks como "passed" sin verificar porque hay presión para deployar.

Solución: Agrega accountability: cada check manual requiere reviewer y evidencia específica. Resalta items sin evidencia. Implementa vencimiento de 30 días para re-verificación.

4. El checklist de producción tiene demasiados items

Problema: Con 57+ items, el equipo pierde motivación para completarlo.

Solución: Automatiza todo lo posible — objetivo: ≤ 15 checks manuales. Agrupa por owner para que nadie vea los 57, solo los suyos. Después de los primeros 3 deploys, solo re-verifica items que cambiaron.

5. El pipeline CI tarda demasiado por security checks

Problema: Los checks agregan 10+ minutos al pipeline.

Solución: Paraleliza checks independientes en jobs separados. Cachea pip-audit y re-ejecuta solo cuando requirements.txt cambie. Ejecuta suite adversarial completa solo en merges a main; en PRs un subset. Objetivo: < 5 minutos de overhead.

Ejercicios

Ejercicio 1: Construir checklist completo con auto-checks

Implementa un DeploymentChecklist con 15+ items de 4 categorías. Registra auto-checks para 5+ items. Ejecuta auto-checks, marca 3 manuales, genera reporte.

from datetime import datetime

checklist = DeploymentChecklist(project_name="AI Assistant v2", environment="production")

items_def = [
    ("S-01", CheckCategory.SECRETS, "No .env committed", CheckType.AUTOMATIC, "git check"),
    ("S-02", CheckCategory.SECRETS, "Keys in vault", CheckType.MANUAL, "Vault health"),
    ("S-06", CheckCategory.SECRETS, "No secrets in logs", CheckType.AUTOMATIC, "Log scan"),
    ("A-01", CheckCategory.API_SECURITY, "HTTPS enforced", CheckType.AUTOMATIC, "curl check"),
    ("A-03", CheckCategory.API_SECURITY, "Rate limiting active", CheckType.AUTOMATIC, "Config check"),
    ("A-04", CheckCategory.API_SECURITY, "Auth required", CheckType.AUTOMATIC, "401 test"),
    ("IO-01", CheckCategory.INPUT_OUTPUT, "Injection detector active", CheckType.AUTOMATIC, "Payload test"),
    ("IO-02", CheckCategory.INPUT_OUTPUT, "System prompt hardened", CheckType.MANUAL, "Prompt review"),
    ("IO-03", CheckCategory.INPUT_OUTPUT, "Output filter active", CheckType.AUTOMATIC, "Output test"),
    ("P-01", CheckCategory.PII, "PII scanner input", CheckType.AUTOMATIC, "PII test"),
    ("P-02", CheckCategory.PII, "PII scanner output", CheckType.AUTOMATIC, "PII test"),
    ("P-03", CheckCategory.PII, "Logs no PII", CheckType.MANUAL, "Log sample"),
    ("M-01", CheckCategory.MONITORING, "Structured logging", CheckType.AUTOMATIC, "Format check"),
    ("L-01", CheckCategory.LLM_SPECIFIC, "Model pinned", CheckType.AUTOMATIC, "Config check"),
    ("L-06", CheckCategory.LLM_SPECIFIC, "OWASP updated", CheckType.MANUAL, "Doc review"),
]

for id_, cat, desc, ctype, verif in items_def:
    checklist.add_item(ChecklistItem(id=id_, category=cat, description=desc, check_type=ctype, verification_method=verif))

# Registrar auto-checks (simulados)
for id_, fn in [("S-01", lambda: True), ("S-06", lambda: True), ("A-01", lambda: True),
                ("A-03", lambda: False), ("A-04", lambda: True), ("IO-01", lambda: True),
                ("IO-03", lambda: True), ("P-01", lambda: True), ("P-02", lambda: True),
                ("M-01", lambda: True), ("L-01", lambda: True)]:
    checklist.register_auto_check(id_, fn)

print(f"Executed {checklist.run_automatic_checks()} auto checks\n")

checklist.mark_manual("S-02", True, "Ana García", "Vault verified 2026-03-10")
checklist.mark_manual("IO-02", True, "Carlos López", "Prompt v2.3 reviewed")
checklist.mark_manual("P-03", False, "Diana Ruiz", "Found emails in 3 log entries")

checklist.print_report()

# Output esperado:
# =======================================================
# Deployment Checklist: AI Assistant v2 (production)
# =======================================================
# Total: 15 | Pass: 12 | Fail: 2 | Pending: 1
# Pass rate: 80%
# ❌ DEPLOY BLOCKED — 2 failed
# Failed:
#   ❌ A-03: Rate limiting active
#   ❌ P-03: Logs no PII

Ejercicio 2: Crear un environment-aware runner

Implementa un runner que filtre el checklist por entorno. Development: solo required. Staging: required + optional advisory. Production: todo obligatorio.

from pydantic import BaseModel, Field

class EnvironmentRunner(BaseModel):
    configs: dict[str, EnvironmentConfig] = Field(default_factory=dict)

    def run_for(self, env: str, items: list[ChecklistItem]) -> dict:
        cfg = self.configs.get(env)
        if not cfg:
            return {"error": f"Unknown: {env}"}
        required = [i for i in items if i.id in cfg.required_checks]
        optional = [i for i in items if i.id in cfg.optional_checks]
        req_fail = sum(1 for i in required if i.status.value == "fail")
        deploy_ok = req_fail == 0 if cfg.strict_mode else req_fail <= 1
        return {
            "environment": env,
            "required": f"{sum(1 for i in required if i.status.value == 'pass')}/{len(required)}",
            "optional": f"{sum(1 for i in optional if i.status.value == 'pass')}/{len(optional)}",
            "deploy": deploy_ok,
            "failed": [i.id for i in required if i.status.value == "fail"],
        }

    def compare(self, items: list[ChecklistItem]) -> str:
        lines = ["=== Environment Comparison ==="]
        for env in self.configs:
            r = self.run_for(env, items)
            icon = "✅" if r["deploy"] else "❌"
            lines.append(f"{icon} {env.upper()}: {r['required']} required, {r['optional']} optional")
            if r["failed"]:
                lines.append(f"   Failed: {', '.join(r['failed'])}")
        return "\n".join(lines)

runner = EnvironmentRunner(configs=build_env_configs())

# Simular items con A-03 y P-03 fallando
sample = []
for id_, cat, desc, ctype, verif in items_def:
    item = ChecklistItem(id=id_, category=cat, description=desc, check_type=ctype, verification_method=verif)
    item.status = CheckStatus.FAIL if id_ in ("A-03", "P-03") else CheckStatus.PASS
    sample.append(item)

print(runner.compare(sample))

Ejercicio 3: Implementar checklist diff entre deploys

Crea una función que compare el checklist entre dos deploys y muestre qué mejoró, empeoró, o se mantuvo.

from pydantic import BaseModel, Field
from datetime import datetime

class DeploySnapshot(BaseModel):
    deploy_id: str
    date: datetime
    results: dict[str, str]  # item_id → status

def checklist_diff(before: DeploySnapshot, after: DeploySnapshot) -> dict:
    improved, regressed, unchanged = [], [], []
    for fid, old in before.results.items():
        new = after.results.get(fid, "removed")
        if old == new:
            unchanged.append(fid)
        elif old == "fail" and new == "pass":
            improved.append(fid)
        elif old == "pass" and new == "fail":
            regressed.append(fid)
    new_checks = [fid for fid in after.results if fid not in before.results]
    return {"improved": improved, "regressed": regressed, "unchanged": unchanged, "new": new_checks}

v1 = DeploySnapshot(deploy_id="v2.0", date=datetime(2026, 3, 1), results={"S-01": "pass", "A-03": "fail", "P-03": "fail", "IO-01": "pass"})
v2 = DeploySnapshot(deploy_id="v2.1", date=datetime(2026, 3, 14), results={"S-01": "pass", "A-03": "pass", "P-03": "pass", "IO-01": "pass", "L-01": "pass"})

d = checklist_diff(v1, v2)
print(f"v2.0 → v2.1:")
print(f"  📈 Improved: {d['improved']}")
print(f"  📉 Regressed: {d['regressed']}")
print(f"  ➡️  Unchanged: {d['unchanged']}")
print(f"  🆕 New: {d['new']}")

Ejercicio 4: Crear un Slack notifier para el checklist

Implementa un ChecklistNotifier que genere un payload de Slack con status, failures, y call-to-action.

import json
from pydantic import BaseModel, Field
from datetime import datetime

class ChecklistNotifier(BaseModel):
    webhook_url: str
    channel: str = "#deploys"
    mention_on_failure: list[str] = Field(default_factory=lambda: ["@security-team"])

    def build_payload(self, report: ChecklistReport) -> dict:
        emoji = "✅" if report.deploy_approved else "🚨"
        status = "APPROVED" if report.deploy_approved else "BLOCKED"
        color = "#36a64f" if report.deploy_approved else "#cc0000"

        header = f"{emoji} Deploy Checklist: *{status}*"
        if not report.deploy_approved:
            header += f"\n{' '.join(self.mention_on_failure)}"

        stats = (f"*Project:* {report.project_name} | *Env:* {report.environment}\n"
                 f"*Pass rate:* {report.pass_rate:.0f}% ({report.passed}/{report.total_items})")

        blocks = [
            {"type": "section", "text": {"type": "mrkdwn", "text": header}},
            {"type": "section", "text": {"type": "mrkdwn", "text": stats}},
        ]

        if report.failed_items:
            fail_text = "*Failed:*\n" + "\n".join(f"• `{f['id']}` {f['description']}" for f in report.failed_items)
            blocks.append({"type": "section", "text": {"type": "mrkdwn", "text": fail_text}})

        if not report.deploy_approved:
            blocks.append({"type": "section", "text": {"type": "mrkdwn", "text": "⚡ Fix failed checks and re-run pipeline."}})

        return {"channel": self.channel, "attachments": [{"color": color, "blocks": blocks}]}

notifier = ChecklistNotifier(webhook_url="https://hooks.slack.com/services/T00/B00/X")
sample_report = ChecklistReport(
    project_name="AI Assistant v2", environment="production",
    run_date=datetime.now(), total_items=15, passed=13, failed=2,
    pending=0, skipped=0, pass_rate=86.7, deploy_approved=False,
    failed_items=[{"id": "A-03", "description": "Rate limiting", "notes": ""}, {"id": "P-03", "description": "PII in logs", "notes": ""}],
    summary="BLOCKED",
)
print(json.dumps(notifier.build_payload(sample_report), indent=2, ensure_ascii=False))

Resumen

• 🔒 Un deployment checklist para AI incluye verificaciones de system prompt, injection defense, PII scanning, y model configuration — no solo HTTPS y CORS
• 📋 Los 50+ items se organizan en 7 categorías: Secrets, API Security, Input/Output, PII, Monitoring, Infrastructure, LLM-Specific
• ⚙️ Las verificaciones automáticas eliminan error humano: .env no committeado, no secrets en código, rate limiting, PII scanner activo
• 👤 Las verificaciones manuales requieren evidencia documentada: quién verificó, cuándo, y con qué prueba
• 🔄 Pre-launch checks bloquean el deploy; post-launch checks validan en producción sin bloquear pero requieren respuesta
• 🏗️ La configuración por entorno escala la rigurosidad: development 3 checks, staging 14, production 57
• 🤖 La integración CI/CD con GitHub Actions ejecuta checks automáticamente como gate pre-deploy en cada push a main
• 📱 Las notificaciones a Slack dan visibilidad inmediata de failures al equipo de seguridad

Próxima cápsula: En la cápsula 05 vas a documentar las decisiones de seguridad de tu sistema y crear un incident response runbook profesional.

Recursos adicionales

1. OWASP AI Security Deployment Guide — Lineamientos de deployment específicos para AI
2. CIS Benchmarks — Benchmarks de seguridad de infraestructura
3. GitHub Actions Security Hardening — Seguridad para workflows CI/CD
4. pip-audit — Python Dependency Scanner — Scanner de vulnerabilidades para dependencias Python
5. Trivy — Container Security Scanner — Scanner para imágenes Docker e IaC
6. NIST SP 800-53 — Security Controls — Catálogo de controles de seguridad de referencia
7. Drata — Compliance Automation — Plataforma de automatización de compliance y checklists

Creado: Marzo 2026 Versión: 1.0