3. Deployment Automation — De Git Push a Producción

Descripción

En esta cápsula vas a automatizar el flujo completo de deployment: desde un git push hasta tu sistema AI corriendo en producción. No solo el deploy — también environment promotion (local → staging → producción) y rollback strategy cuando algo sale mal. Al terminar, un push a main dispara automáticamente tests, build, deployment, y validación.

Contexto: La cápsula anterior definió la arquitectura de integración. Ahora la implementas. Cada paso del pipeline que antes hacías manualmente se convierte en un job automatizado de GitHub Actions. La automatización elimina errores humanos y hace el deployment reproducible.

El Principio: Un Push, Un Deploy

Por qué automatizar

Manual deployment:
  1. Correr tests localmente (a veces se olvida)
  2. Hacer build del Docker image
  3. Subir image al registry
  4. Conectarse a la plataforma
  5. Triggear deploy
  6. Verificar que funciona
  Total: 10-20 min, propenso a errores, depende de que TÚ lo hagas

Automated deployment:
  1. git push main
  Total: 3-5 min, reproducible, ocurre siempre igual

En AI development, iteras constantemente: ajustas prompts, cambias parámetros, experimentas con modelos. Si cada deploy tarda 20 minutos manuales, haces 2-3 al día. Con automatización, haces 10+. La velocidad de iteración impacta directamente la calidad del sistema.

GitHub Actions: El Pipeline Completo

Estructura del workflow

# .github/workflows/deploy.yml
name: Deploy AI System to Production

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  workflow_dispatch:

concurrency:
  group: deployment-${{ github.ref }}
  cancel-in-progress: true

env:
  PYTHON_VERSION: "3.11"
  APP_NAME: "mi-ai-app"

El bloque concurrency es importante: si haces dos pushes rápidos, el segundo cancela el primero. No quieres dos deploys simultáneos compitiendo.

Job 1: Test

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}
          cache: 'pip'

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
          pip install pytest httpx pytest-asyncio

      - name: Run unit tests
        run: pytest tests/unit/ -v --tb=short -q
        env:
          ENVIRONMENT: test

      - name: Run integration tests
        run: pytest tests/integration/ -v --tb=short -q
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          ENVIRONMENT: test

Job 2: Build y verificar Docker image

  build:
    needs: test
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Build Docker image
        run: docker build -t ${{ env.APP_NAME }}:${{ github.sha }} .

      - name: Verify image starts
        run: |
          docker run -d --name verify \
            -p 8000:8000 \
            -e OPENAI_API_KEY=test-key \
            -e ENVIRONMENT=test \
            ${{ env.APP_NAME }}:${{ github.sha }}
          sleep 5
          status=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8000/health || echo "000")
          docker logs verify
          docker stop verify && docker rm verify
          if [ "$status" != "200" ]; then
            echo "Image verification failed: status $status"
            exit 1
          fi
          echo "Image verified successfully"

Job 3: Deploy (variantes por plataforma)

Render:

  deploy:
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    needs: build
    runs-on: ubuntu-latest
    timeout-minutes: 10
    environment: production
    steps:
      - name: Trigger Render deploy
        run: |
          response=$(curl -s -o /dev/null -w "%{http_code}" \
            -X POST "${{ secrets.RENDER_DEPLOY_HOOK_URL }}")
          if [ "$response" != "200" ] && [ "$response" != "201" ]; then
            echo "Deploy trigger failed: $response"
            exit 1
          fi
          echo "Deploy triggered successfully"

Railway:

  deploy:
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    needs: build
    runs-on: ubuntu-latest
    timeout-minutes: 10
    environment: production
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Install Railway CLI
        run: npm install -g @railway/cli

      - name: Deploy to Railway
        run: railway up --detach --service ${{ vars.RAILWAY_SERVICE_ID }}
        env:
          RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}

Fly.io:

  deploy:
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    needs: build
    runs-on: ubuntu-latest
    timeout-minutes: 10
    environment: production
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup flyctl
        uses: superfly/flyctl-actions/setup-flyctl@master

      - name: Deploy to Fly.io
        run: flyctl deploy --remote-only
        env:
          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

Job 4: Validación post-deploy

  validate:
    needs: deploy
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Wait for deployment to stabilize
        run: sleep 45

      - name: Health check with retry
        run: |
          MAX_RETRIES=5
          RETRY_DELAY=15
          for i in $(seq 1 $MAX_RETRIES); do
            status=$(curl -s -o /dev/null -w "%{http_code}" \
              "${{ vars.PRODUCTION_URL }}/health")
            if [ "$status" = "200" ]; then
              echo "Health check passed on attempt $i"
              exit 0
            fi
            echo "Attempt $i/$MAX_RETRIES: status $status. Retrying in ${RETRY_DELAY}s..."
            sleep $RETRY_DELAY
          done
          echo "Health check failed after $MAX_RETRIES attempts"
          exit 1

      - name: Smoke test - inference
        run: |
          response=$(curl -s -w "\n%{http_code}" -X POST \
            "${{ vars.PRODUCTION_URL }}/api/inference" \
            -H "Content-Type: application/json" \
            -d '{"prompt": "Respond with exactly: SMOKE_TEST_OK"}')
          http_code=$(echo "$response" | tail -1)
          body=$(echo "$response" | head -1)
          echo "Status: $http_code"
          echo "Body: $body"
          if [ "$http_code" != "200" ]; then
            echo "Smoke test failed: HTTP $http_code"
            exit 1
          fi
          echo "Smoke test passed"

      - name: Notify success
        if: success()
        run: echo "Deployment validated successfully at $(date)"

      - name: Notify failure
        if: failure()
        run: echo "DEPLOYMENT VALIDATION FAILED - manual intervention required"

Environment Promotion

El flujo: local → staging → producción

┌─────────────┐     ┌─────────────┐     ┌─────────────────┐
│   LOCAL      │────►│  STAGING    │────►│  PRODUCTION     │
│             │     │             │     │                 │
│ docker      │     │ Same image  │     │ Same image      │
│ compose up  │     │ staging     │     │ production      │
│             │     │ env vars    │     │ env vars        │
│ .env.local  │     │ Subset of   │     │ Full traffic    │
│             │     │ traffic     │     │                 │
└─────────────┘     └─────────────┘     └─────────────────┘
     develop             test               release
     branch           PR/staging            main branch

Implementación con GitHub Actions environments

# Para staging (se activa en PRs o push a develop)
deploy-staging:
  if: github.event_name == 'pull_request'
  needs: build
  runs-on: ubuntu-latest
  environment: staging
  steps:
    - uses: actions/checkout@v4
    - name: Deploy to staging
      run: |
        # Tu plataforma puede tener un servicio separado para staging
        # Railway: diferente servicio, mismo proyecto
        # Render: diferente servicio, misma cuenta
        # Fly.io: diferente app (mi-app-staging)
        echo "Deploying to staging..."

# Para producción (solo push a main, después de staging)
deploy-production:
  if: github.ref == 'refs/heads/main' && github.event_name == 'push'
  needs: build
  runs-on: ubuntu-latest
  environment: production
  steps:
    - uses: actions/checkout@v4
    - name: Deploy to production
      run: echo "Deploying to production..."

Configuración de environments en GitHub

GitHub → Settings → Environments:

Staging:
  - Protection rules: None (deploy rápido para testing)
  - Secrets: OPENAI_API_KEY (staging key si tienes una)
  - Variables: PRODUCTION_URL=https://staging.tu-app.railway.app

Production:
  - Protection rules: Required reviewers (opcional pero recomendado)
  - Secrets: OPENAI_API_KEY, PLATFORM_TOKEN
  - Variables: PRODUCTION_URL=https://tu-app.railway.app

Rollback Strategy

Cuando un deploy sale mal

No todos los deploys son exitosos. Necesitas un plan para volver a la versión anterior.

Opción 1: Re-deploy del commit anterior

# Identificar último commit bueno
git log --oneline -5
# abc1234 (HEAD -> main) feat: update prompt template  ← ESTE ROMPIÓ
# def5678 fix: adjust timeout                          ← ESTE FUNCIONABA

# Crear branch de rollback y forzar deploy
git checkout def5678
git checkout -b hotfix/rollback-to-def5678
git push origin hotfix/rollback-to-def5678

# Merge a main para triggear deploy
# O manualmente en la plataforma: deploy commit def5678

Opción 2: Rollback desde la plataforma

# Render: rollback desde el dashboard
# Settings → Manual Deploy → seleccionar deploy anterior

# Railway: rollback
railway rollback

# Fly.io: rollback a release anterior
flyctl releases
flyctl deploy --image registry.fly.io/mi-app:version-anterior

# AWS Lambda: apuntar alias a versión anterior
aws lambda update-alias \
    --function-name mi-funcion \
    --name production \
    --function-version 5  # versión anterior

Opción 3: Workflow de rollback automatizado

# .github/workflows/rollback.yml
name: Rollback Deployment

on:
  workflow_dispatch:
    inputs:
      commit_sha:
        description: 'Commit SHA to rollback to'
        required: true
      reason:
        description: 'Reason for rollback'
        required: true

jobs:
  rollback:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Checkout specific commit
        uses: actions/checkout@v4
        with:
          ref: ${{ github.event.inputs.commit_sha }}

      - name: Log rollback
        run: |
          echo "Rolling back to ${{ github.event.inputs.commit_sha }}"
          echo "Reason: ${{ github.event.inputs.reason }}"
          echo "Triggered by: ${{ github.actor }}"
          echo "Time: $(date -u)"

      - name: Deploy rollback version
        run: |
          # Tu deploy command aquí
          echo "Deploying rollback..."

      - name: Validate rollback
        run: |
          sleep 45
          status=$(curl -s -o /dev/null -w "%{http_code}" \
            "${{ vars.PRODUCTION_URL }}/health")
          if [ "$status" = "200" ]; then
            echo "Rollback successful"
          else
            echo "CRITICAL: Rollback failed, status: $status"
            exit 1
          fi

Cuándo hacer rollback

ROLLBACK INMEDIATO (< 5 min):
├── Health check falla después del deploy
├── Error rate > 50% en los primeros 2 minutos
└── Smoke test de inferencia falla

EVALUAR PRIMERO (5-30 min):
├── Latencia incrementó pero el sistema funciona
├── Error rate subió pero < 10%
└── Un endpoint específico falla, otros funcionan

NO REQUIERE ROLLBACK:
├── Logs muestran warnings pero no errores
├── Latencia marginal (< 20% incremento)
└── Issue cosmético (formato de respuesta diferente)

Secrets Management

Qué secrets necesitas y dónde configurarlos

GitHub Secrets (Settings → Secrets and variables → Actions):
├── OPENAI_API_KEY          # Tu API key de OpenAI
├── RENDER_DEPLOY_HOOK_URL  # Si usas Render
├── RAILWAY_TOKEN           # Si usas Railway
├── FLY_API_TOKEN           # Si usas Fly.io
├── AWS_ACCESS_KEY_ID       # Si usas AWS
└── AWS_SECRET_ACCESS_KEY   # Si usas AWS

GitHub Variables (Settings → Secrets and variables → Actions → Variables):
├── PRODUCTION_URL          # https://tu-app.platform.app
├── RAILWAY_SERVICE_ID      # Si usas Railway
└── APP_NAME                # Nombre de tu app

Plataforma (dashboard de Render/Railway/Fly.io):
├── OPENAI_API_KEY          # La misma key o una diferente para prod
├── ENVIRONMENT             # "production"
├── LOG_LEVEL               # "WARNING" o "INFO"
└── [Otras variables de tu app]

Verificar que los secrets están configurados

# scripts/verify_secrets.py
"""Verifica que las variables de entorno necesarias están configuradas."""
import os
import sys

REQUIRED = {
    "OPENAI_API_KEY": "API key for LLM inference",
    "ENVIRONMENT": "Current environment (development/staging/production)",
}

OPTIONAL = {
    "LOG_LEVEL": ("Logging level", "INFO"),
    "CORS_ORIGINS": ("Allowed CORS origins", '["*"]'),
}

def verify():
    errors = []
    warnings = []

    for var, description in REQUIRED.items():
        value = os.environ.get(var)
        if not value:
            errors.append(f"MISSING: {var} — {description}")
        else:
            masked = value[:4] + "..." + value[-4:] if len(value) > 8 else "***"
            print(f"  {var}: {masked}")

    for var, (description, default) in OPTIONAL.items():
        value = os.environ.get(var)
        if not value:
            warnings.append(f"OPTIONAL: {var} not set, using default: {default}")
        else:
            print(f"  {var}: {value}")

    if warnings:
        print(f"\nWarnings ({len(warnings)}):")
        for w in warnings:
            print(f"  ⚠️ {w}")

    if errors:
        print(f"\nErrors ({len(errors)}):")
        for e in errors:
            print(f"  ❌ {e}")
        sys.exit(1)

    print("\nAll required secrets verified.")

if __name__ == "__main__":
    verify()

Troubleshooting

Problema 1: "El pipeline se queda en pending"

Causa: GitHub Actions tiene un límite de concurrent jobs o hay un runner no disponible.

Solución:

# Añadir timeout a todos los jobs
jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10  # Kill después de 10 min

Verifica en Actions → All workflows si hay jobs encolados. Cancela los obsoletos manualmente.

Problema 2: "Deploy trigger funciona pero la app no actualiza"

Causa: Cache de la plataforma, image tag duplicado, o el deploy se completó con la versión vieja.

Solución:

# Usar SHA del commit como tag para evitar cache
docker build -t mi-app:${{ github.sha }} .

# En Railway: forzar rebuild
railway up --detach

# En Render: verificar en dashboard que el deploy es el commit correcto

Problema 3: "Rollback no funciona — el commit anterior también falla"

Causa: Cambio en variables de entorno o servicio externo, no en código.

Solución:

# Verificar si el problema es el código o las env vars
# 1. Revisar si se cambió algún secret
# 2. Verificar que las APIs externas (OpenAI) están respondiendo
curl -s https://api.openai.com/v1/models \
    -H "Authorization: Bearer $OPENAI_API_KEY" | head -c 200
# 3. Si la API externa está caída, el rollback no resolverá nada

Problema 4: "El job de validate falla por timeout"

Causa: La plataforma tarda más de lo esperado en deploy, o el container tiene un cold start largo.

Solución:

# Incrementar el wait time y los retries
- name: Wait for deployment
  run: sleep 60  # Dar más tiempo

- name: Health check with extended retry
  run: |
    MAX_RETRIES=8
    RETRY_DELAY=20
    # ... (retry loop más largo)

Problema 5: "Los tests pasan local pero fallan en CI"

Causa: Diferencia en versiones, variables no configuradas, o tests que dependen de servicios externos.

Solución:

# Usa la misma versión de Python
python-version: "3.11"  # No "3.x" — sé específico

# Congela dependencias exactas
pip freeze > requirements.txt

# Separa tests que necesitan API key
# tests/unit/ → sin secrets
# tests/integration/ → con secrets

Ejercicios Prácticos

Ejercicio 1: Pipeline completo end-to-end

Crea un pipeline de GitHub Actions que haga test → build → deploy → validate para tu plataforma elegida.

# .github/workflows/deploy.yml
name: Deploy AI System

on:
  push:
    branches: [main]
  workflow_dispatch:

concurrency:
  group: deploy-${{ github.ref }}
  cancel-in-progress: true

env:
  PYTHON_VERSION: "3.11"

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}
          cache: 'pip'
      - run: pip install -r requirements.txt && pip install pytest httpx
      - run: pytest tests/ -v --tb=short
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          ENVIRONMENT: test

  build:
    needs: test
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - run: docker build -t mi-app:${{ github.sha }} .
      - run: |
          docker run -d --name verify -p 8000:8000 \
            -e ENVIRONMENT=test -e OPENAI_API_KEY=fake \
            mi-app:${{ github.sha }}
          sleep 5
          curl -f http://localhost:8000/health
          docker stop verify && docker rm verify

  deploy:
    if: github.ref == 'refs/heads/main'
    needs: build
    runs-on: ubuntu-latest
    timeout-minutes: 10
    environment: production
    steps:
      - uses: actions/checkout@v4
      - name: Deploy
        run: curl -X POST "${{ secrets.RENDER_DEPLOY_HOOK_URL }}"

  validate:
    needs: deploy
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - name: Wait and verify
        run: |
          sleep 60
          for i in 1 2 3 4 5; do
            if curl -sf "${{ vars.PRODUCTION_URL }}/health"; then
              echo "Validation passed"
              exit 0
            fi
            sleep 15
          done
          exit 1

Ejercicio 2: Workflow de rollback

Crea un workflow de workflow_dispatch que permita hacer rollback a un commit específico.

# .github/workflows/rollback.yml
name: Rollback

on:
  workflow_dispatch:
    inputs:
      commit_sha:
        description: 'Commit to rollback to'
        required: true
      reason:
        description: 'Reason'
        required: true

jobs:
  rollback:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.inputs.commit_sha }}

      - name: Log
        run: |
          echo "Rollback to: ${{ github.event.inputs.commit_sha }}"
          echo "Reason: ${{ github.event.inputs.reason }}"
          echo "By: ${{ github.actor }} at $(date -u)"

      - name: Deploy previous version
        run: curl -X POST "${{ secrets.RENDER_DEPLOY_HOOK_URL }}"

      - name: Validate
        run: |
          sleep 60
          curl -sf "${{ vars.PRODUCTION_URL }}/health" || exit 1
          echo "Rollback validated"

Ejercicio 3: Script de verificación de secrets

Crea el script scripts/verify_secrets.py y ejecútalo en tu pipeline antes del deploy.

# scripts/verify_secrets.py
import os
import sys

REQUIRED_VARS = {
    "OPENAI_API_KEY": "Required for LLM inference",
    "ENVIRONMENT": "Must be development, staging, or production",
}

def main():
    missing = []
    for var, desc in REQUIRED_VARS.items():
        val = os.environ.get(var, "")
        if not val:
            missing.append(f"{var}: {desc}")
        else:
            safe = val[:3] + "***" if len(val) > 3 else "***"
            print(f"  {var}={safe}")

    if missing:
        print("Missing required variables:")
        for m in missing:
            print(f"  ❌ {m}")
        sys.exit(1)

    print("All required variables present.")

if __name__ == "__main__":
    main()

- name: Verify secrets
  run: python scripts/verify_secrets.py
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
    ENVIRONMENT: production

Ejercicio 4: Simula un rollback

Haz un deploy intencional que falle (cambia el health check endpoint a uno que no existe) y luego haz rollback al commit anterior.

# 1. Crea un commit que rompa el deploy
# En main.py, cambia el health check path:
# @app.get("/health") → @app.get("/healthz")
# El pipeline buscará /health que ya no existe

# 2. Push y observa el pipeline fallar en validate
git add -A && git commit -m "break: test rollback" && git push

# 3. El job validate fallará (health check retorna 404)

# 4. Identifica el commit anterior
git log --oneline -3
# abc1234 break: test rollback     ← ESTE
# def5678 last working version     ← ROLLBACK A ESTE

# 5. Rollback: ve a GitHub → Actions → Rollback workflow
# Input: def5678
# Reason: "Health check endpoint changed accidentally"

# 6. O revert el commit
git revert abc1234
git push  # Esto triggerea un nuevo deploy con el código bueno

# 7. Verificar que el deploy funciona
curl https://tu-app.platform.app/health
# {"status": "healthy"}

Resumen

• Un push, un deploy: git push main debe disparar automáticamente test → build → deploy → validate
• Concurrency control: usa concurrency en GitHub Actions para evitar deploys simultáneos
• Environment promotion: local → staging → producción con el mismo image y diferente config
• Rollback strategy: ten al menos dos métodos (re-deploy commit anterior + rollback desde plataforma)
• Secrets management: secrets en GitHub Secrets para CI/CD, en la plataforma para runtime
• Timeout en todo: cada job necesita timeout-minutes para evitar pipelines colgados
• Validate siempre: un deploy sin validación automática post-deploy es un deploy a ciegas

Recursos Adicionales

1. GitHub Actions — Environments — Configuración de environments
2. GitHub Actions — Concurrency — Control de concurrencia
3. GitHub Actions — Encrypted Secrets — Gestión de secrets
4. Render — Continuous Deployment — Deploys automáticos en Render
5. Railway — GitHub Deployments — Auto-deploy desde GitHub en Railway
6. Fly.io — GitHub Actions Deploy — CI/CD con Fly.io