Module 6: Deployment Pipelines

8. Project — Staging Deploy Pipeline

Next →

Overview

This is Module 6's project. You integrate everything you learned into a complete deployment pipeline: Module 5's Docker image gets built, deployed to staging automatically, validated with smoke tests, a reviewer approves, and it gets deployed to production — with an automatic rollback if something fails. The result is a pipeline that takes your code from a push to production with staging as a safety net, approval as protection, and rollback as the emergency plan.

What you build: A GitHub Actions workflow that implements the complete deployment pattern: test → build → deploy staging → smoke test → approval → deploy production → verify → roll back if it fails. It includes Job Summaries, annotations, and all the environment configuration.

Why it matters: This pipeline is the backbone of Module 8 (the capstone project). In Module 8, you add monitoring and advanced notifications (Slack, scheduled runs). But the mechanics of deployment — staging, approval, production, rollback — is what you build here.

Prerequisites

Before starting, verify that you have:

  • ✅ A repository on GitHub with GitHub Actions enabled
  • ✅ Module 5's Docker pipeline working (or you'll use the build here)
  • ✅ GitHub Environments configured: staging and production
  • ✅ Protection rules on production: a required reviewer (yourself is fine)
  • ✅ Knowledge of capsules 02-07 of this module
  • ✅ A server accessible via SSH for the deploy (or you'll simulate with localhost)

If you don't have a server

If you don't have a staging/production server, you can simulate the deploy locally using docker compose on the same runner. The pipeline works the same way — the difference is that staging and production run on the same runner instead of on separate servers.

# Simulation: deploy with docker compose on the runner
- name: Deploy to staging (simulated)
  run: |
    IMAGE_TAG=sha-${{ github.sha }} docker compose \
      -f docker-compose.yml -f docker-compose.staging.yml \
      up -d --pull always

Project structure

The files you're going to create/modify

your-ai-project/
├── .github/
│   └── workflows/
│       ├── ci.yml                    # CI pipeline (Modules 1-3, existing)
│       ├── docker.yml                # Docker build (Module 5, existing)
│       └── deploy.yml                # Deploy pipeline (THIS PROJECT)
├── docker-compose.yml                # The base compose
├── docker-compose.staging.yml        # The staging override
├── docker-compose.prod.yml           # The production override
├── Dockerfile                        # The Dockerfile (Module 5)
├── scripts/
│   ├── deploy.sh                     # The deploy script
│   └── smoke-test.sh                 # The smoke test script
├── src/
│   ├── __init__.py
│   ├── main.py                       # FastAPI app
│   ├── config.py                     # Settings
│   └── chain.py                      # AI pipeline
├── tests/
│   ├── __init__.py
│   └── test_main.py
├── requirements.txt
└── requirements-dev.txt

Step 1: The Docker Compose files

docker-compose.yml (the base)

# docker-compose.yml
services:
  ai-api:
    image: ghcr.io/${GITHUB_REPOSITORY:-user/ai-api}:${IMAGE_TAG:-latest}
    ports:
      - "${APP_PORT:-8000}:8000"
    environment:
      - APP_ENV=${APP_ENV:-development}
      - LOG_LEVEL=${LOG_LEVEL:-info}
    env_file:
      - .env
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 40s
    restart: unless-stopped

docker-compose.staging.yml (the staging override)

# docker-compose.staging.yml
services:
  ai-api:
    ports:
      - "8001:8000"
    environment:
      - APP_ENV=staging
      - LOG_LEVEL=debug

docker-compose.prod.yml (the production override)

# docker-compose.prod.yml
services:
  ai-api:
    ports:
      - "8000:8000"
    environment:
      - APP_ENV=production
      - LOG_LEVEL=warning
    deploy:
      resources:
        limits:
          memory: 2G

Step 2: The deploy and smoke test scripts

scripts/deploy.sh

#!/bin/bash
set -euo pipefail

ENVIRONMENT=${1:?"Usage: deploy.sh <staging|production> <image_tag>"}
IMAGE_TAG=${2:?"Usage: deploy.sh <staging|production> <image_tag>"}

COMPOSE_FILE="docker-compose.yml"
case "$ENVIRONMENT" in
  staging)    OVERRIDE="docker-compose.staging.yml" ;;
  production) OVERRIDE="docker-compose.prod.yml" ;;
  *)          echo "Error: Unknown environment '$ENVIRONMENT'"; exit 1 ;;
esac

echo "=== Deploy: $IMAGE_TAG → $ENVIRONMENT ==="

CURRENT_IMAGE=$(docker compose -f "$COMPOSE_FILE" -f "$OVERRIDE" \
  ps --format '{{.Image}}' 2>/dev/null | head -1 || echo "none")
echo "Current: $CURRENT_IMAGE"
echo "New:     ghcr.io/user/ai-api:$IMAGE_TAG"

export IMAGE_TAG
echo "Pulling image..."
docker compose -f "$COMPOSE_FILE" -f "$OVERRIDE" pull 2>/dev/null

echo "Starting containers..."
docker compose -f "$COMPOSE_FILE" -f "$OVERRIDE" up -d --remove-orphans

echo "Waiting for health check..."
for i in $(seq 1 30); do
  HEALTH=$(docker compose -f "$COMPOSE_FILE" -f "$OVERRIDE" \
    ps --format '{{.Health}}' 2>/dev/null | head -1 || echo "unknown")

  if [ "$HEALTH" = "healthy" ]; then
    echo "✅ Deploy successful: $ENVIRONMENT healthy after $((i*2))s"
    exit 0
  fi
  echo "  Health: $HEALTH ($i/30)"
  sleep 2
done

echo "❌ Health check failed after 60s"
echo "Container logs:"
docker compose -f "$COMPOSE_FILE" -f "$OVERRIDE" logs --tail=20

echo "Rolling back to: $CURRENT_IMAGE"
if [ "$CURRENT_IMAGE" != "none" ]; then
  PREV_TAG=$(echo "$CURRENT_IMAGE" | cut -d: -f2)
  export IMAGE_TAG="$PREV_TAG"
  docker compose -f "$COMPOSE_FILE" -f "$OVERRIDE" up -d --remove-orphans
  echo "Rolled back to $PREV_TAG"
fi

exit 1

scripts/smoke-test.sh

#!/bin/bash
set -euo pipefail

URL=${1:?"Usage: smoke-test.sh <base_url>"}
MAX_ATTEMPTS=${2:-30}
INTERVAL=${3:-2}

echo "=== Smoke test: $URL ==="

for i in $(seq 1 "$MAX_ATTEMPTS"); do
  HTTP_CODE=$(curl -sf -o /dev/null -w "%{http_code}" "$URL/health" 2>/dev/null || echo "000")

  if [ "$HTTP_CODE" = "200" ]; then
    RESPONSE_TIME=$(curl -sf -w "%{time_total}" -o /dev/null "$URL/health" 2>/dev/null || echo "N/A")
    echo "✅ Healthy after attempt $i — HTTP $HTTP_CODE — ${RESPONSE_TIME}s"

    BODY=$(curl -sf "$URL/health" 2>/dev/null || echo "{}")
    echo "Response: $BODY"
    exit 0
  fi

  echo "  Attempt $i/$MAX_ATTEMPTS — HTTP $HTTP_CODE"
  sleep "$INTERVAL"
done

echo "❌ Smoke test failed after $MAX_ATTEMPTS attempts ($((MAX_ATTEMPTS * INTERVAL))s)"
exit 1
# Give them execute permissions
chmod +x scripts/deploy.sh scripts/smoke-test.sh

Step 3: Configure the GitHub Environments

Staging

GitHub → Settings → Environments → New environment → "staging"

Deployment branches: Selected → main
Protection rules: none (an automatic deploy)

Environment secrets:
  DEPLOY_SSH_KEY = (the SSH key for the staging server)
  OPENAI_API_KEY = sk-staging-abc123 (a limited budget)

Environment variables:
  DEPLOY_HOST = staging.your-app.com
  APP_ENV = staging

Production

GitHub → Settings → Environments → New environment → "production"

Deployment branches: Selected → main
Protection rules:
  ☑ Required reviewers: @your-username
  ☑ Wait timer: 0 minutes (or 5 if you want a cooling period)

Environment secrets:
  DEPLOY_SSH_KEY = (the SSH key for the production server)
  OPENAI_API_KEY = sk-prod-xyz789 (production)

Environment variables:
  DEPLOY_HOST = your-app.com
  APP_ENV = production

Step 4: The Workflow — Deploy Pipeline

The complete workflow

# .github/workflows/deploy.yml
name: Deploy Pipeline

on:
  push:
    branches: [main]
  workflow_dispatch:
    inputs:
      image_tag:
        description: "Image tag to deploy (e.g., sha-abc1234)"
        required: false
        type: string
      skip_staging:
        description: "Skip staging (for rollback)"
        required: false
        type: boolean
        default: false

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  # ─────────────────────────────────────────────
  # Job 1: Tests
  # ─────────────────────────────────────────────
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"

      - name: Install dependencies
        run: pip install -r requirements-dev.txt

      - name: Lint
        run: ruff check src/ tests/

      - name: Tests
        run: pytest tests/ -v --tb=short

      - name: Test summary
        if: always()
        run: |
          echo "### Tests" >> $GITHUB_STEP_SUMMARY
          echo "- Lint: ${{ steps.lint.outcome == 'failure' && '❌' || '✅' }}" >> $GITHUB_STEP_SUMMARY
          echo "- Tests: ${{ steps.tests.outcome == 'failure' && '❌' || '✅' }}" >> $GITHUB_STEP_SUMMARY

  # ─────────────────────────────────────────────
  # Job 2: Build & Push the Docker Image
  # ─────────────────────────────────────────────
  build:
    needs: test
    runs-on: ubuntu-latest
    timeout-minutes: 20
    permissions:
      contents: read
      packages: write
    outputs:
      image_tag: ${{ steps.tag.outputs.tag }}
      full_image: ${{ steps.tag.outputs.full }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Determine image tag
        id: tag
        run: |
          if [ "${{ github.event.inputs.image_tag }}" != "" ]; then
            TAG="${{ github.event.inputs.image_tag }}"
          else
            TAG="sha-$(echo ${{ github.sha }} | cut -c1-12)"
          fi
          echo "tag=$TAG" >> $GITHUB_OUTPUT
          echo "full=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:$TAG" >> $GITHUB_OUTPUT
          echo "Image tag: $TAG"

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to GHCR
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Docker metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=raw,value=${{ steps.tag.outputs.tag }}
            type=raw,value=latest,enable={{is_default_branch}}

      - name: Build and push
        uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
          build-args: |
            GIT_SHA=${{ github.sha }}
            BUILD_DATE=${{ github.event.head_commit.timestamp }}

      - name: Build summary
        run: |
          echo "### Docker Build" >> $GITHUB_STEP_SUMMARY
          echo "- Image: \`${{ steps.tag.outputs.full }}\`" >> $GITHUB_STEP_SUMMARY
          echo "- Cache: GHA backend" >> $GITHUB_STEP_SUMMARY

  # ─────────────────────────────────────────────
  # Job 3: Deploy to Staging
  # ─────────────────────────────────────────────
  deploy-staging:
    needs: build
    if: ${{ github.event.inputs.skip_staging != 'true' }}
    runs-on: ubuntu-latest
    timeout-minutes: 10
    environment:
      name: staging
      url: https://${{ vars.DEPLOY_HOST || 'staging.your-app.com' }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Deploy to staging
        env:
          SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
          HOST: ${{ vars.DEPLOY_HOST }}
          IMAGE_TAG: ${{ needs.build.outputs.image_tag }}
        run: |
          echo "Deploying $IMAGE_TAG to staging ($HOST)"

          if [ -n "$SSH_KEY" ] && [ -n "$HOST" ]; then
            echo "$SSH_KEY" > /tmp/key && chmod 600 /tmp/key
            ssh -o StrictHostKeyChecking=no -i /tmp/key deploy@$HOST \
              "cd /app && IMAGE_TAG=$IMAGE_TAG bash scripts/deploy.sh staging $IMAGE_TAG"
            rm /tmp/key
          else
            echo "No SSH key or host configured — simulating deploy"
            echo "IMAGE_TAG=$IMAGE_TAG docker compose -f docker-compose.yml -f docker-compose.staging.yml up -d"
          fi

      - name: Smoke test staging
        env:
          HOST: ${{ vars.DEPLOY_HOST }}
        run: |
          STAGING_URL="https://${HOST:-staging.your-app.com}"
          echo "Running smoke test against $STAGING_URL"

          for i in $(seq 1 30); do
            HTTP_CODE=$(curl -sf -o /dev/null -w "%{http_code}" "$STAGING_URL/health" 2>/dev/null || echo "000")
            if [ "$HTTP_CODE" = "200" ]; then
              RESPONSE_TIME=$(curl -sf -w "%{time_total}" -o /dev/null "$STAGING_URL/health")
              echo "::notice title=Staging OK::Healthy after ${i}s — response time: ${RESPONSE_TIME}s"
              exit 0
            fi
            echo "Attempt $i/30 — HTTP $HTTP_CODE"
            sleep 2
          done
          echo "::error title=Staging Failed::Health check failed after 60s"
          exit 1

      - name: Staging summary
        run: |
          echo "### Staging Deployment" >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "| | |" >> $GITHUB_STEP_SUMMARY
          echo "|---|---|" >> $GITHUB_STEP_SUMMARY
          echo "| **Image** | \`${{ needs.build.outputs.image_tag }}\` |" >> $GITHUB_STEP_SUMMARY
          echo "| **URL** | [staging](https://${{ vars.DEPLOY_HOST || 'staging.your-app.com' }}) |" >> $GITHUB_STEP_SUMMARY
          echo "| **Health** | ✅ Passing |" >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "### Reviewer Checklist" >> $GITHUB_STEP_SUMMARY
          echo "- [ ] \`/health\` responds 200" >> $GITHUB_STEP_SUMMARY
          echo "- [ ] \`/chat\` returns coherent responses" >> $GITHUB_STEP_SUMMARY
          echo "- [ ] Response latency < 5 seconds" >> $GITHUB_STEP_SUMMARY
          echo "- [ ] No errors in logs" >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "**Approve to deploy to production.**" >> $GITHUB_STEP_SUMMARY

  # ─────────────────────────────────────────────
  # Job 4: Deploy to Production (with approval)
  # ─────────────────────────────────────────────
  deploy-production:
    needs: [build, deploy-staging]
    runs-on: ubuntu-latest
    timeout-minutes: 10
    environment:
      name: production
      url: https://${{ vars.DEPLOY_HOST || 'your-app.com' }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      # Save the current version for the rollback
      - name: Get current production version
        id: current
        env:
          SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
          HOST: ${{ vars.DEPLOY_HOST }}
        run: |
          if [ -n "$SSH_KEY" ] && [ -n "$HOST" ]; then
            echo "$SSH_KEY" > /tmp/key && chmod 600 /tmp/key
            CURRENT=$(ssh -o StrictHostKeyChecking=no -i /tmp/key deploy@$HOST \
              "cd /app && docker compose ps --format '{{.Image}}'" 2>/dev/null \
              | head -1 | cut -d: -f2 || echo "unknown")
            rm /tmp/key
          else
            CURRENT="unknown"
          fi
          echo "tag=${CURRENT:-unknown}" >> $GITHUB_OUTPUT
          echo "Current production version: ${CURRENT:-unknown}"

      # Deploy the new version
      - name: Deploy to production
        env:
          SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
          HOST: ${{ vars.DEPLOY_HOST }}
          IMAGE_TAG: ${{ needs.build.outputs.image_tag }}
        run: |
          echo "Deploying $IMAGE_TAG to production ($HOST)"

          if [ -n "$SSH_KEY" ] && [ -n "$HOST" ]; then
            echo "$SSH_KEY" > /tmp/key && chmod 600 /tmp/key
            ssh -o StrictHostKeyChecking=no -i /tmp/key deploy@$HOST \
              "cd /app && IMAGE_TAG=$IMAGE_TAG bash scripts/deploy.sh production $IMAGE_TAG"
            rm /tmp/key
          else
            echo "No SSH key or host configured — simulating deploy"
            echo "IMAGE_TAG=$IMAGE_TAG docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d"
          fi

      # Verify the deployment
      - name: Verify production
        id: verify
        continue-on-error: true
        env:
          HOST: ${{ vars.DEPLOY_HOST }}
        run: |
          PROD_URL="https://${HOST:-your-app.com}"
          echo "Verifying production at $PROD_URL"

          sleep 10

          for i in $(seq 1 30); do
            HTTP_CODE=$(curl -sf -o /dev/null -w "%{http_code}" "$PROD_URL/health" 2>/dev/null || echo "000")
            if [ "$HTTP_CODE" = "200" ]; then
              RESPONSE_TIME=$(curl -sf -w "%{time_total}" -o /dev/null "$PROD_URL/health")
              echo "::notice title=Production OK::Healthy after $((10 + i*2))s — ${RESPONSE_TIME}s"
              exit 0
            fi
            echo "Attempt $i/30 — HTTP $HTTP_CODE"
            sleep 2
          done

          echo "::error title=Production Health Failed::Health check failed after 70s"
          exit 1

      # Roll back on failure
      - name: Rollback on failure
        if: steps.verify.outcome == 'failure'
        env:
          SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
          HOST: ${{ vars.DEPLOY_HOST }}
        run: |
          PREV="${{ steps.current.outputs.tag }}"
          echo "::error title=Rollback::Deploy failed — rolling back to $PREV"

          if [ "$PREV" = "unknown" ] || [ -z "$PREV" ]; then
            echo "::error::Cannot rollback — no previous version (first deploy?)"
            echo "Fix the issue and push a new commit."
            exit 1
          fi

          if [ -n "$SSH_KEY" ] && [ -n "$HOST" ]; then
            echo "$SSH_KEY" > /tmp/key && chmod 600 /tmp/key
            ssh -o StrictHostKeyChecking=no -i /tmp/key deploy@$HOST \
              "cd /app && IMAGE_TAG=$PREV bash scripts/deploy.sh production $PREV"
            rm /tmp/key
          fi

          sleep 10
          PROD_URL="https://${HOST:-your-app.com}"
          for i in $(seq 1 15); do
            if curl -sf "$PROD_URL/health" > /dev/null 2>&1; then
              echo "Rollback successful — production running $PREV"
              exit 1
            fi
            sleep 2
          done

          echo "::error::CRITICAL: Rollback also failed! Manual intervention required."
          exit 1

      # Summary
      - name: Production summary
        if: always()
        run: |
          echo "### Production Deployment" >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY

          if [ "${{ steps.verify.outcome }}" = "success" ]; then
            echo "**Status:** ✅ Deployed successfully" >> $GITHUB_STEP_SUMMARY
            echo "" >> $GITHUB_STEP_SUMMARY
            echo "| | |" >> $GITHUB_STEP_SUMMARY
            echo "|---|---|" >> $GITHUB_STEP_SUMMARY
            echo "| **Image** | \`${{ needs.build.outputs.image_tag }}\` |" >> $GITHUB_STEP_SUMMARY
            echo "| **URL** | [production](https://${{ vars.DEPLOY_HOST || 'your-app.com' }}) |" >> $GITHUB_STEP_SUMMARY
            echo "| **Commit** | [\`$(echo ${{ github.sha }} | cut -c1-7)\`](${{ github.server_url }}/${{ github.repository }}/commit/${{ github.sha }}) |" >> $GITHUB_STEP_SUMMARY
            echo "| **Author** | @${{ github.actor }} |" >> $GITHUB_STEP_SUMMARY
          else
            echo "**Status:** ❌ Failed — Rolled back" >> $GITHUB_STEP_SUMMARY
            echo "" >> $GITHUB_STEP_SUMMARY
            echo "| | |" >> $GITHUB_STEP_SUMMARY
            echo "|---|---|" >> $GITHUB_STEP_SUMMARY
            echo "| **Failed image** | \`${{ needs.build.outputs.image_tag }}\` |" >> $GITHUB_STEP_SUMMARY
            echo "| **Rolled back to** | \`${{ steps.current.outputs.tag }}\` |" >> $GITHUB_STEP_SUMMARY
            echo "| **Commit** | [\`$(echo ${{ github.sha }} | cut -c1-7)\`](${{ github.server_url }}/${{ github.repository }}/commit/${{ github.sha }}) |" >> $GITHUB_STEP_SUMMARY
            echo "" >> $GITHUB_STEP_SUMMARY
            echo "**Action required:** Check logs and fix before next deploy." >> $GITHUB_STEP_SUMMARY
          fi

Step 5: Breaking down the workflow

The complete flow

Push to main
    ↓
┌──────────────────────────────────────────────────────┐
│  Job: test (automatic)                                │
│  1. Checkout → Setup Python                           │
│  2. Lint (ruff) → Tests (pytest)                      │
│  ✅ Pass → continue                                   │
│  ❌ Fail → the pipeline stops                         │
└──────────────────────────────────────────────────────┘
    ↓ (needs: test)
┌──────────────────────────────────────────────────────┐
│  Job: build (automatic)                               │
│  1. Determine tag → Setup Buildx → Login GHCR         │
│  2. Metadata → Build → Push to GHCR                   │
│  Output: image_tag for the deploy jobs                │
└──────────────────────────────────────────────────────┘
    ↓ (needs: build)
┌──────────────────────────────────────────────────────┐
│  Job: deploy-staging (automatic, env: staging)        │
│  1. SSH → deploy.sh staging $IMAGE_TAG                │
│  2. Smoke test → /health 200                          │
│  3. Summary with the reviewer checklist               │
└──────────────────────────────────────────────────────┘
    ↓ (needs: deploy-staging)
┌──────────────────────────────────────────────────────┐
│  ⏸️  APPROVAL GATE                                    │
│  Environment: production (required reviewers)          │
│  The reviewer verifies staging → Approves             │
└──────────────────────────────────────────────────────┘
    ↓
┌──────────────────────────────────────────────────────┐
│  Job: deploy-production (post-approval)               │
│  1. Save the current version                          │
│  2. SSH → deploy.sh production $IMAGE_TAG             │
│  3. Verify /health → 200                              │
│  4. If it fails → roll back to the previous version   │
│  5. Summary with the final result                     │
└──────────────────────────────────────────────────────┘

Behavior depending on the event

EventTestBuildStagingApprovalProduction
Push to main✅✅ Build + push✅ Auto deploy⏸️ It waits✅ Post-approval
workflow_dispatch✅✅ (it uses the given tag)✅ or skip⏸️ It waits✅ Post-approval
workflow_dispatch (skip staging)✅✅⏭️ Skip⏸️ It waits✅ Post-approval

Step 6: Running and verifying

The first deploy: triggering the pipeline

git add .
git commit -m "feat: add staging deploy pipeline"
git push origin main

Verifying in GitHub Actions

  1. The Actions tab: Go to your repo → Actions → "Deploy Pipeline"
  2. Verify the chain: test → build → deploy-staging → (waiting) → deploy-production
  3. Review the summaries: Every job has its summary visible in the run's tab
  4. The staging URL: Open staging and verify it manually
  5. Approve: Click "Review deployments" → Approve
  6. Production: Verify that the deploy to production succeeded

Verifying the rollback (a simulation)

To test that the rollback works, you can simulate a failure:

# Option 1: Deploy an image that doesn't exist
# Temporarily edit deploy.yml to use a nonexistent tag
# This will make the pull fail and trigger the rollback

# Option 2: Deploy an app that fails the health check
# Temporarily edit main.py so /health returns 500
# Push → pipeline → staging fails → it never reaches production
# Or skip staging → production fails → rollback

Verifying locally

# Pull the image from the registry
docker pull ghcr.io/your-user/your-repo:latest

# Run it locally
docker run -p 8000:8000 ghcr.io/your-user/your-repo:latest

# Test
curl http://localhost:8000/health

Step 7: A manual rollback with workflow_dispatch

From the UI

GitHub → Actions → Deploy Pipeline → Run workflow

  Branch: main
  image_tag: sha-prev1234   (the tag of the version you want to go back to)
  skip_staging: true         (for a fast rollback straight to production)

  [Run workflow]

From the CLI

# List the available tags
gh api /user/packages/container/ai-api/versions \
  --jq '.[0:5] | .[].metadata.container.tags[]'

# Trigger the rollback
gh workflow run deploy.yml \
  -f image_tag=sha-prev1234 \
  -f skip_staging=true

Step 8: Verifying the times

The first run (no cache)

Job: test            → ~30s (lint + tests)
Job: build           → ~120s (build from scratch + push)
Job: deploy-staging  → ~45s (deploy + smoke test)
⏸️ Approval           → Variable (it depends on the reviewer)
Job: deploy-production → ~45s (deploy + verify)

Total automatic: ~4 min (not counting the approval)

Subsequent runs (with a cache)

Job: test            → ~25s
Job: build           → ~45s (cache hit)
Job: deploy-staging  → ~30s
⏸️ Approval           → Variable
Job: deploy-production → ~30s

Total automatic: ~2 min (not counting the approval)

Completeness checklist

Verify that your project meets every requirement:

GitHub Environments

  • The staging environment created with deployment branches = main
  • The production environment created with required reviewers + deployment branches = main
  • Secrets configured per environment (DEPLOY_SSH_KEY, OPENAI_API_KEY)
  • Variables configured per environment (DEPLOY_HOST, APP_ENV)

Workflow

  • A trigger on push to main + workflow_dispatch
  • A test job with lint + pytest
  • A build job with a Docker build+push and an image_tag output
  • A deploy-staging job with environment: staging
  • The deploy uses the build job's image_tag
  • A smoke test in staging with a health check loop
  • A staging summary with the reviewer checklist
  • A deploy-production job with environment: production (the approval gate)
  • Save the current version before the deploy
  • A health check in production with continue-on-error: true
  • An automatic rollback if the health check fails
  • The edge case: a first deploy with no previous version
  • A production summary (success, or failure + rollback)

Docker Compose

  • A base compose with a healthcheck and a restart policy
  • An override for staging (port, log level)
  • An override for production (port, memory limit)
  • The IMAGE_TAG variable for changing the image

Scripts

  • deploy.sh with a health check and an integrated rollback
  • smoke-test.sh with a retry loop and timeouts

Verification

  • The pipeline passes in GitHub Actions
  • Staging gets deployed automatically
  • The approval gate works (the workflow pauses)
  • After approving, production gets deployed
  • The summaries are visible in each job
  • The rollback works when the health check fails

Optional extensions

Extension 1: A deploy notification via a commit comment

- name: Comment on commit
  if: success()
  uses: actions/github-script@v7
  with:
    script: |
      await github.rest.repos.createCommitComment({
        owner: context.repo.owner,
        repo: context.repo.repo,
        commit_sha: context.sha,
        body: `✅ Deployed to production\n\nImage: \`${{ needs.build.outputs.image_tag }}\`\nURL: https://${{ vars.DEPLOY_HOST }}`
      });

Extension 2: Deploy only during business hours

deploy-production:
  if: |
    needs.deploy-staging.result == 'success' &&
    (github.event_name == 'workflow_dispatch' || 
     (contains('1,2,3,4,5', format('{0}', steps.day.outputs.dow)) && 
      steps.hour.outputs.hour >= 9 && steps.hour.outputs.hour < 17))

Extension 3: Concurrent deploy protection

concurrency:
  group: deploy-${{ github.ref }}
  cancel-in-progress: false

This guarantees that only one deploy pipeline runs at a time. If there's a deploy in progress, the next one waits.

Project troubleshooting

1. deploy-staging fails but test and build passed

Cause: The error is in the deploy step — SSH, permissions, or docker compose.

Solution: Review the deploy-staging job's logs. The most common errors are: the wrong SSH key, the host isn't reachable, docker compose isn't installed on the server.

2. Approval timeout — nobody approves

Cause: The reviewer didn't see the notification. Default: a 30-day timeout.

Solution:

gh run list --status waiting
gh run review <run-id> --approve --body "Staging verified"

3. The rollback doesn't work because the previous version is "unknown"

Cause: It's the first deploy, or the "get current version" step failed.

Solution: The step already handles this case:

if [ "$PREV" = "unknown" ]; then
  echo "Cannot rollback — first deploy"
  exit 1
fi

For a first deploy, the solution is to fix forward: fix it and push again.

4. The smoke test passes but the app doesn't work correctly

Cause: The smoke test only verifies /health. If the bug is in /chat or another endpoint, the smoke test doesn't catch it.

Solution: Add more complete smoke tests:

curl -sf "$URL/health" > /dev/null || exit 1

curl -sf -X POST "$URL/chat" \
  -H "Content-Type: application/json" \
  -d '{"message":"test"}' > /dev/null || exit 1

5. Production gets deployed twice (two fast pushes)

Cause: Two fast pushes generate two pipeline runs. Both can reach production.

Solution: Add concurrency: to the workflow:

concurrency:
  group: deploy-production
  cancel-in-progress: false

This serializes the deploys — the second one waits for the first to finish.

Connection with Module 7

Your pipeline deploys automatically with staging, approval, and rollback. Module 7 adds:

Module 6 (this one):
  Push → test → build → deploy staging → approve → deploy production
  Notifications: Job Summaries, annotations, GitHub native

Module 7 (the next one):
  + Slack notifications on every deploy
  + Scheduled health checks (cron)
  + Reusable workflows
  + Composite actions
  + Scheduled cost monitoring

Module 8 (the final one):
  Everything integrated into a single production-grade pipeline

The transition: you have a working deployment pipeline. Now you need visibility (who finds out?) and automation (how do you monitor outside of deploys?).

Evidence of success

By completing this project, you should have:

  • A deploy.yml workflow that passes in GitHub Actions
  • An automatic deploy to staging on every push to main
  • Smoke tests that verify staging
  • An approval gate that pauses the workflow before production
  • A deploy to production after the approval
  • An automatic rollback if production's health check fails
  • Job Summaries with the deploy's details in each job
  • workflow_dispatch for a manual rollback
  • Docker Compose configured for staging and production
  • Reusable deploy scripts

If you tick every one → you're ready for Module 7.

Summary

  • ✅ The pipeline has 4 jobs: test → build → deploy-staging → deploy-production
  • ✅ Staging is automatic — it gets deployed on every push to main with no intervention
  • ✅ Production has an approval gate — the workflow pauses until a reviewer approves
  • ✅ An automatic rollback — if production's health check fails, it redeploys the previous version
  • ✅ The edge cases are handled: a first deploy with no previous version, a rollback that also fails
  • ✅ Job Summaries give visibility into the status of every pipeline stage
  • ✅ workflow_dispatch allows a manual rollback to any tag
  • ✅ Docker Compose with per-environment overrides (staging, production)
  • ✅ This pipeline is the foundation of Module 8 — the production-grade capstone project

Additional resources

  1. GitHub Actions — Environments — Environments for deployment
  2. GitHub Actions — Job outputs — Passing data between jobs
  3. Docker Compose — Override files — Compose with per-environment overrides
  4. GitHub Actions — Concurrency — Preventing concurrent deploys
  5. GitHub Actions — workflow_dispatch — Manual triggers for a rollback
  6. GitHub CLI — gh run — Managing workflow runs from the CLI
← Previous7. Deploy NotificationsNext →1. Introduction: Monitoring, Notifications and Advanced Patterns