Module 7: Monitoring, Notifications, and Advanced Patterns

4. Scheduled Workflows and Cron

Overview

So far, your workflows run on events: a push, a PR, a manual dispatch. But there are tasks that need to run on time: a nightly health check that verifies your AI app is still responding, a weekly cost report, a recomputation of prompt regression baselines every week. These tasks have no code trigger — they have a time trigger.

GitHub Actions supports scheduled workflows using cron syntax. Cron is Unix's scheduling system — a compact notation for expressing "every Monday at 3am" or "every 6 hours" or "the first day of every month." The syntax looks cryptic the first time (0 3 * * 1), but once you understand the 5 fields, it's simple and powerful.

For AI systems, scheduled workflows are critical for a reason that doesn't exist in traditional software: LLM providers update models silently. OpenAI can update gpt-4o-mini without telling you. The output changes, your prompt regression baseline no longer matches, and your next PR fails for reasons that have nothing to do with your code. A nightly scheduled workflow that re-runs the evaluations detects these changes before they affect your development flow.

Connection with the final pipeline: In the capstone pipeline (Module 8), the scheduled workflows run nightly baseline checks and weekly cost reports that validate that the AI system is still working correctly between deployments.


Cron syntax: The 5 fields

┌───────────── minute (0-59)
│ ┌───────────── hour (0-23)
│ │ ┌───────────── day of month (1-31)
│ │ │ ┌───────────── month (1-12)
│ │ │ │ ┌───────────── day of week (0-6, 0=Sunday)
│ │ │ │ │
* * * * *

The special values

CharacterMeaningExample
*Any value* * * * * = every minute
,A list of values0,30 * * * * = minute 0 and 30
-A range0 9-17 * * * = every hour from 9am to 5pm
/An interval*/15 * * * * = every 15 minutes

Common examples

0 3 * * *       → Every day at 3:00 AM UTC
0 9 * * 1       → Every Monday at 9:00 AM UTC
0 0 * * 0       → Every Sunday at midnight UTC
0 */6 * * *     → Every 6 hours (00:00, 06:00, 12:00, 18:00)
30 8 * * 1-5    → Monday to Friday at 8:30 AM UTC
0 0 1 * *       → The first day of every month at midnight
0 12 * * 1,3,5  → Monday, Wednesday and Friday at noon
*/30 * * * *    → Every 30 minutes

The time zone

GitHub Actions uses UTC for cron. If you're in Mexico City (UTC-6), 3:00 AM UTC is 9:00 PM the previous day in CDMX. Always convert to UTC before configuring the cron.

Zone               UTC offset    "3am local" in UTC
─────────────────────────────────────────────────
CDMX (CST)         -6           09:00 UTC
Bogotá (COT)       -5           08:00 UTC
Madrid (CET)       +1           02:00 UTC
Buenos Aires (ART) -3           06:00 UTC

Scheduled workflows in GitHub Actions

The basic syntax

on:
  schedule:
    - cron: "0 3 * * *"

You can have multiple schedules:

on:
  schedule:
    - cron: "0 3 * * *"    # Nightly at 3am UTC
    - cron: "0 9 * * 1"    # Weekly: Monday 9am UTC

Important limitations

  1. The minimum interval: 5 minutes. */1 * * * * (every minute) isn't supported.
  2. Approximate execution. GitHub doesn't guarantee exact execution — there can be delays of minutes during periods of high demand.
  3. Only on the default branch. Scheduled workflows only run on the default branch (main/master). If you define a schedule on a feature branch, it won't run.
  4. They get disabled automatically. If a repo has no activity for 60 days, the scheduled workflows get disabled. GitHub notifies you and you can reactivate them.

Verifying that the scheduled workflows run

Your repo → Actions → The left sidebar → Select the workflow
→ Filter by event: schedule
→ Verify that the runs appear on the expected dates

AI-Specific Schedules

Schedule 1: A Nightly Prompt Regression Baseline

# .github/workflows/nightly-baseline.yml
name: Nightly Prompt Baseline

on:
  schedule:
    - cron: "0 4 * * *"  # 4am UTC, daily
  workflow_dispatch:

jobs:
  update-baseline:
    runs-on: ubuntu-latest
    timeout-minutes: 15

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Run prompt evaluations
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          python scripts/prompt_regression.py \
            --mode generate-baseline \
            --output baselines/current.json

      - name: Compare with existing baseline
        id: compare
        run: |
          if [ -f baselines/current.json ] && [ -f baselines/previous.json ]; then
            python scripts/compare_baselines.py \
              --current baselines/current.json \
              --previous baselines/previous.json \
              --threshold 0.1 > comparison.txt 2>&1
            
            if grep -q "DRIFT_DETECTED" comparison.txt; then
              echo "drift=true" >> $GITHUB_OUTPUT
              echo "Drift detected!"
              cat comparison.txt
            else
              echo "drift=false" >> $GITHUB_OUTPUT
              echo "No significant drift"
            fi
          else
            echo "drift=false" >> $GITHUB_OUTPUT
            echo "No previous baseline to compare"
          fi

      - name: Upload baseline artifact
        uses: actions/upload-artifact@v4
        with:
          name: prompt-baseline-${{ github.run_number }}
          path: baselines/current.json
          retention-days: 30

      - name: Notify if drift detected
        if: steps.compare.outputs.drift == 'true'
        uses: slackapi/slack-github-action@v2.0.0
        with:
          webhook-type: incoming-webhook
          webhook: ${{ secrets.SLACK_WEBHOOK_URL }}
          payload: |
            {
              "text": "⚠️ Prompt drift detected in nightly baseline check",
              "blocks": [
                {
                  "type": "section",
                  "text": {
                    "type": "mrkdwn",
                    "text": "⚠️ *Prompt Drift Detected*\nThe nightly baseline check found significant drift in LLM outputs.\nThis may indicate a model update from the provider.\n<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Details>"
                  }
                }
              ]
            }

Why nightly?

Frequency     The advantage                   The cost
──────────────────────────────────────────────────────
Every hour    It detects drift in minutes     ~$3.60/day (API calls)
Nightly       It detects drift in <24h        ~$0.15/day
Weekly        It detects drift in <7 days     ~$0.02/day

Nightly is the sweet spot: it detects changes in under 24 hours at a reasonable cost. Every hour is excessive for most projects. Weekly can let too much time slip by.

Schedule 2: A Weekly Cost Report

# .github/workflows/weekly-cost-report.yml
name: Weekly Cost Report

on:
  schedule:
    - cron: "0 9 * * 1"  # Monday 9am UTC
  workflow_dispatch:

jobs:
  cost-report:
    runs-on: ubuntu-latest
    timeout-minutes: 10

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Generate cost report
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          python scripts/cost_report.py \
            --period weekly \
            --output cost-report.json

      - name: Generate summary
        id: summary
        run: |
          python -c "
          import json
          with open('cost-report.json') as f:
              report = json.load(f)
          total = report.get('total_cost', 0)
          prev = report.get('previous_period_cost', 0)
          change = ((total - prev) / prev * 100) if prev > 0 else 0
          print(f'total={total:.2f}')
          print(f'change={change:.1f}')
          print(f'alert={\"true\" if change > 20 else \"false\"}')
          " >> $GITHUB_OUTPUT

      - name: Upload cost report
        uses: actions/upload-artifact@v4
        with:
          name: cost-report-${{ github.run_number }}
          path: cost-report.json
          retention-days: 90

      - name: Notify if cost spike
        if: steps.summary.outputs.alert == 'true'
        uses: slackapi/slack-github-action@v2.0.0
        with:
          webhook-type: incoming-webhook
          webhook: ${{ secrets.SLACK_WEBHOOK_URL }}
          payload: |
            {
              "text": "💰 Cost spike detected: +${{ steps.summary.outputs.change }}% vs last week"
            }

Schedule 3: A Periodic Health Check

# .github/workflows/health-check.yml
name: Health Check

on:
  schedule:
    - cron: "0 */6 * * *"  # Every 6 hours
  workflow_dispatch:

jobs:
  health:
    runs-on: ubuntu-latest
    timeout-minutes: 5

    steps:
      - name: Check app health
        id: health
        continue-on-error: true
        run: |
          STATUS=$(curl -sf -o /dev/null -w "%{http_code}" \
            https://your-app.com/health 2>/dev/null || echo "000")
          
          echo "status=$STATUS" >> $GITHUB_OUTPUT
          
          if [ "$STATUS" = "200" ]; then
            echo "App is healthy (HTTP $STATUS)"
          else
            echo "App is unhealthy (HTTP $STATUS)"
            exit 1
          fi

      - name: Check AI endpoint
        id: ai-health
        continue-on-error: true
        run: |
          RESPONSE=$(curl -sf -X POST https://your-app.com/api/chat \
            -H "Content-Type: application/json" \
            -d '{"message":"health check ping"}' \
            --max-time 30 2>/dev/null)
          
          if [ $? -eq 0 ] && [ -n "$RESPONSE" ]; then
            echo "AI endpoint responding"
            echo "healthy=true" >> $GITHUB_OUTPUT
          else
            echo "AI endpoint not responding"
            echo "healthy=false" >> $GITHUB_OUTPUT
            exit 1
          fi

      - name: Report status
        run: |
          echo "## Health Check Report" >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "| Check | Status |" >> $GITHUB_STEP_SUMMARY
          echo "|-------|--------|" >> $GITHUB_STEP_SUMMARY
          echo "| App Health | ${{ steps.health.outcome }} |" >> $GITHUB_STEP_SUMMARY
          echo "| AI Endpoint | ${{ steps.ai-health.outcome }} |" >> $GITHUB_STEP_SUMMARY

      - name: Alert on failure
        if: steps.health.outcome == 'failure' || steps.ai-health.outcome == 'failure'
        uses: slackapi/slack-github-action@v2.0.0
        with:
          webhook-type: incoming-webhook
          webhook: ${{ secrets.SLACK_WEBHOOK_URL }}
          payload: |
            {
              "text": "🚨 Health check failed! App: ${{ steps.health.outcome }}, AI: ${{ steps.ai-health.outcome }}"
            }

The silent problem: Model updates

Why this doesn't exist in traditional software

In traditional software, dependencies don't change unless you update them. If you use requests==2.31.0, that package is identical today and a year from now.

With LLMs, the model you use can change without your consent:

Month 1: gpt-4o-mini responds "The capital of France is Paris"
  → Your test passes ✅

Month 2: OpenAI updates gpt-4o-mini silently
  → gpt-4o-mini responds "Paris is the capital of France"
  → Your test fails because the format changed ❌
  → But your code didn't change at all

The solution: Scheduled checks

on:
  schedule:
    - cron: "0 4 * * *"  # Nightly

The scheduled check runs your prompt regression suite every night. If a model changed, you detect it at 4am — not when a developer tries to merge their PR at 2pm and everything fails mysteriously.

The flow with and without scheduled checks

WITHOUT scheduled checks:
  The model changes (silently) → Nobody knows → 
  A developer opens a PR → The tests fail → Confusion: "I didn't change anything" →
  2 hours of investigating → They discover the model changed → They update the baseline

WITH scheduled checks:
  The model changes → The nightly check detects drift → A Slack alert →
  The baseline gets updated automatically → PRs keep working →
  The developer never even finds out about the change (which is ideal)

Combining a schedule with other triggers

A workflow can have multiple triggers. This is useful when you want the same workflow to run both on demand and on a schedule:

on:
  push:
    branches: [main]
  schedule:
    - cron: "0 4 * * *"
  workflow_dispatch:

Detecting which trigger activated the workflow

- name: Identify trigger
  run: |
    echo "Event: ${{ github.event_name }}"
    # Outputs: "push", "schedule", "workflow_dispatch", etc.

- name: Run different logic based on trigger
  run: |
    if [ "${{ github.event_name }}" = "schedule" ]; then
      echo "Running scheduled maintenance"
      python scripts/update_baselines.py
    else
      echo "Running standard CI checks"
      python scripts/prompt_regression.py --mode check
    fi

Comparisons

Scheduled vs Event-driven workflows

AspectScheduledEvent-driven
TriggerTime (cron)An event (push, PR)
FrequencyFixed (daily, weekly)Variable (it depends on activity)
CostPredictableVariable
Use caseMaintenance, health checksThe CI/CD pipeline
UrgencyLow (it can wait)High (it blocks merges)

cron in GitHub Actions vs cron in Linux

AspectGitHub Actions cronLinux cron
PrecisionApproximate (±minutes)Exact (±seconds)
Minimum interval5 minutes1 minute
Time zoneUTC onlyConfigurable
EnvironmentAn ephemeral runnerA persistent machine
LogsIn the Actions UIIn syslog/journald
InactivityIt gets disabled after 60 daysAlways active

Troubleshooting

"My scheduled workflow doesn't run"

Cause 1: The workflow is on a feature branch, not on the default branch (main).

Solution: Scheduled workflows only run on the default branch. Merge your change to main first.

Cause 2: The repo has had no activity in 60 days.

Solution: GitHub disables scheduled workflows in inactive repos. Go to Actions → select the workflow → "Enable workflow".

Cause 3: The cron is badly formatted.

Solution: Use crontab.guru to verify your cron expression.

"The workflow runs but at a different time than expected"

Cause: The cron uses UTC, not your local time zone.

Solution: Convert your local time to UTC. If you want it to run at 10pm CDMX (UTC-6), the cron is 0 4 * * * (4am UTC = 10pm CDMX).

"The scheduled workflow runs twice"

Cause: You have two schedules that coincide, or the workflow has both schedule and push and a push coincides with the schedule.

Solution: Check your triggers. If you have schedule and push, both can trigger the workflow independently. You can use github.event_name to distinguish.

"I want the scheduled workflow to commit changes"

Cause: Scheduled workflows check out in a detached HEAD and can't push directly.

Solution: Configure git with a token that has write permissions:

- uses: actions/checkout@v4
  with:
    token: ${{ secrets.GITHUB_TOKEN }}

- name: Configure git
  run: |
    git config user.name "github-actions[bot]"
    git config user.email "github-actions[bot]@users.noreply.github.com"

- name: Commit and push changes
  run: |
    git add baselines/
    if git diff --staged --quiet; then
      echo "No changes to commit"
    else
      git commit -m "chore: update prompt baselines [skip ci]"
      git push
    fi

The [skip ci] in the commit message prevents the push from triggering another pipeline run.


Exercises

Exercise 1: Write cron expressions

Write the cron expression for each scenario:

  1. Every day at 6am UTC
  2. Every Monday and Thursday at 2pm UTC
  3. Every 8 hours
  4. The first day of every month at midnight UTC
  5. Every day from Monday to Friday at 9:30am UTC
See solution
1. 0 6 * * *         → Every day at 6:00 AM UTC
2. 0 14 * * 1,4      → Monday and Thursday at 2:00 PM UTC
3. 0 */8 * * *       → Every 8 hours (00:00, 08:00, 16:00)
4. 0 0 1 * *         → The first day of every month at midnight
5. 30 9 * * 1-5      → Monday to Friday at 9:30 AM UTC

Verify at crontab.guru — type the expression and it shows you the next runs.

Exercise 2: Create a nightly health check

Write a complete workflow that runs every night at 3am UTC, does a health check on your application, and notifies Slack if it fails.

See solution
name: Nightly Health Check

on:
  schedule:
    - cron: "0 3 * * *"
  workflow_dispatch:

jobs:
  health-check:
    runs-on: ubuntu-latest
    timeout-minutes: 5

    steps:
      - name: Check application health
        id: health
        continue-on-error: true
        run: |
          for i in $(seq 1 3); do
            STATUS=$(curl -sf -o /dev/null -w "%{http_code}" \
              https://your-app.com/health 2>/dev/null || echo "000")
            if [ "$STATUS" = "200" ]; then
              echo "Healthy on attempt $i"
              echo "result=healthy" >> $GITHUB_OUTPUT
              exit 0
            fi
            echo "Attempt $i failed (HTTP $STATUS)"
            sleep 5
          done
          echo "result=unhealthy" >> $GITHUB_OUTPUT
          exit 1

      - name: Alert on failure
        if: steps.health.outcome == 'failure'
        uses: slackapi/slack-github-action@v2.0.0
        with:
          webhook-type: incoming-webhook
          webhook: ${{ secrets.SLACK_WEBHOOK_URL }}
          payload: |
            {
              "text": "🚨 Nightly health check FAILED — app may be down"
            }

      - name: Summary
        if: always()
        run: |
          echo "## Nightly Health Check" >> $GITHUB_STEP_SUMMARY
          echo "**Status:** ${{ steps.health.outputs.result }}" >> $GITHUB_STEP_SUMMARY
          echo "**Time:** $(date -u)" >> $GITHUB_STEP_SUMMARY

The health check has 3 retries with 5 seconds between each one to tolerate transient failures. It only alerts if all 3 attempts fail.

Exercise 3: A scheduled baseline update with an auto-commit

Create a workflow that runs weekly, regenerates the prompt regression baselines, and makes an automatic commit if there are changes.

See solution
name: Weekly Baseline Update

on:
  schedule:
    - cron: "0 5 * * 0"  # Sunday 5am UTC
  workflow_dispatch:

jobs:
  update-baselines:
    runs-on: ubuntu-latest
    timeout-minutes: 15

    steps:
      - uses: actions/checkout@v4
        with:
          token: ${{ secrets.GITHUB_TOKEN }}

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - run: pip install -r requirements.txt

      - name: Generate new baselines
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          python scripts/prompt_regression.py \
            --mode generate-baseline \
            --output baselines/current.json

      - name: Check for changes
        id: changes
        run: |
          if git diff --quiet baselines/; then
            echo "changed=false" >> $GITHUB_OUTPUT
            echo "No baseline changes detected"
          else
            echo "changed=true" >> $GITHUB_OUTPUT
            echo "Baseline changes detected:"
            git diff --stat baselines/
          fi

      - name: Commit and push
        if: steps.changes.outputs.changed == 'true'
        run: |
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          git add baselines/
          git commit -m "chore: update prompt baselines [skip ci]"
          git push

      - name: Notify if baselines changed
        if: steps.changes.outputs.changed == 'true'
        uses: slackapi/slack-github-action@v2.0.0
        with:
          webhook-type: incoming-webhook
          webhook: ${{ secrets.SLACK_WEBHOOK_URL }}
          payload: |
            {
              "text": "📊 Prompt baselines updated automatically. Review changes if needed."
            }

Exercise 4: Convert local times to UTC cron

Your team is in CDMX (UTC-6). Write the cron expressions for:

  1. A health check at 8am local time
  2. A cost report at 5pm local time on Fridays
  3. A baseline update at 11pm local time on Sundays
See solution
CDMX = UTC-6, so local time + 6 = UTC

1. 8am CDMX = 14:00 UTC → cron: "0 14 * * *"
2. 5pm CDMX Friday = 23:00 UTC Friday → cron: "0 23 * * 5"
3. 11pm CDMX Sunday = 05:00 UTC Monday → cron: "0 5 * * 1"

Case 3 is the tricky one: 11pm Sunday in CDMX is 5am Monday in UTC. The day of the week changes when converting to UTC.


Summary

  • Cron syntax has 5 fields: minute, hour, day of month, month, day of week
  • GitHub Actions uses UTC — always convert your local time before configuring
  • Scheduled workflows only run on the default branch (main/master)
  • AI-specific schedules: nightly prompt baselines, weekly cost reports, periodic health checks
  • LLM providers update models silently — scheduled checks detect drift before it affects your pipeline
  • Combine triggers: a workflow can have schedule + push + workflow_dispatch
  • Auto-committing baselines with [skip ci] to avoid pipeline loops
  • The limitations: a 5-minute minimum, approximate execution, they get disabled after 60 days of inactivity

Additional resources

  1. Crontab Guru - An interactive tool for creating and verifying cron expressions
  2. GitHub Actions Schedule Events - Official schedule documentation
  3. GitHub Actions — Disabling Workflows - How to reactivate disabled workflows
  4. World Time Buddy - A time zone converter
  5. GitHub Actions Checkout — Token - How to push from a workflow
  6. OpenAI Model Updates - The change history of OpenAI's models