7. Debugging Workflows

Overview

Workflows are going to fail. Guaranteed. A step fails because of an incompatible dependency, the YAML has an indentation error, a test passes locally but fails in CI, or a secret isn't configured. The difference between a developer who gets frustrated and a productive one is the ability to debug workflows efficiently.

In this capsule you're going to learn to navigate GitHub Actions' UI, read logs, identify the root cause of failures, and apply debugging techniques that will save you hours. This skill is used in all the following modules — every time something fails in your pipeline, you'll come back to these techniques.

GitHub Actions' UI: Essential navigation

Level 1: The Actions tab

When you go to your repo → Actions, you see the list of workflow runs. Each run has:

• Status: ✅ green (success), ❌ red (failure), 🟡 yellow (in progress), ⚪ gray (cancelled)
• Workflow name: Which workflow ran
• Trigger: What fired it (push, PR, schedule, manual)
• Branch: On which branch
• Commit: Which commit fired it
• Duration: How long it took
• Timestamp: When it ran

Level 2: The workflow run

When you click a run, you see the workflow's jobs:

CI Pipeline (run #45)
├── ✅ lint      (32s)
├── ❌ test      (1m 23s)    ← This one failed
└── ⚪ build     (skipped)   ← It was cancelled because test failed

Level 3: The job

When you click a job, you see all of its steps:

❌ test
├── ✅ Set up job           (2s)
├── ✅ Checkout code        (1s)
├── ✅ Setup Python         (15s)
├── ✅ Install dependencies (45s)
├── ❌ Run tests            (20s)    ← Here's the error
├── ⚪ Post Checkout code   (skipped)
└── ⚪ Complete job         (skipped)

Level 4: The step

When you click a step, you see the full output:

Run pytest tests/ -v
============================= test session starts ==============================
platform linux -- Python 3.12.0, pytest-8.1.1
collected 5 items

tests/test_main.py::test_greet PASSED
tests/test_main.py::test_greet_empty PASSED
tests/test_main.py::test_calculate_cost PASSED
tests/test_main.py::test_calculate_cost_large PASSED
tests/test_main.py::test_api_call FAILED

FAILED tests/test_main.py::test_api_call - ConnectionError: API not available
========================= 1 failed, 4 passed in 2.31s =========================
Error: Process completed with exit code 1.

Key insight: Always scroll to the end of a failed step's log. The error message is at the end, not at the beginning.

The 8 most common types of failure

1. YAML syntax error

Symptom: The workflow doesn't show up in Actions or shows up with an error.

Example:

# ❌ Incorrect indentation
jobs:
  test:
  runs-on: ubuntu-latest    # Should be indented

How to spot it: GitHub shows "Invalid workflow file" in the Actions tab.

Fix: Validate the YAML with yamllint or the VS Code extension. Check the indentation carefully.

2. Dependency not found

Symptom: pip install fails with "No matching distribution found".

Example log:

ERROR: Could not find a version that satisfies the requirement torch==2.5.0
ERROR: No matching distribution found for torch==2.5.0
Error: Process completed with exit code 1.

Fix: Check that the package's name and version are correct. Try without an exact version: torch>=2.0.

3. Import error

Symptom: The script or the tests fail with ModuleNotFoundError.

Example log:

ModuleNotFoundError: No module named 'openai'

Fix: Make sure the dependency is in requirements.txt and that pip install -r requirements.txt runs before the step that needs it.

4. Test failure

Symptom: pytest returns exit code 1.

How to read the output:

FAILED tests/test_main.py::test_api_call
    AssertionError: assert 'error' not in response
    where response = {'error': 'API key not set', 'status': 401}

Fix: Read the assertion message. Does the test expect behavior that depends on a secret/API key that isn't configured in CI? That's a common pattern — you'll solve it in Module 4.

5. Timeout

Symptom: The job is cancelled due to a timeout.

Example:

The job running on runner GitHub Actions 2 has exceeded the maximum execution
time of 360 minutes.

Fix: Add timeout-minutes to the job:

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10

For AI apps, this is critical: a test that calls an LLM API and hangs can run for 6 hours.

6. File not found

Symptom: "No such file or directory".

Example:

python: can't open file '/home/runner/work/myapp/myapp/src/main.py':
[Errno 2] No such file or directory

Most common cause: You forgot actions/checkout@v4. Without checkout, the runner doesn't have your code.

Fix: Make sure actions/checkout@v4 is the first step.

7. Insufficient permissions

Symptom: "Permission denied" or "Resource not accessible by integration".

Example:

Error: Resource not accessible by integration

Cause: The workflow needs permissions it doesn't have. This is most common in PRs from forks.

Fix: Add permissions to the workflow:

jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write

8. Rate limiting

Symptom: A step fails with "rate limit exceeded" or similar.

Example:

ERROR: 429 Too Many Requests - Rate limit exceeded

Cause: Too many requests to an external service (npm, pip, Docker Hub, APIs).

Fix: Implement caching (Module 2) to reduce the number of downloads. For APIs, implement retry logic with backoff.

Debugging techniques

Technique 1: Add diagnostic steps

When you don't understand why something fails, add steps that show the system's state:

steps:
  - uses: actions/checkout@v4

  - name: Debug - Show directory structure
    run: |
      echo "Current directory: $(pwd)"
      echo "Files in repo:"
      ls -la
      echo ""
      echo "Python files:"
      find . -name "*.py" | head -20

  - name: Debug - Show Python info
    run: |
      python --version
      pip --version
      echo "Installed packages:"
      pip list

  - name: Debug - Show environment
    run: |
      echo "HOME: $HOME"
      echo "PATH: $PATH"
      echo "Event: ${{ github.event_name }}"
      echo "Ref: ${{ github.ref }}"
      echo "SHA: ${{ github.sha }}"

Important: Never do echo ${{ secrets.X }} — even though GitHub masks it with ***, it's bad practice.

Technique 2: continue-on-error

When a step fails and you want to see what happens with the following steps:

steps:
  - name: Step that might fail
    continue-on-error: true        # Doesn't stop the job if it fails
    run: pytest tests/ -v

  - name: Debug - Show what happened
    run: |
      echo "Previous step exit code: $?"
      ls -la test-results/ || echo "No test results directory"

continue-on-error: true marks the step as a "soft failure" — the job continues but the step shows up with a ⚠️ in the UI.

Technique 3: Use workflow_dispatch to iterate quickly

Instead of commit → push → wait → check logs → repeat, use workflow_dispatch:

1. Add workflow_dispatch to the trigger (you already have it from capsule 05)
2. Modify the workflow with diagnostic steps
3. Run it manually from Actions
4. Check the logs
5. Adjust and repeat

This is much faster than the commit cycle.

Technique 4: Reproduce locally

Sometimes it's faster to reproduce the problem on your machine:

# Simulate the runner's environment
docker run -it --rm ubuntu:22.04 bash

# Inside the container:
apt-get update && apt-get install -y python3 python3-pip git
git clone https://github.com/YOUR_USERNAME/my-ai-project.git
cd my-ai-project
pip3 install -r requirements.txt
python3 -m pytest tests/ -v

If the test passes on your machine but fails in CI, the problem is in environment differences (Python version, system dependencies, environment variables).

Technique 5: Enable debug logging

GitHub Actions has a debug mode that shows detailed logs:

1. Go to your repo → Settings → Secrets and variables → Actions
2. Add a variable: ACTIONS_RUNNER_DEBUG = true
3. Add a variable: ACTIONS_STEP_DEBUG = true
4. Re-run the workflow

The logs now include detailed information about each step, including the runner's internal configuration.

Note: Turn debug logging off when you're done — it generates far more logs and can slow down runs.

Debugging patterns for AI workflows

Pattern: "Test passes locally, fails in CI"

Checklist:

1. Is the Python version the same? (python --version in both)
2. Are the dependencies the same? (pip freeze in both)
3. Does the test depend on a local file that isn't in the repo?
4. Does the test depend on an environment variable (API_KEY, DATABASE_URL)?
5. Does the test depend on an external service (API, database)?

The most common ones in AI apps:

• API key not configured in CI (you'll solve this in Module 4)
• A test that calls OpenAI without a mock (you'll solve this in testing patterns)
• A dependency that requires system binaries (torch, numpy with BLAS)

Pattern: "The workflow took too long and was cancelled"

Checklist:

1. Do you have timeout-minutes configured?
2. Did a test hang waiting for an API response?
3. Is pip install downloading heavy dependencies without caching?

Quick fix:

jobs:
  test:
    timeout-minutes: 15    # Adjust as needed

Pattern: "The same workflow sometimes passes, sometimes fails"

This is a flaky test — a test that isn't deterministic. Very common in AI apps:

• Tests that call real LLMs (responses vary)
• Tests that depend on timing (timeouts, race conditions)
• Tests that depend on external services (APIs with rate limits)

Fix: Identify the flaky test (it shows up in the logs) and decide how to handle it — mock, retry, or mark it as @pytest.mark.skip temporarily.

Useful tools

GitHub CLI (gh)

# See recent runs
gh run list --limit 5

# See the details of a specific run
gh run view 12345678

# See a run's logs
gh run view 12345678 --log

# Re-run a failed run
gh run rerun 12345678

act — Run workflows locally

# Install
brew install act          # macOS
# or
curl https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash

# Run the workflow
act push                  # Simulates a push event
act workflow_dispatch     # Simulates a manual trigger

Limitation: act doesn't exactly replicate GitHub's environment. It's useful for debugging YAML and logic, but not for environment problems.

Exercises

Exercise 1: Identify the error

This workflow fails. Why?

name: CI
on: push
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Install and test
        run: |
          pip install -r requirements.txt
          pytest tests/ -v

steps:
  - uses: actions/checkout@v4     # Brings the code to the runner
  - run: |
      pip install -r requirements.txt
      pytest tests/ -v

Exercise 2: Add diagnostics

Your workflow fails at the "Run tests" step but you don't understand why. Which diagnostic steps would you add BEFORE the tests step?

steps:
  - uses: actions/checkout@v4

  - name: Debug - Directory contents
    run: |
      echo "Working directory: $(pwd)"
      ls -la
      echo ""
      echo "Tests directory:"
      ls -la tests/ || echo "tests/ does not exist!"

  - name: Debug - Python environment
    run: |
      python --version
      pip list
      echo ""
      echo "PYTHONPATH: $PYTHONPATH"

  - name: Debug - requirements
    run: |
      echo "requirements.txt contents:"
      cat requirements.txt

  - run: pip install -r requirements.txt

  - name: Debug - Installed packages
    run: pip list

  - name: Run tests
    run: pytest tests/ -v --tb=long

Exercise 3: Timeout strategy

Your AI project has 3 types of tests: unit tests (fast, no API), integration tests (they call OpenAI, they can be slow), and end-to-end tests (full pipeline, the slowest). How would you configure timeouts?

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    timeout-minutes: 5               # Unit tests: max 5 min
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements.txt
      - run: pytest tests/unit/ -v --timeout=10

  integration-tests:
    runs-on: ubuntu-latest
    timeout-minutes: 15              # Integration: max 15 min
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements.txt
      - run: pytest tests/integration/ -v --timeout=60

  e2e-tests:
    runs-on: ubuntu-latest
    timeout-minutes: 30              # E2E: max 30 min
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements.txt
      - run: pytest tests/e2e/ -v --timeout=120

Exercise 4: Debug with continue-on-error

Your workflow has a step that fails intermittently and you want to see what information is available when it fails. Modify this step so it doesn't stop the job:

- name: Run AI evaluation
  run: python scripts/evaluate_prompts.py

- name: Run AI evaluation
  id: evaluation
  continue-on-error: true
  run: python scripts/evaluate_prompts.py

- name: Check evaluation result
  if: steps.evaluation.outcome == 'failure'
  run: |
    echo "⚠️ Evaluation failed - collecting debug info"
    echo "Exit code of evaluation step: failure"
    cat evaluation_results.json 2>/dev/null || echo "No results file"
    cat evaluation_errors.log 2>/dev/null || echo "No error log"

Summary

• ✅ Navigate the UI: Actions → workflow run → job → step → log output
• ✅ The error is at the end of a failed step's log — scroll down
• ✅ 8 common types of failure: YAML syntax, dependency, import, test, timeout, file not found, permissions, rate limit
• ✅ 5 debugging techniques: Diagnostic steps, continue-on-error, workflow_dispatch, reproduce locally, debug logging
• ✅ AI-specific patterns: Missing API keys, flaky tests with LLMs, timeouts from slow API calls
• ✅ timeout-minutes is mandatory for AI workflows — a hung test can run for 6 hours
• ✅ GitHub CLI (gh) and act are complementary tools for efficient debugging

Additional resources

1. Monitoring and troubleshooting workflows - Official troubleshooting guide
2. Enabling debug logging - How to turn on debug logs
3. Using workflow run logs - How to read run logs
4. nektos/act - Run workflows locally
5. GitHub CLI - run commands - gh run commands for workflows
6. Troubleshooting GitHub Actions - Official FAQ