7. Quality Gates — Blocking the Merge

Overview

So far, your CI checks run on every push and PR. If they fail, you see a red icon on the PR. But you can click "Merge pull request" anyway. The checks are informational, not mandatory.

A quality gate is a check that MUST pass before a PR can be merged. It's not a warning — it's a real block: GitHub disables the merge button until every required check is green. Trying to merge a PR that fails a required check is impossible.

The difference between "we have CI" and "we have quality gates" is the difference between "we know something is failing" and "we can't ship something that's failing." It's enforcement, not monitoring.

In the context of AI applications, quality gates are especially important. A change to a system prompt can degrade the quality of the responses without breaking any unit test. If the prompt regression test and the cost estimation are informational checks that can be ignored, eventually someone will ignore them. If they're quality gates, quality and cost are protected automatically.

This capsule teaches you to configure GitHub Branch Protection Rules to turn your CI checks into mandatory quality gates.

What a quality gate is

In CI/CD, a quality gate is a checkpoint that must pass in order to move forward — like an airport's security control. They're not suggestions; if the check fails, you stop.

Developer → Push code → CI checks run → Do they all pass?
                                              │
                                        ┌─────┴─────┐
                                        │ YES        │ NO
                                        │ Merge      │ Blocked
                                        │ enabled    │ Can't
                                        └───────────┘ merge

Quality gate vs status check

The technical mechanism is: GitHub Branch Protection Rules + Required Status Checks = Quality Gates.

Configuring Branch Protection Rules

Step 1: Go to the repository's settings

GitHub → Your repository → Settings → Branches

In the "Branch protection rules" section, click "Add branch protection rule".

Step 2: Define the protected branch

In "Branch name pattern", type:

main

This applies the rule only to the main branch. You can also use patterns like release/* for all the release branches.

Step 3: Enable "Require status checks to pass before merging"

☑ Require status checks to pass before merging

Now you need to select which checks are mandatory.

Step 4: Select the required status checks

Below the checkbox, a search field appears. The names that show up are the names of the jobs in your workflows:

jobs:
  lint-and-test:        # ← This name appears as a status check
    runs-on: ubuntu-latest
    steps: ...

  docker-build:         # ← This name appears as a status check
    runs-on: ubuntu-latest
    steps: ...

Step 5: Enable "Require branches to be up to date"

☑ Require branches to be up to date before merging

This requires the PR's branch to be up to date with main before merging. If main moved forward since you created the PR, you must sync first.

Step 6: Save the rule

Click "Create" or "Save changes".

The experience: What the developer sees

A PR with every check passing

Checks
  ✅ lint-and-test — All checks passed
  ✅ docker-build — Build successful

[Merge pull request] ← Green button, enabled

A PR with a failing check

Checks
  ✅ lint-and-test — All checks passed
  ❌ docker-build — Build failed

⚠️ Merging is blocked
  Required status check "docker-build" is failing

[Merge pull request] ← Gray button, disabled

There is no way to merge it (unless you're an admin with bypass).

Configuring quality gates for AI checks

The checks that should be mandatory

The checks that could be optional

Example: A workflow with clear checks

name: CI Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  # Separate jobs let each check fail independently — reviewers see exactly which gate blocked the merge
  lint:
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install ruff mypy
      - run: ruff check src/ tests/
      - run: mypy src/ --ignore-missing-imports

  test:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install -r requirements.txt
      - run: pytest tests/ -v --timeout=30 --junitxml=test-results/report.xml
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-results
          path: test-results/
          retention-days: 30

  prompt-regression:
    runs-on: ubuntu-latest
    # Higher timeout than lint/cost because LLM API calls add latency and can occasionally retry
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install -r requirements.txt
      - run: python scripts/evaluate_prompts.py --baseline baselines/prompts.json

  cost-estimation:
    runs-on: ubuntu-latest
    # Cost check uses tiktoken locally — no API key, deterministic, safe as a required check
    timeout-minutes: 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install -r requirements.txt
      - run: python scripts/estimate_cost.py --threshold 5.00

  docker-build:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - run: docker build -t myapp:test .
      - run: docker run --rm myapp:test python -c "from src.main import app; print('OK')"

In Branch Protection, you select all of these jobs as required.

Selecting checks: All vs selective

Option 1: Every check is required

The advantage: Nothing gets merged without passing everything. The risk: If a check fails for external reasons (e.g. the OpenAI API is down), nobody can merge anything.

Option 2: Only deterministic checks

The advantage: The checks that depend on external APIs don't block the merge. The risk: Someone can merge a PR that degrades the prompts or raises the costs.

The recommendation for AI projects

Use a mixed approach:

1. Required: lint, test, docker-build, cost-estimation (deterministic — it counts tokens without calling the API)
2. Required with a fallback: prompt-regression using deterministic evaluations (keyword matching) as the required one
3. Informational: Integration tests with the real API

Admin bypass

Branch Protection has an option:

☐ Do not allow bypassing the above settings

If it is NOT checked, administrators can merge PRs even when the checks fail. If it IS checked, not even admins can bypass.

For most teams, leave bypass enabled for admins but establish a social rule: bypass is only used in emergencies. If it gets used more than once a month, the checks are too strict or the team isn't respecting the process.

Additional Branch Protection rules

The complete recommended configuration

Branch protection rule for: main

☑ Require a pull request before merging
  ☑ Require approvals: 1
  ☑ Dismiss stale approvals on new pushes

☑ Require status checks to pass before merging
  ☑ Require branches to be up to date
  Required checks: lint, test, cost-estimation, docker-build

☑ Require conversation resolution before merging

☐ Require signed commits
☐ Require linear history

☐ Do not allow bypassing the above settings
  (leave bypass for admins in emergencies)

Status checks in the PR's UI

The possible states

Re-running failed checks

If a check fails for transient reasons (e.g. a network timeout):

1. Click "Details" on the failed check
2. On the workflow run's page, click "Re-run failed jobs"

Also from the CLI:

gh run rerun <run-id> --failed

name: AI Quality Checks

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint:
    name: Lint & Type Check
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install ruff mypy
      - run: ruff check src/ tests/
      - run: mypy src/ --ignore-missing-imports

  test:
    name: Unit Tests
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install -r requirements.txt
      - run: pytest tests/ -v --timeout=30

  docker:
    name: Docker Build
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - run: docker build -t myapp:test .

git add .github/workflows/ci.yml
git commit -m "Add CI workflow with lint, test, and Docker build"
git push origin main

Settings → Branches → Add branch protection rule
  Branch name pattern: main
  ☑ Require a pull request before merging
  ☑ Require status checks to pass before merging
    Search: "Lint" → select "Lint & Type Check"
    Search: "Unit" → select "Unit Tests"
    Search: "Docker" → select "Docker Build"
  → Create

git checkout -b test/break-lint
echo "x=1" >> src/main.py
git add src/main.py
git commit -m "Test: break lint"
git push origin test/break-lint

Comparisons

Quality gates: Informational vs mandatory

All required vs selective

Troubleshooting

1. The check doesn't appear in the status checks list

Cause: The workflow has never run on the main branch. GitHub only shows checks that have reported a status at least once.

Solution: Push the workflow to main first, wait for it to complete, and then the checks appear in Branch Protection.

2. "Merge is blocked" but every check is green

Cause: A required check that no longer exists in the workflow but is still configured in Branch Protection, or the workflow didn't trigger and the check is in the "Expected" state.

Solution:

Settings → Branches → Edit rule
  Review the list of required checks
  Remove checks that no longer exist in your workflows
  Save changes

3. I want a check to be required only for PRs, not for pushes to main

Solution: That's the correct behavior if you use Require a pull request before merging. With this, nobody can push directly to main — everything goes through a PR, and the PR requires the checks.

Exercises

Exercise 1: Design a quality gate strategy

Your AI project has these checks in CI:

1. ruff check — Linter
2. mypy — Type checker
3. pytest tests/unit/ — Unit tests
4. pytest tests/integration/ — Integration tests (they call OpenAI)
5. python scripts/evaluate_prompts.py — Prompt regression (keyword matching)
6. python scripts/estimate_cost.py — Cost estimation (counts tokens)
7. docker build — Docker build test

Define which ones should be required status checks and which ones informational.

Exercise 2: Configure Branch Protection step by step

Write the instructions for configuring Branch Protection with these requirements: protect main, require 1 review, require the lint, test, docker-build checks, require the branch to be up to date, allow bypass for admins.

Branch name pattern: main

☑ Require a pull request before merging
  ☑ Required approving reviews: 1
  ☑ Dismiss stale pull request approvals when new pushes are received
  ☐ Require review from Code Owners

☑ Require status checks to pass before merging
  ☑ Require branches to be up to date before merging
  Search and select: lint, test, docker-build

☑ Require conversation resolution before merging

☐ Require signed commits
☐ Require linear history

☐ Do not allow bypassing the above settings
  (leave it unchecked = admins can bypass)

Exercise 3: A workflow with clear check names

Rewrite this workflow so the status check names are descriptive:

name: CI
on: [push, pull_request]
jobs:
  job1:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: ruff check src/
      - run: mypy src/
      - run: pytest tests/ -v
      - run: docker build -t app:test .

name: CI Pipeline
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint:
    name: Lint & Type Check
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install ruff mypy
      - run: ruff check src/ tests/
      - run: mypy src/ --ignore-missing-imports

  test:
    name: Unit Tests
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install -r requirements.txt
      - run: pytest tests/ -v --timeout=30

  docker-build:
    name: Docker Build
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - run: docker build -t app:test .
      - run: docker run --rm app:test python -c "from src.main import app; print('OK')"

Exercise 4: Handle a temporarily broken check

Your CI has 5 required checks. prompt-regression fails because the script has a bug (not because of the PR's code). Nobody can merge. What do you do?

Summary

• ✅ A quality gate is a check that MUST pass — the merge is blocked if it fails
• ✅ Branch Protection Rules on GitHub configure the quality gates
• ✅ Required status checks are the names of your workflow's jobs
• ✅ "Require branches to be up to date" forces the checks to re-run when main moves forward
• ✅ For AI: lint, type check, unit tests, prompt regression, cost estimation, Docker build as required
• ✅ Integration tests with external APIs are better as informational checks (not required)
• ✅ Admin bypass is for emergencies — not for convenience
• ✅ The checks must run at least once on main to appear in Branch Protection
• ✅ Descriptive name: values on the jobs make it easy to find the checks

Additional resources

1. Managing a branch protection rule — GitHub's official guide
2. About protected branches — Concepts and options
3. Required status checks — Troubleshooting status checks
4. About status checks — How status checks work