Módulo 5: Coverage, TDD y CI

PR checks, coverage badges, reportes

Siguiente →

Tienes CI corriendo (cap 06). Pero CI sin enforcement es información, no obligación. Si los tests fallan en CI pero el merge button sigue habilitado, el equipo puede ignorar los failures. Y sin visibility del coverage en cada PR, los reviewers no saben si el código nuevo bajó la cobertura.

Esta cápsula cierra el módulo con los mecanismos operacionales que convierten CI de "información" a "guardian de calidad":

  • Required status checks: PRs con tests rojos NO se pueden mergear
  • Coverage badges: README muestra coverage actual visible
  • Codecov integration: comments en PRs con coverage diff por archivo
  • Branch protection rules: main no acepta merges sin reviews + checks

En esta cápsula vas a configurar branch protection en GitHub, agregar coverage badge al README, integrar Codecov para tracking de trends, y conocer patterns para PR comments automatizados.

Al cerrar, tu CI va a bloquear merges que rompen tests o bajan coverage. Calidad enforced, no opcional.

Required status checks: el guardian principal

Sin required checks, este escenario es real:

PR abierto
  ↓
CI corre
  ↓
Tests fallan (rojo)
  ↓
Developer mergea de todos modos ("urgent fix")
  ↓
Producción rota

Required checks evita esto. Configuración:

Branch protection rules

GitHub UI: Settings → Branches → Branch protection rules → Add rule

Branch name pattern: main

✅ Require a pull request before merging
   ✅ Require approvals: 1
   ✅ Dismiss stale pull request approvals when new commits are pushed

✅ Require status checks to pass before merging
   ✅ Require branches to be up to date before merging

   Status checks that are required:
     ✅ test (3.12)        ← del workflow
     ✅ test (3.13)
     ✅ codecov/project    ← Codecov check
     ✅ codecov/patch      ← Codecov check para PR

✅ Require conversation resolution before merging

✅ Do not allow bypassing the above settings

Configurado: PR con tests rojos NO tiene merge button habilitado. Punto.

Configurar via terraform/IaC

Para teams que infra-as-code, GitHub admin:

# terraform
resource "github_branch_protection" "main" {
  repository_id = github_repository.main.node_id
  pattern       = "main"

  required_status_checks {
    strict = true
    contexts = [
      "test (3.12)",
      "test (3.13)",
      "codecov/project",
    ]
  }

  required_pull_request_reviews {
    required_approving_review_count = 1
    dismiss_stale_reviews            = true
  }
}

Coverage badge en README

Visualidad importa. Badge en README muestra coverage actual a primera vista:

# My Project

![tests](https://github.com/your-org/repo/actions/workflows/test.yml/badge.svg)
[![codecov](https://codecov.io/gh/your-org/repo/branch/main/graph/badge.svg)](https://codecov.io/gh/your-org/repo)

A backend API for...

Resultado en GitHub:

[tests] passing  [codecov] 87%

Badge from GitHub Actions

GitHub provee badges automáticos para workflows:

![workflow status](https://github.com/USER/REPO/actions/workflows/test.yml/badge.svg)

Sustituí USER y REPO. No requiere configuración adicional. Badge actualiza automáticamente.

Badge from Codecov

[![codecov](https://codecov.io/gh/USER/REPO/branch/main/graph/badge.svg)](https://codecov.io/gh/USER/REPO)

Requiere setup de Codecov (próxima sección). El número del badge actualiza cuando se sube nuevo coverage.

Otros badges útiles

# Python version
![Python](https://img.shields.io/badge/python-3.12+-blue)

# License
![License](https://img.shields.io/badge/license-MIT-green)

# pytest
![pytest](https://img.shields.io/badge/tested%20with-pytest-46a2f1)

# Code style
![Ruff](https://img.shields.io/badge/code%20style-ruff-orange)

Shields.io genera badges custom. Útil para metadata del proyecto.

Codecov integration

Codecov es la herramienta más popular para coverage tracking. Free para projects open source, paid plans for private repos.

Setup en 3 pasos

1. Activar Codecov

  • Login en codecov.io con GitHub
  • Authorize Codecov for your repo
  • Codecov detecta automáticamente

2. Workflow upload

Ya cubierto en cap 06:

- name: Upload coverage
  uses: codecov/codecov-action@v4
  with:
    file: ./coverage.xml
    fail_ci_if_error: false

3. Verify

Después de un PR, Codecov posts comment con coverage diff:

Codecov Report

@@           Coverage Diff           @@
##             main      #42      +/-
=============================================
+ Coverage    87.5%    88.2%    +0.7%
=============================================
+ Files          25       26       +1
  Lines        425      432       +7
=============================================
+ Hits         372      381       +9
- Misses        53       51       -2

Files Changed Coverage      Δ
src/app/services/orders.py  92.3%   <100.0%>   (+1.5%)   ⬆️
src/app/repositories.py     85.0%   <80.0%>    (-5.0%)   ⬇️

Coverage diff por file identifica donde el PR cambió coverage. Útil para review.

Codecov YAML config

# codecov.yml (en root del repo)
coverage:
  status:
    project:
      default:
        target: auto                # tracking automático
        threshold: 0.5%              # permite drop hasta 0.5% sin fail
    patch:
      default:
        target: 80%                  # patches deben tener 80%+ coverage

comment:
  layout: "reach,diff,flags,files"
  behavior: default
  require_changes: true              # solo comentar si hay cambios significativos

flags:
  unit:
    paths:
      - src/app/
    carryforward: true                # mantener si flag no se ejecutó

ignore:
  - "tests/**"
  - "migrations/**"
  - "**/__init__.py"

Sección por sección:

  • coverage.status.project: check del coverage total
    • target: auto — track tendencia (no hardcode 80%)
    • threshold: 0.5% — drop pequeño OK, drop grande falla
  • coverage.status.patch: check del coverage de las líneas nuevas en el PR
    • target: 80% — código nuevo debe tener 80% coverage mínimo
  • comment: configuración del comment automático
  • ignore: files a excluir del coverage tracking

Codecov badges con flags

Si tienes multiple test types (unit, integration, e2e), trackeá separately:

# .github/workflows/test.yml
- name: Upload unit coverage
  uses: codecov/codecov-action@v4
  with:
    file: ./coverage-unit.xml
    flags: unit

- name: Upload integration coverage
  uses: codecov/codecov-action@v4
  with:
    file: ./coverage-integration.xml
    flags: integration
[![unit](https://codecov.io/gh/USER/REPO/branch/main/graph/badge.svg?flag=unit)](https://codecov.io/gh/USER/REPO)
[![integration](https://codecov.io/gh/USER/REPO/branch/main/graph/badge.svg?flag=integration)](https://codecov.io/gh/USER/REPO)

Badges separados por type. Útil cuando integration coverage es naturalmente menor que unit.

PR comments con coverage diff

Codecov comment en PRs es el mecanismo principal de visibility. Pero puedes agregar otros:

Pattern: pytest output como PR comment

# .github/workflows/test.yml
- name: Run tests with summary
  run: pytest --cov=app --cov-report=term-missing | tee pytest-output.txt

- name: Comment PR with test results
  if: github.event_name == 'pull_request'
  uses: actions/github-script@v7
  with:
    script: |
      const fs = require('fs');
      const output = fs.readFileSync('pytest-output.txt', 'utf8');
      const summary = output.split('===').slice(-3).join('===');

      github.rest.issues.createComment({
        issue_number: context.issue.number,
        owner: context.repo.owner,
        repo: context.repo.repo,
        body: '```\n' + summary + '\n```',
      });

Comment con summary del pytest output en cada PR. Útil pero verbose — mejor reservar para cuando Codecov no es suficiente.

Pattern: GitHub Actions Job Summary

- name: Generate summary
  if: always()
  run: |
    echo "## Test Results" >> $GITHUB_STEP_SUMMARY
    echo "" >> $GITHUB_STEP_SUMMARY
    pytest --cov=app --cov-report=term-missing >> $GITHUB_STEP_SUMMARY

$GITHUB_STEP_SUMMARY aparece en la UI del workflow run. Visible sin clicar en steps específicos.

Pattern: PR labels automáticos

- name: Label PR if coverage drops
  if: github.event_name == 'pull_request'
  uses: actions-ecosystem/action-add-labels@v1
  with:
    labels: 'coverage-decreased'
  # Conditional logic to detect drop

Auto-labels para destacar PRs problemáticos. Útil para visibility en the PR list.

Patterns para CI más útil

Pattern: fail fast en lint, slow tests last

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
      - run: pip install ruff mypy
      - run: ruff check .
      - run: mypy src/

  unit-tests:
    needs: lint  # ← solo si lint pasa
    runs-on: ubuntu-latest
    steps: ...

  integration-tests:
    needs: unit-tests
    runs-on: ubuntu-latest
    services:
      postgres: ...
    steps: ...

  e2e-tests:
    needs: integration-tests
    runs-on: ubuntu-latest
    steps: ...

Pipeline lineal:

  1. Lint: 30 segundos. Falla rápido si hay typos.
  2. Unit: 1 minuto. Falla si lógica está rota.
  3. Integration: 2 minutos. Falla si DB queries/integrations fallan.
  4. E2E: 3 minutos. Última línea de defense.

Total worst case: 6.5 min, pero suele fallar antes.

Pattern: changed files only

- name: Get changed files
  id: changed
  uses: tj-actions/changed-files@v44
  with:
    files: |
      src/**
      tests/**

- name: Run tests on changed
  if: steps.changed.outputs.any_changed == 'true'
  run: pytest

Solo corre tests si código relevante cambió. Útil para repos monorepo o cuando docs-only PRs son comunes.

Pattern: schedule full coverage daily

on:
  pull_request: ...
  schedule:
    - cron: '0 6 * * *'  # daily 6am UTC

jobs:
  full-test:
    runs-on: ubuntu-latest
    steps:
      - run: pytest --cov=app -m "not slow"  # PRs

  full-test-with-slow:
    if: github.event_name == 'schedule'
    runs-on: ubuntu-latest
    steps:
      - run: pytest --cov=app  # incluye slow

Daily run con tests slow + scheduled jobs. PRs son fast.

Patterns avanzados: PR templates

GitHub permite PR templates que pre-llenan el body del PR:

<!-- .github/pull_request_template.md -->
## Description
<!-- What does this PR do? -->

## Type of change
- [ ] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Refactor
- [ ] Documentation

## Testing
- [ ] Unit tests added/updated
- [ ] Integration tests added/updated
- [ ] E2E tests added/updated (if applicable)
- [ ] Manual testing performed

## Coverage
- [ ] Coverage maintained or improved
- [ ] No new untested business logic

## Checklist
- [ ] Code follows project style
- [ ] Tests passing locally
- [ ] Documentation updated
- [ ] No secrets in code

Reviewers ven el template automáticamente. Standardiza qué se valida.

Required reviews + auto-merge

Required reviews

Settings → Branches → main → ☑️ Require approvals: 1

PRs requieren al menos 1 approval antes de merge. Combina con required checks para enforcement completo.

Auto-merge para Dependabot

Dependabot abre PRs para dependency updates. Mergear cada uno manualmente es overhead:

# .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: "pip"
    directory: "/"
    schedule:
      interval: "weekly"
    open-pull-requests-limit: 5
# .github/workflows/auto-merge.yml
name: Auto-merge Dependabot

on: pull_request

jobs:
  auto-merge:
    if: github.actor == 'dependabot[bot]'
    runs-on: ubuntu-latest
    steps:
      - name: Auto-merge minor and patch updates
        uses: dependabot/fetch-metadata@v2
        id: dependabot-metadata
      - if: steps.dependabot-metadata.outputs.update-type == 'version-update:semver-minor' ||
            steps.dependabot-metadata.outputs.update-type == 'version-update:semver-patch'
        run: gh pr merge --auto --merge "$PR_URL"
        env:
          PR_URL: ${{ github.event.pull_request.html_url }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Auto-merge: updates minor/patch (low risk) merging automáticamente cuando CI passes. Major updates requieren review manual.

Visualization: Codecov dashboard

Codecov dashboard provee:

Coverage trends:
  ┌──────────────────────────┐
  │ Coverage over time:        │
  │ ┌─────────────────────┐    │
  │ │     ╱─╲             │    │
  │ │  ╱─╯   ╲╱─╲         │    │
  │ │╱──        ─╲────    │    │
  │ └─────────────────────┘    │
  │ Jan  Feb  Mar  Apr  May    │
  │ 75%  78%  81%  82%  84%    │
  └──────────────────────────┘

Files most affecting coverage:
  src/app/services/orders.py     -2.5%
  src/app/auth.py                +1.0%
  src/app/repositories.py        +0.5%

Coverage by tag:
  unit: 92.5%
  integration: 78.3%
  e2e: 65.2%

Métricas útiles:

  • Coverage trend identifica deterioration gradual
  • Files with most impact identifica donde focar tests
  • Coverage by tag muestra balance entre tipos de tests

Trampas y errores comunes

Trampa 1: required checks sin enforce

Branch protection: ✅ Require status checks to pass
                   ❌ Do not allow bypassing  ← falta

Sin "Do not allow bypassing", admins pueden mergear con tests rojos. Para enforcement real, marcalo.

Trampa 2: status checks names incorrectos

# Workflow
jobs:
  test:  # ← job name
Branch protection:
  Required: "Tests"  ← incorrecto, no matchea
  Required: "test"   ← correcto si usas matrix
  Required: "test (3.12)"  ← correcto si matrix con python-version

Si los names no matchean exactly, las checks no enforce.

Trampa 3: Codecov sin token (private repos)

Para projects públicos: Codecov funciona sin token. Para private repos: necesitas token de Codecov agregado a GitHub Secrets:

- uses: codecov/codecov-action@v4
  with:
    token: ${{ secrets.CODECOV_TOKEN }}  # ← required for private
    file: ./coverage.xml

Trampa 4: badge URL hardcoded a una branch

[![coverage](https://codecov.io/gh/USER/REPO/branch/main/graph/badge.svg)]
                                                    ^^^ branch hardcoded

Si renombras main a develop, badge se rompe. Generalmente OK porque main es estable. Pero ten en cuenta.

Trampa 5: workflow no triggers en PRs from forks

PRs from forks (typical en open source) tienen restricciones:

  • Secrets NO disponibles (security)
  • pull_request event triggers el workflow
  • pull_request_target triggers con secrets pero es peligroso si workflow corre código del PR (security risk)

Para forks: workflow básico sin secrets. Codecov upload puede fallar — fail_ci_if_error: false evita romper.

Trampa 6: comment spam en PRs

Si Codecov + workflow + bot custom todos comentan:

PR comments:
  Codecov bot: 1 comment
  CI bot: 1 comment
  Coverage bot: 1 comment
  ...

Cluttered. Configura un solo bot principal (Codecov default suficiente).

Trampa 7: required checks que nunca corren

on:
  push:
    branches: [main]  # ← solo main
Branch protection requires: "test"

Si workflow solo corre en push a main (no en PRs), PRs no triggern el check, el check no aparece, y como nunca aparece, GitHub considera "passing" automáticamente.

Fix: workflow debe triggern en PRs.

on:
  push:
    branches: [main]
  pull_request:

Ejercicio: configurar branch protection + Codecov

  1. Setup Codecov:
    • Login en codecov.io con GitHub
    • Activá tu repo
    • Verifica que el badge funciona en el README
  2. Crea codecov.yml con config recomendada (project + patch checks)
  3. Configura branch protection:
    • Settings → Branches → Add rule
    • Branch pattern: main
    • Require approvals: 1
    • Require status checks (incluí los del workflow + Codecov)
    • Do not allow bypassing
  4. Crea PR template en .github/pull_request_template.md
  5. Test:
    • Push branch con tests rojos → verifica que merge button está disabled
    • Push branch con tests verdes pero coverage drop → verifica comment de Codecov
Solución de configuración 
# codecov.yml
coverage:
  status:
    project:
      default:
        target: auto
        threshold: 0.5%
    patch:
      default:
        target: 80%

comment:
  layout: "reach,diff,flags,files"
  require_changes: true

ignore:
  - "tests/**"
  - "migrations/**"
  - "**/__init__.py"
  - "src/app/cli/**"
<!-- .github/pull_request_template.md -->
## Description
<!-- What and why -->

## Type
- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Docs

## Testing
- [ ] Tests added/updated
- [ ] Coverage maintained/improved
- [ ] Manually tested

## Checklist
- [ ] Tests passing
- [ ] Linting clean
- [ ] Docs updated
- [ ] No secrets
Branch protection rules for main:
  ✅ Require pull request before merging
     ✅ Require approvals: 1
  ✅ Require status checks to pass
     - test (3.12)
     - test (3.13)
     - codecov/project
     - codecov/patch
  ✅ Require branches to be up to date
  ✅ Do not allow bypassing

Resumen y siguiente paso

Lo que aprendiste en esta cápsula:

  • Required status checks bloquean merges con tests rojos
  • Branch protection rules configuran reviews + checks + bypass settings
  • Coverage badges en README muestran status visible
  • Codecov integration provee comments en PRs con coverage diff
  • codecov.yml configura targets, thresholds, ignores
  • PR templates standardizan information per PR
  • Auto-merge para Dependabot automatiza dependency updates seguros
  • Trampas: required check names mal escritos, secrets en forks, comment spam

Checkpoint antes de avanzar

Antes de continuar a la última cápsula, deberías poder:

  • ✅ Configurar branch protection con required checks
  • ✅ Agregar coverage badge al README
  • ✅ Setup Codecov para tu repo
  • ✅ Configurar codecov.yml con project + patch targets
  • ✅ Crear PR template estándar para tu organización

Puente a la última cápsula

Tienes todo:

  • Coverage measurement + interpretation + thresholds (caps 02-04)
  • TDD workflow (cap 05)
  • GitHub Actions workflow (cap 06)
  • PR enforcement + Codecov (esta cap 07)

La cápsula 08 es el mini-proyecto integrador que combina todo en un setup completo:

  • Coverage 80%+ con threshold enforced
  • TDD workflow documented
  • GitHub Actions workflow optimizado
  • Codecov integration funcional
  • Branch protection rules configuradas

Es el estado production-grade que vas a aplicar al proyecto del Módulo 6.

Recursos

  1. GitHub branch protection rules
  2. Codecov docs — referencia completa
  3. Codecov YAML reference — config detallada
  4. Shields.io — generadores de badges
  5. GitHub PR templates
  6. Dependabot config — para auto-merge setup
← AnteriorGitHub Actions: workflow básicoSiguiente →Proyecto: pipeline CI con tests + coverage