Módulo 5: Coverage, TDD y CI

Proyecto: pipeline CI con tests + coverage

Siguiente →

Cierre del Módulo 5. Las cápsulas 01-07 te dieron las técnicas; esta cápsula las integra en un mini-proyecto real que combina:

  • pytest-cov con line + branch coverage (caps 02-04)
  • --cov-fail-under=80 threshold enforcement (cap 04)
  • TDD discipline documentada en CONTRIBUTING (cap 05)
  • GitHub Actions workflow con services + matrix + caching (cap 06)
  • Codecov integration con comments en PRs (cap 07)
  • Branch protection rules que bloquean merges sin tests (cap 07)

Al cerrar, vas a tener un pipeline production-grade que:

  • Corre tests automáticamente en cada push y PR
  • Falla CI si coverage drops debajo de 80%
  • Comenta en cada PR con coverage diff
  • Bloquea merges si tests fallan o coverage drops
  • Genera badges visibles en README

Es el estado production-grade que vas a aplicar al proyecto final del Módulo 6.

El alcance: setup completo

Vamos a configurar para una API FastAPI:

my-api/
├── .github/
│   ├── workflows/
│   │   └── test.yml              # CI workflow
│   └── pull_request_template.md  # PR template
├── codecov.yml                    # Codecov config
├── pyproject.toml                 # pytest-cov config
├── README.md                      # con badges
├── CONTRIBUTING.md                # TDD workflow docs
└── src/
    └── app/
        └── ...

Paso 1: pyproject.toml completo

[project]
name = "my-api"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "fastapi>=0.110",
    "sqlalchemy>=2.0",
    "asyncpg>=0.29",
    "psycopg2-binary>=2.9",
    "pydantic[email]>=2.6",
    "pydantic-settings>=2.2",
    "PyJWT>=2.8",
    "pwdlib[bcrypt]>=0.2",
]

[project.optional-dependencies]
test = [
    "pytest>=8.2",
    "pytest-cov>=5.0",
    "pytest-mock>=3.12",
    "pytest-asyncio>=0.23",
    "pytest-sugar>=1.0",
    "pytest-randomly>=3.15",
    "pytest-clarity>=1.0",
    "factory-boy>=3.3",
    "pytest-factoryboy>=2.7",
    "respx>=0.21",
    "testcontainers[postgres]>=4.0",
    "httpx>=0.27",
]

dev = [
    "ruff>=0.4",
    "mypy>=1.10",
]

[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
where = ["src"]


# ============================================================
# pytest configuration
# ============================================================
[tool.pytest.ini_options]
minversion = "8.0"
testpaths = ["tests"]
pythonpath = ["src"]
addopts = [
    "-ra",
    "--strict-markers",
    "--strict-config",
    "--showlocals",
    "--cov=app",
    "--cov-branch",
    "--cov-report=term-missing",
    "--cov-report=html:htmlcov",
    "--cov-report=xml:coverage.xml",
    "--cov-fail-under=80",
]
markers = [
    "unit: tests aislados",
    "integration: tests con DB",
    "e2e: tests HTTP completos",
    "slow: > 1s",
]
asyncio_mode = "auto"
filterwarnings = [
    "error",
    "ignore::DeprecationWarning:asyncpg",
]


# ============================================================
# Coverage configuration
# ============================================================
[tool.coverage.run]
branch = true
source = ["src/app"]
omit = [
    "*/tests/*",
    "*/migrations/*",
    "*/__init__.py",
    "*/conftest.py",
]


[tool.coverage.report]
exclude_lines = [
    "pragma: no cover",
    "raise NotImplementedError",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
    "@(abc\\.)?abstractmethod",
]
show_missing = true
precision = 2
skip_covered = false


[tool.coverage.html]
directory = "htmlcov"


[tool.coverage.xml]
output = "coverage.xml"


# ============================================================
# Ruff (linting)
# ============================================================
[tool.ruff]
line-length = 100
target-version = "py312"

[tool.ruff.lint]
select = ["E", "F", "W", "I", "N", "UP", "S", "B", "C4", "RUF"]
ignore = [
    "E501",  # line too long (handled by formatter)
    "S101",  # use of assert (OK in tests)
]

[tool.ruff.lint.per-file-ignores]
"tests/**" = ["S101", "S105", "S106"]  # asserts y dummy passwords OK in tests


# ============================================================
# mypy (type checking)
# ============================================================
[tool.mypy]
python_version = "3.12"
strict = true
ignore_missing_imports = true

Paso 2: GitHub Actions workflow

# .github/workflows/test.yml
name: tests

on:
  push:
    branches: [main]
  pull_request:
  workflow_dispatch:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

env:
  FORCE_COLOR: 1
  PYTHONUNBUFFERED: 1

jobs:
  lint:
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
          cache-dependency-path: pyproject.toml

      - name: Install dev dependencies
        run: |
          pip install --upgrade pip
          pip install -e ".[dev]"

      - name: Lint with ruff
        run: ruff check src tests

      - name: Type check with mypy
        run: mypy src

  test:
    needs: lint
    runs-on: ubuntu-latest
    timeout-minutes: 15

    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.12", "3.13"]

    services:
      postgres:
        image: postgres:16-alpine
        env:
          POSTGRES_PASSWORD: test
          POSTGRES_DB: test_db
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v4

      - name: Setup Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: "pip"
          cache-dependency-path: pyproject.toml

      - name: Install dependencies
        run: |
          pip install --upgrade pip
          pip install -e ".[test]"

      - name: Run unit tests
        run: pytest -m "unit" --cov=app --cov-branch --cov-report=xml --cov-report=term

      - name: Run integration tests
        env:
          TEST_DATABASE_URL: postgresql://postgres:test@localhost:5432/test_db
        run: pytest -m "integration" --cov=app --cov-append --cov-report=xml

      - name: Run e2e tests
        env:
          TEST_DATABASE_URL: postgresql://postgres:test@localhost:5432/test_db
        run: pytest -m "e2e" --cov=app --cov-append --cov-report=xml

      - name: Final coverage report
        run: |
          coverage report --fail-under=80
          coverage html

      - name: Upload coverage to Codecov
        if: matrix.python-version == '3.12'
        uses: codecov/codecov-action@v4
        with:
          file: ./coverage.xml
          fail_ci_if_error: false

      - name: Upload artifacts on failure
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: pytest-results-${{ matrix.python-version }}
          path: |
            htmlcov/
            .pytest_cache/
            coverage.xml
          retention-days: 7

  status:
    needs: [lint, test]
    runs-on: ubuntu-latest
    if: always()
    steps:
      - name: All checks passed
        run: |
          if [[ "${{ needs.lint.result }}" != "success" ]] || \
             [[ "${{ needs.test.result }}" != "success" ]]; then
            echo "One or more checks failed"
            exit 1
          fi
          echo "All checks passed!"

Lo que hace este workflow:

  1. lint corre primero (rápido) — ruff + mypy
  2. test corre después si lint pasa, en matrix Python 3.12 + 3.13
  3. Tests separados por marker (unit → integration → e2e) con --cov-append
  4. Codecov upload solo desde Python 3.12 (evitar duplicates)
  5. Artifacts upload si tests fallan
  6. Job status final que pasa solo si lint + test pasaron

Paso 3: codecov.yml

# codecov.yml
coverage:
  precision: 2
  round: down
  range: "70...90"
  status:
    project:
      default:
        target: auto
        threshold: 0.5%
    patch:
      default:
        target: 80%

comment:
  layout: "reach,diff,flags,files"
  behavior: default
  require_changes: true
  show_carryforward_flags: false

flags:
  unit:
    paths:
      - src/app/
    carryforward: true
  integration:
    paths:
      - src/app/
    carryforward: true
  e2e:
    paths:
      - src/app/
    carryforward: true

ignore:
  - "tests/**"
  - "migrations/**"
  - "**/__init__.py"
  - "src/app/cli/**"
  - "src/app/admin/internal/**"

Paso 4: README con badges

# My API

[![tests](https://github.com/USER/REPO/actions/workflows/test.yml/badge.svg)](https://github.com/USER/REPO/actions/workflows/test.yml)
[![codecov](https://codecov.io/gh/USER/REPO/branch/main/graph/badge.svg)](https://codecov.io/gh/USER/REPO)
[![Python](https://img.shields.io/badge/python-3.12+-blue)](https://www.python.org)
[![Ruff](https://img.shields.io/badge/code%20style-ruff-orange)](https://github.com/astral-sh/ruff)

A REST API for ...

## Quick start

\`\`\`bash
pip install -e ".[test]"
pytest
\`\`\`

## Coverage

Coverage threshold: **80%** (line + branch).

See [coverage report](https://codecov.io/gh/USER/REPO).

## Contributing

See [CONTRIBUTING.md](./CONTRIBUTING.md) for development workflow.

Paso 5: CONTRIBUTING.md con TDD workflow

# Contributing

## Development workflow

### Setup

\`\`\`bash
git clone https://github.com/USER/REPO.git
cd REPO
pip install -e ".[test,dev]"
\`\`\`

### TDD workflow

We use TDD for business logic. The cycle:

1. **🔴 RED:** write a failing test
2. **🟢 GREEN:** implement the minimum to pass
3. **🔵 REFACTOR:** improve code while tests stay green

Example commit pattern:

\`\`\`bash
git commit -m "test: validate_password rejects short passwords"
git commit -m "feat: implement min length validation"
git commit -m "test: validate_password rejects long passwords"
git commit -m "feat: implement max length validation"
git commit -m "refactor: extract validation rules to list"
\`\`\`

### Running tests

\`\`\`bash
# All tests
pytest

# Fast feedback (skip slow)
pytest -m "not slow"

# Specific marker
pytest -m "unit"
pytest -m "integration"

# With HTML coverage
pytest --cov-report=html
open htmlcov/index.html
\`\`\`

### Coverage requirements

- Project total: 80% minimum (line + branch)
- New code in PR: 80% minimum (`patch` check in Codecov)

### PR checklist

Before opening PR:

- [ ] Tests added/updated
- [ ] Coverage maintained or improved
- [ ] Linting clean (`ruff check`)
- [ ] Type checking clean (`mypy src`)
- [ ] No secrets in code
- [ ] CHANGELOG updated if user-facing change

CI will verify all of these automatically.

Paso 6: PR template

<!-- .github/pull_request_template.md -->
## Description

<!-- What does this PR do? Why? -->

## Type of change

- [ ] Bug fix (regression test added)
- [ ] New feature (with TDD)
- [ ] Refactor (tests pass before and after)
- [ ] Documentation
- [ ] Performance improvement

## Testing

- [ ] Unit tests added/updated
- [ ] Integration tests added/updated (if applicable)
- [ ] E2E tests added/updated (if applicable)
- [ ] Manual testing performed

## Coverage

- [ ] Coverage maintained or improved
- [ ] New business logic has corresponding tests

## Checklist

- [ ] Code follows project style (`ruff check` clean)
- [ ] Type hints clean (`mypy src` clean)
- [ ] Tests pass locally
- [ ] No secrets / credentials in code
- [ ] Documentation updated if needed
- [ ] CHANGELOG updated for user-facing changes

## Additional context

<!-- Screenshots, context for reviewers, related issues -->

Paso 7: branch protection rules

GitHub UI: Settings → Branches → Branch protection rules → Add rule

Branch name pattern: main

✅ Require a pull request before merging
   ✅ Require approvals: 1
   ✅ Dismiss stale pull request approvals when new commits are pushed
   ✅ Require review from Code Owners (if applicable)

✅ Require status checks to pass before merging
   ✅ Require branches to be up to date before merging

   Required status checks:
     ✅ lint
     ✅ test (3.12)
     ✅ test (3.13)
     ✅ status
     ✅ codecov/project
     ✅ codecov/patch

✅ Require conversation resolution before merging

✅ Require signed commits (optional, recommended)

✅ Do not allow bypassing the above settings
   ✅ Allow administrators to bypass: NO

Paso 8: verificación

Test 1: tests rojos bloquean merge

# Branch con tests rojos
git checkout -b feature/broken
# ... código que rompe tests ...
git push -u origin feature/broken
# Open PR

Resultado esperado:

  • CI corre y falla
  • PR muestra ❌ "1 failing check"
  • Merge button disabled
  • Codecov bot comenta "Coverage drop detected"

Test 2: coverage drop bloquea merge

git checkout -b feature/lower-coverage
# Agregas código sin tests
git push -u origin feature/lower-coverage
# Open PR

Resultado esperado:

  • Tests pasan, pero --cov-fail-under=80 falla en CI
  • Codecov comenta con coverage drop específico
  • Merge button disabled

Test 3: PR válido se mergea

git checkout -b feature/with-tests
# Agregas feature con tests + coverage maintained
git push -u origin feature/with-tests
# Open PR

Resultado esperado:

  • CI verde (lint + test 3.12 + test 3.13 + status + codecov)
  • Codecov comenta "Coverage maintained at 87.5%"
  • 1 approval recibido
  • Merge button enabled

Análisis: lo que tienes ahora

✅ M01-M04 (testing técnico)
   - Unit tests con mocks (M02)
   - Integration tests con DB real (M03)
   - E2E tests con TestClient (M04)

✅ M05 (coverage + TDD + CI — este módulo)
   - Coverage measurement con line + branch
   - --cov-fail-under=80 enforce threshold
   - TDD workflow documented in CONTRIBUTING
   - GitHub Actions con lint + matrix + services
   - Codecov integration con comments en PRs
   - Branch protection con required checks
   - PR template estandarizado

Resultado: test suite profesional con automation completa.

Métricas operacionales

Después de unas semanas con este setup:

Métricas que vas a tener:

- Coverage trend (Codecov dashboard)
- Tiempo promedio de PR check (GitHub Actions)
- Número de PRs bloqueados por tests vs por coverage
- Files con menor coverage (Codecov)
- Tests más lentos (--durations en pytest)


Métricas saludables:

- Coverage: 80-90% line, 70-85% branch
- CI duration: < 10 min total
- Test failures: < 5% de runs (mayormente tests flaky a fixear)
- Coverage trend: estable o creciendo


Métricas warning:

- Coverage decreciendo durante 2+ semanas → focar en tests
- CI duration > 15 min → optimizar (caching, parallel)
- Tests flaky frecuentes → investigar (race conditions)
- Coverage drops grandes en PRs → revisar review process

Resumen del Módulo 5

Cap 01 → Marco mental: las 3 disciplinas
Cap 02 → pytest-cov setup y reportes
Cap 03 → Interpretar coverage (line vs branch)
Cap 04 → Coverage thresholds y exclusions
Cap 05 → TDD workflow red-green-refactor
Cap 06 → GitHub Actions workflow básico
Cap 07 → PR checks + Codecov + badges
Cap 08 → Mini-proyecto integrador (este)

Tienes:

  • Coverage measurement automático en cada test run
  • Threshold enforcement que falla CI si coverage drops
  • TDD discipline documentada para el equipo
  • GitHub Actions con lint + tests matrix + services
  • Codecov integration con comments por PR
  • Branch protection que bloquea merges sin tests
  • README badges visibles
  • PR template estandarizado
  • CONTRIBUTING que onboardea developers nuevos

Checkpoint del módulo completo

Antes de avanzar al Módulo 6 (Proyecto Final), deberías poder:

  • ✅ Configurar pytest-cov con line + branch coverage
  • ✅ Aplicar --cov-fail-under=80 enforced en CI
  • ✅ Ejecutar un ciclo TDD red-green-refactor
  • ✅ Escribir GitHub Actions workflow con services + matrix
  • ✅ Setup Codecov con comments en PRs
  • ✅ Configurar branch protection con required checks
  • ✅ Aplicar todo lo anterior a tu proyecto real

Puente al Módulo 6

Tienes:

  • 5 módulos de técnica: pytest, mocking, integration, E2E, coverage/CI
  • 44 cápsulas cubriendo cada aspecto

El Módulo 6 es el capstone: construir un test suite production-grade completo para una API real, integrando TODO lo aprendido. Vas a:

  • Tomar una API existente (o usar la del path)
  • Aplicar test pyramid: 70% unit + 25% integration + 5% e2e
  • Coverage 80%+ enforced
  • CI pipeline completo
  • Documentación de testing strategy
  • Reporte presentable como portfolio piece

M06 es donde internalizas todo porque lo haces end-to-end en un proyecto real, no aislado.

Recursos

  1. pytest-cov docs — referencia
  2. GitHub Actions docs — referencia
  3. Codecov docs — referencia
  4. Kent Beck — TDD — el libro de TDD
  5. Continuous Delivery — Jez Humble — perspectiva amplia de CI
  6. Cosmic Python — patterns de architecture testeable
  7. Real Python — Testing — overview comprehensive
← AnteriorPR checks, coverage badges, reportesSiguiente →Módulo 6: Proyecto — Test Suite Completo