Módulo 2: Bias and Fairness — Detection, Measurement, Mitigation

8. Mini-proyecto: Bias Audit Toolkit

Siguiente →

Cerrás el módulo construyendo un Bias Audit Toolkit — un repo Python reusable con métricas, tests, mitigations, y dashboard. Lo vas a aplicar a sistemas AI futuros, refinándolo a medida que aprendés más.

Es el equivalente del Ethics Impact Analysis (M1/08) pero operacional: en lugar de un documento, es código que corre y produce reportes.

Especificación del entregable

Mínimo viable

  • ✅ Library Python con módulos para:
    • Fairness metrics (demographic parity, equalized odds, calibration).
    • Bias detection (slicing, counterfactual).
    • Mitigation (pre-processing baseline).
  • ✅ Tests unitarios verificando cada métrica con datasets sintéticos.
  • ✅ Script audit que toma un modelo + dataset y genera reporte completo.
  • ✅ CLI: corrible desde command line.
  • ✅ README documentando proceso de audit.

Stretch goals

  • ⭐ Integrar en CI de un sistema real: fairness gate como pytest test.
  • ⭐ Dashboard Streamlit/Gradio con visualizations.
  • ⭐ Más mitigations: implementar adversarial debiasing.
  • ⭐ Multi-attribute slicing intersectional automatizado.
  • ⭐ Integration con AIF360 o Fairlearn para benchmarking.

Estructura del repo

bias-audit-toolkit/
├── README.md
├── pyproject.toml
├── src/
│   ├── __init__.py
│   ├── metrics/
│   │   ├── __init__.py
│   │   ├── demographic_parity.py
│   │   ├── equalized_odds.py
│   │   └── calibration.py
│   ├── detection/
│   │   ├── __init__.py
│   │   ├── slicing.py
│   │   └── counterfactual.py
│   ├── mitigation/
│   │   ├── __init__.py
│   │   ├── reweighting.py
│   │   └── threshold_tuning.py
│   ├── audit.py        ← orquestador
│   └── cli.py          ← entry point
├── tests/
│   ├── test_demographic_parity.py
│   ├── test_equalized_odds.py
│   ├── test_calibration.py
│   ├── test_slicing.py
│   └── test_counterfactual.py
└── examples/
    ├── audit_example.py
    └── data/
        └── synthetic_loan_data.csv

Implementación paso a paso

Paso 1: setup del repo

mkdir bias-audit-toolkit
cd bias-audit-toolkit
git init
python -m venv .venv
source .venv/bin/activate
pip install pandas numpy scikit-learn matplotlib seaborn pytest

pyproject.toml:

[project]
name = "bias-audit-toolkit"
version = "0.1.0"
description = "Toolkit for auditing bias in AI systems"
dependencies = [
    "pandas",
    "numpy",
    "scikit-learn",
    "matplotlib",
    "seaborn",
]

[project.optional-dependencies]
dev = ["pytest", "ruff", "mypy"]

Paso 2: implementar metrics

src/metrics/demographic_parity.py:

import pandas as pd

def demographic_parity(predictions, groups, favorable_outcome=1):
    """
    Calcula demographic parity ratio.
    
    Returns:
        dict con rates per group, ratio, y meets_4_5_rule.
    """
    df = pd.DataFrame({
        'prediction': predictions,
        'group': groups,
    })
    
    rates = df.groupby('group')['prediction'].apply(
        lambda x: (x == favorable_outcome).mean()
    )
    
    if len(rates) < 2:
        return {'error': 'Need at least 2 groups'}
    
    parity_ratio = rates.min() / rates.max() if rates.max() > 0 else 0
    
    return {
        'rates_by_group': rates.to_dict(),
        'parity_ratio': parity_ratio,
        'meets_4_5_rule': parity_ratio >= 0.80,
    }

src/metrics/equalized_odds.py:

from sklearn.metrics import confusion_matrix
import pandas as pd

def equalized_odds(predictions, ground_truth, groups, favorable_outcome=1):
    """
    Calcula TPR y FPR per group.
    """
    df = pd.DataFrame({
        'pred': predictions, 
        'true': ground_truth, 
        'group': groups
    })
    
    metrics_per_group = {}
    for group_name, group_df in df.groupby('group'):
        y_true = (group_df['true'] == favorable_outcome).astype(int)
        y_pred = (group_df['pred'] == favorable_outcome).astype(int)
        
        cm = confusion_matrix(y_true, y_pred, labels=[0, 1])
        tn, fp, fn, tp = cm.ravel()
        
        tpr = tp / (tp + fn) if (tp + fn) > 0 else 0
        fpr = fp / (fp + tn) if (fp + tn) > 0 else 0
        
        metrics_per_group[group_name] = {
            'TPR': tpr,
            'FPR': fpr,
            'n': len(group_df),
        }
    
    tprs = [m['TPR'] for m in metrics_per_group.values()]
    fprs = [m['FPR'] for m in metrics_per_group.values()]
    
    return {
        'metrics_per_group': metrics_per_group,
        'tpr_difference': max(tprs) - min(tprs) if tprs else 0,
        'fpr_difference': max(fprs) - min(fprs) if fprs else 0,
        'meets_threshold_5pct': (
            (max(tprs) - min(tprs) < 0.05 if tprs else True) and
            (max(fprs) - min(fprs) < 0.05 if fprs else True)
        ),
    }

(continuá con calibration.py similar)

Paso 3: implementar detection

src/detection/slicing.py:

import pandas as pd
from sklearn.metrics import accuracy_score, precision_score, recall_score

def slice_by_subgroup(predictions, ground_truth, groups, attribute_name='',
                      min_samples_per_group=30):
    """
    Slicing por subgrupo demográfico.
    """
    df = pd.DataFrame({
        'pred': predictions,
        'true': ground_truth,
        'group': groups,
    })
    
    results = {}
    for group_name, group_df in df.groupby('group'):
        if len(group_df) < min_samples_per_group:
            continue
        
        results[group_name] = {
            'n': len(group_df),
            'accuracy': accuracy_score(group_df['true'], group_df['pred']),
            'precision': precision_score(
                group_df['true'], group_df['pred'], zero_division=0
            ),
            'recall': recall_score(
                group_df['true'], group_df['pred'], zero_division=0
            ),
            'positive_rate': (group_df['pred'] == 1).mean(),
        }
    
    accuracies = [r['accuracy'] for r in results.values()]
    
    return {
        'attribute': attribute_name,
        'metrics_per_group': results,
        'accuracy_difference': max(accuracies) - min(accuracies) if accuracies else 0,
    }

src/detection/counterfactual.py:

def counterfactual_test(model, inputs, swap_function, threshold=0.05):
    """
    Test counterfactual genérico. swap_function debe definir cómo transformar inputs.
    """
    results = []
    for input_original in inputs:
        score_original = model(input_original)
        input_swapped = swap_function(input_original)
        score_swapped = model(input_swapped)
        
        delta = abs(score_original - score_swapped)
        
        results.append({
            'original': str(input_original)[:80],
            'swapped': str(input_swapped)[:80],
            'score_original': score_original,
            'score_swapped': score_swapped,
            'delta': delta,
            'fair': delta < threshold,
        })
    
    df = pd.DataFrame(results)
    
    return {
        'mean_delta': df['delta'].mean(),
        'max_delta': df['delta'].max(),
        'pct_fair': df['fair'].mean(),
        'examples_unfair': df[~df['fair']].head(5).to_dict('records'),
    }

Paso 4: implementar mitigation

src/mitigation/reweighting.py:

import pandas as pd

def compute_sample_weights(df, group_column):
    """
    Compute weights inversamente proporcional a frecuencia del grupo.
    """
    group_counts = df[group_column].value_counts()
    total = len(df)
    n_groups = len(group_counts)
    
    weights = df[group_column].apply(
        lambda g: total / (n_groups * group_counts[g])
    )
    
    return weights.values

src/mitigation/threshold_tuning.py:

def find_thresholds_for_demographic_parity(scores, groups, target_rate=0.5):
    """
    Encontrar thresholds per group para satisfacer DP.
    """
    df = pd.DataFrame({'score': scores, 'group': groups})
    
    thresholds = {}
    for group_name, group_df in df.groupby('group'):
        sorted_scores = sorted(group_df['score'].values, reverse=True)
        idx = int(target_rate * len(sorted_scores))
        thresholds[group_name] = sorted_scores[idx] if idx < len(sorted_scores) else 0
    
    return thresholds


def apply_per_group_threshold(scores, groups, thresholds, default=0.5):
    """Aplicar thresholds per group."""
    return [
        1 if s > thresholds.get(g, default) else 0
        for s, g in zip(scores, groups)
    ]

Paso 5: orchestrador

src/audit.py:

import pandas as pd
from .metrics.demographic_parity import demographic_parity
from .metrics.equalized_odds import equalized_odds
from .detection.slicing import slice_by_subgroup

def audit_model(predictions, ground_truth, groups_dict, output_path=None):
    """
    Audit completo de un modelo.
    
    Args:
        predictions: array de predictions del modelo.
        ground_truth: labels reales.
        groups_dict: dict {attribute_name: array of group labels}.
        output_path: path para guardar reporte (opcional).
    
    Returns:
        dict con todos los resultados.
    """
    report = {'attributes': {}}
    
    for attr_name, groups in groups_dict.items():
        attr_report = {
            'demographic_parity': demographic_parity(predictions, groups),
            'equalized_odds': equalized_odds(predictions, ground_truth, groups),
            'slicing': slice_by_subgroup(predictions, ground_truth, groups,
                                          attribute_name=attr_name),
        }
        report['attributes'][attr_name] = attr_report
    
    report['summary'] = generate_summary(report)
    
    if output_path:
        save_report(report, output_path)
    
    return report


def generate_summary(report):
    """Genera summary executivo."""
    issues = []
    for attr_name, attr_report in report['attributes'].items():
        dp = attr_report['demographic_parity']
        if not dp.get('meets_4_5_rule', True):
            issues.append(
                f"{attr_name}: DP ratio {dp['parity_ratio']:.3f} fails 4/5 rule"
            )
        
        eo = attr_report['equalized_odds']
        if not eo.get('meets_threshold_5pct', True):
            issues.append(
                f"{attr_name}: EO violation (TPR diff {eo['tpr_difference']:.3f}, "
                f"FPR diff {eo['fpr_difference']:.3f})"
            )
    
    return {
        'total_issues': len(issues),
        'issues': issues,
        'overall_pass': len(issues) == 0,
    }


def save_report(report, output_path):
    """Save report como JSON o markdown."""
    import json
    with open(output_path, 'w') as f:
        json.dump(report, f, indent=2, default=str)

Paso 6: CLI

src/cli.py:

import argparse
import pandas as pd
import joblib
from .audit import audit_model

def main():
    parser = argparse.ArgumentParser(description='Bias audit toolkit')
    parser.add_argument('--model', required=True, help='Path to pickled model')
    parser.add_argument('--data', required=True, help='Path to CSV data')
    parser.add_argument('--target', required=True, help='Target column')
    parser.add_argument('--protected', required=True, nargs='+',
                        help='Protected attribute columns')
    parser.add_argument('--output', default='audit_report.json',
                        help='Output path')
    
    args = parser.parse_args()
    
    # Load
    model = joblib.load(args.model)
    df = pd.read_csv(args.data)
    
    # Predictions
    features = df.drop([args.target] + args.protected, axis=1)
    predictions = model.predict(features)
    
    # Audit
    groups_dict = {attr: df[attr].values for attr in args.protected}
    report = audit_model(
        predictions=predictions,
        ground_truth=df[args.target].values,
        groups_dict=groups_dict,
        output_path=args.output,
    )
    
    print(f"Audit complete. {report['summary']['total_issues']} issues found.")
    print(f"Report saved to {args.output}")
    
    if not report['summary']['overall_pass']:
        for issue in report['summary']['issues']:
            print(f"  - {issue}")


if __name__ == '__main__':
    main()

Uso desde command line:

python -m src.cli \
    --model models/loan_model.pkl \
    --data data/test.csv \
    --target approved \
    --protected gender race \
    --output audit_report.json

Paso 7: tests

tests/test_demographic_parity.py:

from src.metrics.demographic_parity import demographic_parity

def test_perfect_parity():
    """Same rate per group → ratio = 1.0."""
    predictions = [1, 0, 1, 0, 1, 0, 1, 0]
    groups = ['A', 'A', 'A', 'A', 'B', 'B', 'B', 'B']
    
    result = demographic_parity(predictions, groups)
    
    assert result['parity_ratio'] == 1.0
    assert result['meets_4_5_rule']

def test_disparate_impact():
    """Different rates → ratio < 1.0."""
    predictions = [1, 1, 1, 0] + [0, 0, 0, 1]  # A: 75%, B: 25%
    groups = ['A']*4 + ['B']*4
    
    result = demographic_parity(predictions, groups)
    
    assert result['rates_by_group']['A'] == 0.75
    assert result['rates_by_group']['B'] == 0.25
    assert result['parity_ratio'] == 0.25 / 0.75  # ~0.333
    assert not result['meets_4_5_rule']

def test_single_group():
    """Edge case: only one group."""
    predictions = [1, 0, 1, 0]
    groups = ['A', 'A', 'A', 'A']
    
    result = demographic_parity(predictions, groups)
    
    assert 'error' in result

Run:

pytest tests/ -v

Paso 8: README

README.md:

# Bias Audit Toolkit

Toolkit Python para auditar bias en sistemas AI.

## Quick start

```bash
pip install -e .

python -m src.cli \
    --model your_model.pkl \
    --data test_data.csv \
    --target outcome \
    --protected gender race

Componentes

Metrics

  • demographic_parity: 4/5 rule compliance check.
  • equalized_odds: TPR/FPR equality.
  • calibration: score-to-probability mapping.

Detection

  • slicing: performance per subgroup.
  • counterfactual: causal bias testing.

Mitigation

  • reweighting: pre-processing weights.
  • threshold_tuning: post-processing per-group thresholds.

Audit process

  1. Train your model on training data.
  2. Generate predictions on test set.
  3. Run audit_model() with predictions + ground truth + protected attributes.
  4. Review report. Apply mitigations as needed.
  5. Re-audit. Iterate until acceptable.

Integration with CI

Add to .github/workflows/ci.yml:

- name: Bias audit gate
  run: |
    python -m src.cli \
        --model artifacts/model.pkl \
        --data data/holdout.csv \
        --target outcome \
        --protected gender \
        --output reports/audit.json
    python scripts/check_audit_gate.py reports/audit.json

check_audit_gate.py exits 1 if any issues, blocking deploy.


---

## Cómo aplicar el toolkit a un sistema real

### Paso 1: identificar atributos protegidos

Vuelve al Ethics Impact Analysis (M1/08). Los stakeholders identificados allí son los grupos a evaluar.

Ejemplos:
- Hiring system: gender, race, age, disability.
- Loan approval: gender, race, age, ZIP-inferred location.
- Healthcare: gender, race, age, insurance status.

### Paso 2: prepar test set con atributos

Idealmente, tu test set incluye los protected attributes annotated. Si no:

- Inferir cuando legalmente permisible (ZIP → race con error rates).
- Sample demographic study sobre subset.
- Counterfactual testing si imposible obtener.

### Paso 3: correr audit

```python
from src.audit import audit_model

predictions = model.predict(test_features)

groups_dict = {
    'gender': test_set['gender'].values,
    'race': test_set['race'].values,
    'age_bracket': test_set['age_bracket'].values,
}

report = audit_model(
    predictions=predictions,
    ground_truth=test_set['target'].values,
    groups_dict=groups_dict,
    output_path='audit_initial.json',
)

Paso 4: interpretar resultados

Si report['summary']['overall_pass'] es True: deploy with monitoring.

Si no:

  1. Identificar issues principales.
  2. Aplicar mitigations apropiadas.
  3. Re-train modelo.
  4. Re-audit.
  5. Iterar hasta acceptable.

Paso 5: documentar trade-offs

## Bias Audit Report

**Initial state**:
- Demographic parity (gender): 0.65 (fails 4/5 rule)
- Equalized odds (gender): TPR diff 12%, FPR diff 8%

**Mitigations applied**:
1. Re-weighted training data by gender.
2. Calibration adjustment per group.

**Final state**:
- Demographic parity (gender): 0.85 ✅
- Equalized odds (gender): TPR diff 4%, FPR diff 3% ✅
- Accuracy: 91% (was 93% pre-mitigation, 2% trade-off)

**Decision**: deploy with monitoring. Trade-off acceptable per stakeholder review.

Cierre del módulo

8 cápsulas

  1. Introducción al módulo.
  2. Demographic parity.
  3. Equalized odds y calibration.
  4. Bias detection: slicing, A/B, counterfactual.
  5. Impossibility theorems.
  6. Mitigation pre-processing.
  7. Mitigation in/post-processing.
  8. Mini-proyecto: Bias Audit Toolkit (esta cápsula).

Lo que tenés ahora

  • Vocabulario técnico: 4 fairness metrics, 3 categorías de mitigation.
  • Implementaciones desde cero de cada concepto.
  • Toolkit reusable que aplicás a cualquier sistema futuro.
  • Comprensión de trade-offs: impossibility theorems, fairness vs accuracy.
  • Integración CI: bias testing como gate de deploy.

Lo que cambió en vos

Antes:

  • "Bias es importante."

Después:

  • "Bias es medible con [4 métricas]. Mi sistema actualmente tiene [X], threshold legal es [Y], aplico [Z mitigation], valido con CI gate, monitoreo en producción."

Concrección. Métricas. Decisiones informadas con paper trail.

Empezamos en el siguiente módulo

Módulo 3: Privacy and Data Protection Fundamentals. La segunda dimensión de impact en personas: protección de datos.

Bias afecta a grupos. Privacy afecta a individuos. La transición es: "Sabés medir si tu sistema discrimina → ahora aprende a proteger los datos de las personas que tu sistema procesa."

Topics: PII, anonymization, k-anonymity, differential privacy, data minimization, retention policies. Compliance fundaments para GDPR (M4) y CCPA/etc (M5-6).

Recursos

  1. AIF360 (IBM) — full toolkit — comprehensive open source.
  2. Fairlearn (Microsoft) — alternative.
  3. Aequitas — Carnegie Mellon toolkit.
  4. Fairness Indicators (Google) — TF integration.
  5. Algorithmic Justice League — research y advocacy.

Cápsula 08 de 08 — Módulo 2 — AI Ethics & Compliance Guide

Fin del módulo 2. Continúa con el módulo 3 (Privacy and Data Protection Fundamentals).

← Anterior7. Mitigation: In-processing y Post-processingSiguiente →1. Introducción al módulo: Privacy and Data Protection Fundamentals