2. Fairness Metric: Demographic Parity

Capsule description

Demographic parity is the most common fairness metric and the first one you'll learn. Also called statistical parity or independence, it measures whether your model's favorable output is independent of the protected group.

Concretely: if your model approves a loan for 60% of men and 30% of women, there is no demographic parity. The probability of "approve" depends on gender.

This capsule covers:

1. The mathematical formula and its intuition.
2. How to compute it in code (numpy + pandas, no external tooling).
3. Disparate Impact Ratio — the legal/regulatory version of the metric (4/5 rule).
4. When demographic parity is the right metric and when it isn't.
5. Known limitations: why demographic parity alone doesn't guarantee fairness in many contexts.

By the end, you'll have a demographic_parity(predictions, groups) function that returns a number, and you'll know when to apply it and when to reach for other metrics.

The formal definition

Demographic parity holds when:

P(Ŷ = 1 | A = a) = P(Ŷ = 1 | A = b)  ∀ a, b ∈ A

Where:

• Ŷ is the model's prediction (1 = favorable outcome).
• A is the protected attribute (gender, race, etc.).
• a, b are specific values of A.

In plain English: the probability that the model predicts a "favorable outcome" must be equal across all groups of the protected attribute.

A concrete example

A loan approval system:

• 1000 men apply → 600 approved → P(approve | M) = 0.60.
• 1000 women apply → 300 approved → P(approve | F) = 0.30.
• Demographic parity ratio: 0.30 / 0.60 = 0.50.

Perfect demographic parity = 1.0. Any value < 1 indicates disparity.

Defining the "favorable outcome"

Worth noting: the metric is defined relative to a "favorable outcome." For the applicant:

• Loan: "approved" is favorable.
• Hiring: "interview" or "hired" is favorable.
• Insurance: "low premium" is favorable.
• Healthcare triage: "appointment scheduled quickly" is favorable.

In some cases you have to invert the metric. If the prediction is "high risk" (a score for how likely a default is), a "high" value can be unfavorable. Make sure you define consistently which one is the "favorable outcome."

Implementation from scratch

No external tools, just numpy:

import numpy as np
import pandas as pd

def demographic_parity(predictions, groups, favorable_outcome=1):
    """
    Compute the demographic parity ratio.
    
    Args:
        predictions: array-like of model predictions (0 or 1).
        groups: array-like of group labels (e.g., 'M', 'F').
        favorable_outcome: value considered favorable (default 1).
    
    Returns:
        dict with rates per group and the parity ratio.
    """
    df = pd.DataFrame({
        'prediction': predictions,
        'group': groups,
    })
    
    # Favorable outcome rate per group
    rates = df.groupby('group')['prediction'].apply(
        lambda x: (x == favorable_outcome).mean()
    )
    
    # Demographic parity ratio: min/max
    # 1.0 = perfect parity; < 1 = disparity
    parity_ratio = rates.min() / rates.max()
    
    return {
        'rates_by_group': rates.to_dict(),
        'parity_ratio': parity_ratio,
        'meets_4_5_rule': parity_ratio >= 0.80,
    }

Usage

predictions = [1, 1, 0, 1, 0, 0, 0, 0, 1, 0]  # 10 predictions
groups      = ['M', 'M', 'F', 'M', 'F', 'F', 'M', 'F', 'M', 'F']

result = demographic_parity(predictions, groups)

print(result)
# {
#     'rates_by_group': {'F': 0.0, 'M': 0.8},
#     'parity_ratio': 0.0,
#     'meets_4_5_rule': False,
# }

P(positive | M) = 4/5 = 0.80. P(positive | F) = 0/5 = 0.0. Ratio = 0.0/0.80 = 0.0. A catastrophe.

A more robust version with error handling

def demographic_parity_robust(predictions, groups, favorable_outcome=1):
    """Robust version with validation."""
    if len(predictions) != len(groups):
        raise ValueError("predictions and groups must have same length")
    
    if len(predictions) < 30:
        raise ValueError("Need at least 30 samples for stable metric")
    
    df = pd.DataFrame({
        'prediction': predictions,
        'group': groups,
    })
    
    rates = df.groupby('group').agg(
        n=('prediction', 'count'),
        favorable_rate=('prediction', lambda x: (x == favorable_outcome).mean()),
    )
    
    # Check for a sufficient sample per group
    min_n = rates['n'].min()
    if min_n < 30:
        raise ValueError(
            f"Smallest group has {min_n} samples; need at least 30 per group"
        )
    
    parity_ratio = rates['favorable_rate'].min() / rates['favorable_rate'].max()
    
    return {
        'rates_by_group': rates.to_dict(),
        'parity_ratio': parity_ratio,
        'meets_4_5_rule': parity_ratio >= 0.80,
        'sample_sizes': rates['n'].to_dict(),
    }

With validation against insufficient sample size. Without enough data per group, the metric has high variance and isn't reliable.

Disparate Impact Ratio: the legal version

Disparate Impact Ratio (DIR) is practically the same thing as the demographic parity ratio, but within a specific US legal framework:

DIR = P(favorable | minority group) / P(favorable | majority group)

The 4/5 Rule (also the "80% Rule"): if DIR < 0.80, there's evidence of disparate impact in the legal sense (EEOC, US courts).

Historical origin: the EEOC (Equal Employment Opportunity Commission) established this rule in the Uniform Guidelines on Employee Selection Procedures (1978) for hiring. Since then, it has been generalized to other regulated decisions.

Implementation

def disparate_impact_ratio(predictions, groups, minority_label, majority_label,
                           favorable_outcome=1):
    """
    DIR per the 4/5 rule (EEOC).
    """
    df = pd.DataFrame({
        'prediction': predictions,
        'group': groups,
    })
    
    rate_minority = df[df['group'] == minority_label]['prediction'].apply(
        lambda x: x == favorable_outcome
    ).mean()
    
    rate_majority = df[df['group'] == majority_label]['prediction'].apply(
        lambda x: x == favorable_outcome
    ).mean()
    
    if rate_majority == 0:
        return None  # Indeterminate
    
    dir_value = rate_minority / rate_majority
    
    return {
        'rate_minority': rate_minority,
        'rate_majority': rate_majority,
        'disparate_impact_ratio': dir_value,
        'passes_4_5_rule': dir_value >= 0.80,
        'legal_concern': dir_value < 0.80,
    }

Use case: auditing a hiring system

import pandas as pd

# Hiring decisions: 1 = invited to interview
hiring_data = pd.DataFrame({
    'gender': ['M']*1000 + ['F']*1000,
    'invited': [1]*420 + [0]*580 + [1]*280 + [0]*720,
})

result = disparate_impact_ratio(
    predictions=hiring_data['invited'],
    groups=hiring_data['gender'],
    minority_label='F',
    majority_label='M',
)

print(result)
# {
#     'rate_minority': 0.28,        # 28% of women invited
#     'rate_majority': 0.42,        # 42% of men invited
#     'disparate_impact_ratio': 0.667,
#     'passes_4_5_rule': False,
#     'legal_concern': True,
# }

DIR = 0.667 < 0.80. A legal concern under the EEOC framework.

When demographic parity is the right metric

Demographic parity is appropriate when:

1. The distribution of outcomes should be independent of the group

By policy choice or by legal requirement. Examples:

• Affirmative action programs: explicitly targeting demographic parity.
• Resource allocation where the distribution should match demographics.
• Marketing exposure: we want each demographic group to see a similar % of relevant ads.

2. You have no reliable ground truth for "merit"

When you can't objectively measure who "deserves" the favorable outcome, demographic parity is a conservative default. Example: in advertising, who "deserves" to see an ad? There's no objective merit. An equal distribution is defensible.

3. Alignment with the regulatory legal framework

The US EEOC framework explicitly uses disparate impact (a variant of demographic parity) as a legal criterion. Compliance requires measuring this metric specifically.

4. High visibility / accountability requirement

When any public disparity would be a problem, demographic parity is the most conservative metric and the easiest to defend.

When demographic parity is NOT appropriate

Three scenarios where demographic parity can be a bad metric:

Scenario 1: there's ground truth for merit and the groups have different base rates

Example: if one group genuinely has a 30% default rate and another has 10%, demographic parity in loan approval would force you to give loans to default-prone applicants, generating harm to them (they lose money, it hurts their credit) and to the bank.

In this case, equalized odds (capsule 03) is more appropriate: equal error rates, not equal approval rates.

Scenario 2: applying it shifts the equilibrium in a harmful way

Imagine a dataset where one group is under-represented (10%). Forcing demographic parity can require approving marginal cases from the minority group, generating false positives that harm the approved individual (a loan that goes to default).

Demographic parity can be useful in the aggregate without being useful for the individual.

Scenario 3: the outcome isn't zero-sum

For some systems (e.g., advertising), one group seeing more ads doesn't mean another sees fewer. Demographic parity in that context may not make sense — the outcome isn't competitive.

Limitations: why demographic parity alone isn't enough

Even when demographic parity is the right metric for your context, on its own it doesn't guarantee fairness.

Limitation 1: it hides calibration issues

You can have perfect demographic parity (50% approval for group A and B) but:

• For group A: approved loans default at 5%.
• For group B: approved loans default at 25%.

Demographic parity = 1.0 (perfect). But group B is being "approved" into worse outcomes. That's a calibration violation (capsule 03).

Limitation 2: it hides accuracy disparities

Demographic parity can hold while the quality of the decision differs by group:

• Group A: 40% true positives + 10% false positives = a 50% approval rate.
• Group B: 10% true positives + 40% false positives = a 50% approval rate.

Demographic parity ✅. But group B is far worse served (more errors).

Limitation 3: a perverse incentive

If demographic parity is the only objective, you can satisfy it with random decisions:

• Approve 50% at random from each group. Demographic parity = 1.0.
• But the decision bears no relation to the applicant's merit.

That's satisfying the metric without serving the purpose.

Implication

Demographic parity is useful but not sufficient. Capsule 03 introduces equalized odds and calibration, which cover the gaps.

A real case: applying it to Apple Card

Apple Card issued significantly lower credit limits to women.

If Goldman Sachs had measured demographic parity (on a sample with inferable or joinable gender):

# Hypothetical post-hoc analysis
apple_card_data = pd.DataFrame({
    'gender': ['M']*500 + ['F']*500,  # balanced sample
    'high_credit_limit': [...],  # 1 if > median
})

result = demographic_parity(
    predictions=apple_card_data['high_credit_limit'],
    groups=apple_card_data['gender'],
)

# Hypothetical result:
# rate_M = 0.55 (55% of men get an above-median limit)
# rate_F = 0.25 (25% of women get an above-median limit)
# parity_ratio = 0.25 / 0.55 = 0.45
# meets_4_5_rule = False (0.45 < 0.80)

Three conclusions:

1. Goldman could have detected it with this simple metric, before launch.
2. It fails the 4/5 rule = a legal concern under EEOC-equivalent frameworks.
3. Independent of "fault": even if Goldman wasn't using gender as a feature, the output showed disparity. A metric on the output, not the inputs, is what detects proxy discrimination.

Traps and common mistakes

1. Comparing the wrong rate

It's sometimes tempting to compare conditional rates instead of marginal ones:

# ❌ Wrong: P(approve | qualified, group=A) vs P(approve | qualified, group=B)
# That's equalized odds, not demographic parity

Demographic parity compares marginal rates: P(approve | group), without conditioning on another variable.

2. Insufficient sample sizes

If you have 5 samples from the minority group, demographic parity is extremely variable. Use at least 30 per group (a basic rule of statistical inference).

3. Defining the "favorable outcome" inconsistently

Sometimes "1" is favorable, sometimes it's unfavorable. Decide explicitly and document it.

4. Reporting only the ratio without the rates

parity_ratio = 0.50 gives less information than rate_A = 0.30, rate_B = 0.60, ratio = 0.50. The rates themselves are critical to understanding the magnitude.

5. Treating 0.79 as "almost 0.80"

The 4/5 rule is a legal threshold. 0.79 fails. It doesn't "almost pass." Apply the threshold strictly.

Self-check

P(Ŷ = 1 | A = a) = P(Ŷ = 1 | A = b)  ∀ a, b

parity_ratio = P(Ŷ = 1 | A = minority) / P(Ŷ = 1 | A = majority)

# tests/test_fairness.py

def test_demographic_parity_gate():
    """Block the deploy if demographic parity fails the 4/5 rule."""
    # Take the model's predictions on the test set
    predictions = model.predict(test_set)
    
    # Check for each protected attribute
    for attribute in ['gender', 'race', 'age_bracket']:
        result = demographic_parity(
            predictions=predictions,
            groups=test_set[attribute],
        )
        
        assert result['meets_4_5_rule'], (
            f"Demographic parity violation for {attribute}: "
            f"ratio={result['parity_ratio']:.3f} (threshold 0.80). "
            f"Rates: {result['rates_by_group']}"
        )

# .github/workflows/ci.yml
- run: pytest tests/test_fairness.py

Summary and next step

• Demographic parity measures whether the favorable output is independent of the protected group.
• Formula: P(Ŷ=1 | A=a) = P(Ŷ=1 | A=b) ∀ a, b.
• Typical legal threshold: the 4/5 rule (DIR ≥ 0.80).
• Implementable in 10 lines of Python with pandas.
• Appropriate when: there's no ground truth for merit, policy requires parity, or the legal framework applies.
• Not sufficient on its own: it hides calibration and accuracy issues. Combine it with other metrics.

Checkpoint: you should be able to implement demographic parity from scratch and apply it to a dataset.

Bridge to the next capsule: capsule 03 introduces equalized odds and calibration — the metrics that cover demographic parity's gaps. Equalized odds measures equality of error rates (false positives, false negatives) across groups. Calibration measures whether "score = 0.7" means the same thing across groups. Both are crucial for systems with reliable ground truth (loans, hiring with verifiable performance, healthcare diagnoses).

Resources

1. 4/5 Rule — EEOC Uniform Guidelines — the US legal basis.
2. Fairness in ML textbook — Chapter 3 — the formal definition.
3. AIF360 — DemographicParity metric — an implementation.
4. Fairlearn — DemographicParity — an alternative.

Next: 03-fairness-metrics-equalized-odds.md — Equalized odds and calibration.

Capsule 02 of 08 — Module 2 — AI Ethics & Compliance Guide