Module 3: Privacy and Data Protection Fundamentals

3. Anonymization vs Pseudonymization

Next →

Capsule description

When data minimization isn't enough (you do need some data), the next layer is to transform the data to reduce privacy risk:

  • Pseudonymization: reversible. You replace identifiers with codes; you can reverse it with a key.
  • Anonymization: irreversible. Once applied, you cannot identify the original individual.

This capsule covers:

  1. Formal definitions and the technical differences.
  2. When each one applies.
  3. Re-identification attacks — why perfect anonymization is hard.
  4. K-anonymity, L-diversity, T-closeness — progressively more robust techniques.
  5. Differential Privacy — the gold standard.
  6. Application to AI: when to anonymize training data, and the trade-offs.

By the end, you'll be able to choose the right technique for your data and understand its limitations.

Pseudonymization

Definition

Replace personal identifiers with pseudonyms (opaque codes). You keep a mapping table that lets you reverse the process if you have the key.

Real:           Pseudonymized:
maria@gmail.com → user_42af8b2
2024-03-15      → 2024-03-15  (unchanged)
NYC             → NYC          (unchanged)

Only the email was replaced. The rest of the data remains.

A simple implementation

import hashlib

def pseudonymize_email(email, secret_key):
    """
    Pseudonymize an email while keeping a deterministic mapping.
    Same email → always the same pseudonym (joinable).
    """
    combined = f"{email}{secret_key}"
    hash_obj = hashlib.sha256(combined.encode())
    return f"user_{hash_obj.hexdigest()[:16]}"

# Usage
secret_key = load_secret_from_env()  # never commit!
pseudonym = pseudonymize_email("maria@gmail.com", secret_key)
# pseudonym = "user_42af8b2c..."

Characteristics

Pros:

  • Reversible: with the key, you can map pseudonym → real.
  • You keep joinability: same person → same pseudonym, so you can join datasets.
  • It reduces surface area: the exposed data contains no direct PII.

Cons:

  • It is NOT anonymization: if the key is compromised, everything is reversed.
  • Under GDPR: pseudonymized data is still personal data and the obligations apply.
  • Re-identification: even without the key, other features can identify individuals.

When to use pseudonymization

Appropriate when:

  1. You need joinability across datasets but not direct PII.
  2. Operations require occasional reversal (auditing, customer support).
  3. As a layer, combined with other techniques.

NOT sufficient when regulation specifically requires "anonymized" data.

Anonymization

Definition

Transform the data such that the original individual cannot be re-identified, not even with auxiliary information.

GDPR Recital 26: anonymized data is NOT personal data — outside GDPR's scope.

Why perfect anonymization is hard

Re-identification attacks:

  1. Linkage attacks: combining "anonymized" datasets with auxiliary data reveals individuals.

    A real case: the Netflix Prize Dataset (2006). Anonymized movie ratings from 480K users. Researchers linked them to public IMDB ratings → they re-identified users with 80%+ accuracy.

  2. Quasi-identifiers: combinations of features that individually aren't PII but together identify someone.

    A real case: Sweeney (2000) showed that {date of birth, ZIP code, gender} uniquely identifies 87% of the US population. None of them is PII on its own.

  3. Background knowledge attacks: an attacker with partial info (knows María lives in NYC, works in tech) can triangulate in small datasets.

Implementation: simply deleting identifiers isn't enough

# ❌ Insufficient
def fake_anonymize(record):
    """Drop email and name. Is that enough?"""
    record.pop('email', None)
    record.pop('name', None)
    return record

# Result: you still have ZIP, age, gender, occupation, etc.
# The combinations identify individuals.

Serious anonymization requires:

  1. Dropping direct identifiers (name, email, SSN).
  2. Generalizing quasi-identifiers (age → bracket, ZIP → first 3 digits).
  3. Suppressing outliers that would be unique.
  4. Adding noise or aggregating to limit precision.

K-Anonymity

The concept

A dataset is k-anonymous if every record is indistinguishable from at least k-1 other records in terms of quasi-identifiers.

NOT k-anonymous dataset (k=1, identifiable):
| Age | ZIP   | Occupation     |
|-----|-------|----------------|
| 23  | 10001 | Software eng   |  ← unique
| 45  | 10003 | Doctor         |  ← unique
| 31  | 10002 | Teacher        |  ← unique

3-anonymous dataset (each row matches at least 2 others):
| Age   | ZIP     | Occupation |
|-------|---------|------------|
| 20-30 | 100**   | Tech       |
| 20-30 | 100**   | Tech       |
| 20-30 | 100**   | Tech       |
| 30-40 | 100**   | Education  |
| 30-40 | 100**   | Education  |
| 30-40 | 100**   | Education  |

K=3 means every record matches at least 2 others on quasi-identifiers. Re-identification requires distinguishing among 3 indistinguishable individuals.

Implementation with generalization

import pandas as pd

def k_anonymize(df, quasi_identifiers, k=3):
    """
    Simplified k-anonymity with generalization.
    """
    # Generalize age to bracket
    if 'age' in quasi_identifiers:
        df['age'] = pd.cut(df['age'], 
                            bins=[0, 20, 30, 40, 50, 60, 100],
                            labels=['<20', '20-29', '30-39', '40-49', '50-59', '60+'])
    
    # Generalize ZIP to first 3 digits
    if 'zip' in quasi_identifiers:
        df['zip'] = df['zip'].astype(str).str[:3] + '**'
    
    # Find equivalence classes
    grouped = df.groupby(quasi_identifiers).size().reset_index(name='count')
    
    # Suppress records in classes < k
    valid_classes = grouped[grouped['count'] >= k][quasi_identifiers]
    
    df_anonymized = df.merge(valid_classes, on=quasi_identifiers, how='inner')
    
    return df_anonymized

K-anonymity's limitations

K-anonymity protects against identity disclosure but not against attribute disclosure:

Equivalence class (3 individuals all in 30-39, NYC):
| Age   | ZIP    | Diagnosis      |
|-------|--------|----------------|
| 30-39 | 100**  | HIV positive   |
| 30-39 | 100**  | HIV positive   |
| 30-39 | 100**  | HIV positive   |

K=3, but they all have the same diagnosis. If you know that a specific person is in this equivalence class, you know their diagnosis.

L-diversity and T-closeness address this gap.

L-Diversity

The concept

A dataset is l-diverse if every equivalence class has at least l distinct values in its sensitive attributes.

3-diverse on "Diagnosis":
| Age   | ZIP    | Diagnosis      |
|-------|--------|----------------|
| 30-39 | 100**  | HIV positive   |
| 30-39 | 100**  | Diabetes       |
| 30-39 | 100**  | Healthy        |

Even if you know someone is in the class, you don't know which of the 3 diagnoses they have.

L-diversity protects against basic attribute disclosure.

Limitations

L-diversity can still leak information:

  • If the attacker knows the target does NOT have diabetes, there's a 50% chance of each of the other two.
  • If the values are skewed (90% one value), there's little protection.

T-Closeness

The concept

A dataset is t-close if the distribution of values in each equivalence class is close to the global distribution (within a threshold t).

It guarantees that knowing someone is in class X tells you no more about their sensitive attribute than knowing the general population does.

The implementation is mathematically complex (Earth Mover's Distance). In practice, libraries like ARX (open source) implement it.

Differential Privacy

The gold standard

Differential Privacy (Dwork et al., 2006) provides a mathematical guarantee about privacy:

Adding or removing a single individual from the dataset changes the analysis output by only a small bounded amount.

Meaning: the output does NOT depend significantly on any single individual. Therefore, information about any individual cannot be inferred.

The mechanism: adding noise

DP works by adding calibrated noise to queries or outputs:

import numpy as np

def differentially_private_count(data, epsilon=1.0):
    """
    A count with a DP guarantee.
    epsilon: privacy budget (lower = more private, less accurate).
    """
    true_count = len(data)
    # Add Laplace noise calibrated to sensitivity / epsilon
    noise = np.random.laplace(0, 1/epsilon)
    return true_count + noise

The privacy budget (epsilon)

  • ε = 0: perfect privacy (the output doesn't depend on the data).
  • ε = ∞: no privacy (the output is exactly determined by the data).
  • Practical range: 0.1 - 10.

Lower ε = better privacy + more noise + less accuracy.

DP in ML training

DP-SGD (Abadi et al., 2016): a variant of stochastic gradient descent that adds noise to the gradients during training. The resulting model is differentially private.

# Pseudo-code
def dp_sgd_step(model, batch, epsilon, delta):
    # Compute gradients per-sample
    per_sample_grads = compute_per_sample_grads(model, batch)
    
    # Clip gradients to limit sensitivity
    clipped_grads = clip(per_sample_grads, max_norm=1.0)
    
    # Average + add noise
    avg_grad = mean(clipped_grads)
    noise = sample_gaussian_noise(scale=clipped_norm/epsilon)
    
    private_grad = avg_grad + noise
    
    # Update model
    model.params -= learning_rate * private_grad

DP's trade-offs

Pros:

  • A mathematical guarantee — it isn't a heuristic.
  • Composable: combining queries each with budget ε₁, ε₂ gives a total privacy of ε₁ + ε₂.
  • Future-proof: it protects against attacks that haven't been discovered yet.

Cons:

  • Accuracy cost: the noise reduces utility. Significantly.
  • Privacy budget management: every query spends budget. Eventually it runs out.
  • Complex implementation: bugs in DP implementations have been published frequently.

When to use DP

  • Sensitive medical or census data.
  • Systems that release public statistics (the US Census 2020 used DP).
  • Federated learning with strong privacy.

For most AI engineering applications, DP is overkill — pseudonymization + k-anonymity + access controls are sufficient. DP is for cases where a mathematical guarantee is required.

Application to AI training data

Scenario: training a classifier

Your data contains PII. You need the training data, and you want to minimize privacy risk.

Options:

Option 1: drop direct identifiers (basic)

training_data.drop(columns=['email', 'name', 'phone', 'address'], inplace=True)

Not enough on its own. The quasi-identifiers remain.

Option 2: pseudonymize identifiers

training_data['email_id'] = pseudonymize(training_data['email'])
training_data.drop(columns=['email', 'name'], inplace=True)

Better, but pseudonymized → still personal data.

Option 3: k-anonymize

quasi_ids = ['age', 'zip', 'gender']
training_data = k_anonymize(training_data, quasi_ids, k=5)

Protects against identity disclosure. Sufficient for most cases.

Option 4: DP-SGD

model = train_with_dp_sgd(data, epsilon=1.0)

A mathematical guarantee, but with an accuracy cost.

The decision matrix

Risk levelRecommendation
Low (public anyway)Drop direct identifiers
Medium (proprietary internal)Pseudonymize + access controls
High (regulated, PII-heavy)K-anonymize + pseudonymize
Critical (medical, legal)DP-SGD or federated learning

Common traps

1. "We removed the names, it's anonymized"

False. Quasi-identifiers (age + ZIP + gender) frequently identify people uniquely.

2. Trusting hashing as anonymization

hash(email) is deterministic — the same email always gives the same hash. Re-identification is trivial with a rainbow table or brute force.

For real anonymization, you need added randomness or k-anonymity.

3. Pseudonymization without securing the key

If the pseudonymization key is compromised, everything gets reversed. Treat the key with the same care as a master password.

4. Anonymization once-and-done

Datasets are dynamic. Anonymization that was sufficient last year can be insufficient today with new auxiliary data available.

5. Forgetting temporal quasi-identifiers

Timestamp + location → highly identifying. "User accessed at 14:32 from NYC" can be unique on a daily basis.

Self-check

1. What's the key difference between anonymization and pseudonymization?

Reversibility:

  • Pseudonymization: reversible with a key. The same individual → the same pseudonym (joinable). But with the key, you can recover the identity.

  • Anonymization: irreversible. There's no key. There's no mathematical way to recover the identity (ideally).

Legal implication (GDPR):

  • Pseudonymized data is personal data — all the obligations apply (Art 6, etc.).
  • Anonymized data is not personal data — out of GDPR's scope.

Practical implication:

  • Pseudonymization is easier to implement and useful when you need occasional reversal (auditing).
  • Anonymization is harder to genuinely achieve because of re-identification attacks.

For many AI use cases, pseudonymization + access controls + minimization is the right combination.

2. Why isn't k-anonymity enough on its own?

K-anonymity protects against identity disclosure (knowing WHICH individual) but NOT against attribute disclosure (knowing WHAT sensitive attributes they have).

Example case: a 3-anonymous group where everyone has the same diagnosis. K=3 is satisfied. But if you know the target is in the group, you know their diagnosis.

L-diversity addresses this: it requires equivalence classes with at least l distinct sensitive values.

T-closeness goes further: it requires the distribution within the class to be similar to the global distribution.

In practice:

  • K-anonymity is basic privacy — the minimum bar.
  • L-diversity is standard privacy — sufficient for many uses.
  • T-closeness is strong privacy — for highly sensitive contexts.
  • Differential privacy is a mathematical guarantee — the strongest, but accuracy-costly.
3. When is differential privacy worth the accuracy cost?

When:

  1. A mathematical guarantee is required: for regulated industries (medical research) or public data releases (census).

  2. Being future-proof matters: you want protection against attacks that haven't been discovered yet.

  3. Composability: you're going to run multiple queries and need privacy budget tracking.

  4. High-risk PII: medical records, legal data, sensitive financial data.

NOT worth it for:

  • Standard ML applications with normal PII.
  • When pseudonymization + k-anonymity are sufficient.
  • When accuracy is critical (e.g., medical diagnosis where false negatives kill).

A common practice: use DP for published statistics, and pseudonymization + minimization for internal ML training. Best of both worlds.

4. How do you defend against re-identification attacks?

Multiple defenses:

  1. Drop direct identifiers: name, email, SSN, etc.

  2. Generalize quasi-identifiers: age bracket, ZIP first 3 digits, etc.

  3. Suppress outliers: records that would be unique get removed or generalized further.

  4. K-anonymity (k≥5): each record matches at least 4 others.

  5. L-diversity (l≥3): equivalence classes have diverse sensitive values.

  6. Limit auxiliary data: minimize the context attackers have.

  7. Access controls: even anonymized data has restricted access.

  8. Re-evaluate periodically: with new data sources, what was anonymous becomes identifiable.

  9. Differential privacy for the highest stakes.

Defense in depth — multiple layers, no single technique. Assume the individual layers will fail; the combination is what provides robustness.

Summary and next step

  • Pseudonymization: reversible. Good for joinability + reduced surface area. Still personal data under GDPR.
  • Anonymization: irreversible. Hard to genuinely achieve because of re-identification attacks.
  • K-anonymity: each record is indistinguishable from k-1 others. Basic privacy.
  • L-diversity, T-closeness: incremental improvements addressing attribute disclosure.
  • Differential Privacy: a mathematical guarantee, the gold standard, with a significant accuracy cost.
  • Defense in depth: combine techniques + access controls + minimization.

Checkpoint: you should be able to choose the right technique based on your data's sensitivity and use case.

Bridge to the next capsule: capsule 04 covers consent in AI systems: informed consent, granularity, revocation, and the specific challenges of getting meaningful consent when users don't understand how AI processes their data.

Resources

  1. k-Anonymity (Sweeney, 2002) — the foundational paper.
  2. Differential Privacy (Dwork et al., 2006) — the foundational paper.
  3. ARX Anonymization Tool — an open source implementation.
  4. DP-SGD (Abadi et al., 2016) — DP for ML training.
  5. Netflix Prize re-identification — the case study.

Next: 04-consent-in-ai.md — Meaningful consent for AI systems.

Capsule 03 of 08 — Module 3 — AI Ethics & Compliance Guide

← Previous2. Data Minimization in AI SystemsNext →4. Consent in AI Systems