Module 3: Privacy and Data Protection Fundamentals

2. Data Minimization in AI Systems

Capsule description

Data minimization is the first practical privacy principle. The idea is simple, the application is revolutionary:

Collect and process only the data strictly necessary for the specific declared purpose. Nothing more.

In traditional software, "more data is better" is the default. In AI specifically, there's a bias toward "let's save everything, we might need it later." This principle inverts the logic: you have to justify every piece of data you collect, not assume it has value.

This capsule covers:

The necessity test: how to distinguish "necessary" from "nice-to-have."
Application across AI lifecycle stages: training, fine-tuning, inference, logging.
Reducing granularity: when aggregations and buckets are enough vs. raw data.
A real case: how to apply minimization to a system with LLMs and RAG.
Honest trade-offs: when minimization reduces model utility.

By the end, you'll be able to review your current system and significantly reduce its privacy risk surface area, without losing capability.

The necessity test

For every piece of data you collect or process, three questions:

1. Is it necessary for the declared purpose?

"Necessary" isn't "useful." Necessary means you cannot fulfill the purpose without this data.

Example:

An AI hiring system:
- Necessary: skills, experience, education.
- Nice-to-have: hobbies, religion, marital status.
- Unnecessary: race (except for internal bias auditing, not for decisions).

2. Could we achieve it with less data, or aggregated data?

Sometimes you don't need raw data, just statistics or categories:

Instead of an exact date of birth → an age bracket.
Instead of an address → a postal code.
Instead of an exact income → an income bracket.

3. What's the minimum threshold of useful data?

Sometimes it's a mind shift:

"How many training samples do we REALLY need?"
"How many features?"
"How much retention?"

If you can't justify the number, it's probably over-collection.

Application across the AI lifecycle

Data minimization applies at multiple stages:

Training data

The question: which samples and which features are truly necessary for the model?

Application:

Samples: do you need 1M training samples, or will 100K do? Small models rarely need more than 100K well-curated ones.
Features: drop features that don't contribute signal. Use feature importance analysis.
Time range: do you need 10 years of history, or are 2 years enough?

A dual benefit: less data = less privacy exposure + lower compute cost.

Inference / API requests

The question: which inputs do we actually need to answer the query?

Application:

# Before (over-collection)
@app.post("/recommend")
async def recommend(user_id: str):
    user = await fetch_user_full_profile(user_id)  # ❌ everything: name, email, phone, address, history
    recommendations = model.predict(user)
    return recommendations

# After (minimization)
@app.post("/recommend")
async def recommend(user_id: str):
    # Only the features the model actually uses
    user_features = await fetch_user_features(user_id, fields=['interests', 'past_views'])
    recommendations = model.predict(user_features)
    return recommendations

If the model only uses interests and past_views, don't fetch name/email/phone.

Logging

The question: what information in the logs is necessary for debugging vs. nice-to-have?

Application:

# ❌ Over-logging
logger.info(f"User {user.full_name} ({user.email}) requested recommendation for {user.full_address}")

# ✅ Minimal logging
logger.info(f"User {anonymize(user.id)} requested recommendation, features: {feature_names}")

Logs are persistent — frequently more persistent than your database. Log minimization is critical.

Conversation history (LLMs)

The question: do you need to keep the full conversation history?

Application:

Active session: full history for coherence during the session.
Persisted long-term: maybe a summary, not the full history.
Used for fine-tuning: anonymize the PII first, or don't use it.

Conversations frequently contain PII. Persisting them without necessity is a classic minimization violation.

Reducing granularity

Many privacy issues come from unnecessary granularity. Reducing granularity reduces risk.

Table: high granularity → low granularity

High	Low	Appropriate when
Birth date (1985-03-14)	Age bracket (30-40)	Recommendation, not medical
GPS coordinates	Postal code	Location features for recommendation
Exact salary ($87,500)	Income bracket ($75K-100K)	Risk scoring
Full IP address	Country + ISP	Geo content delivery
Browsing URL	Domain	Behavioral features
Exact timestamp	Hour bucket	Temporal patterns

Every reduction in granularity:

Loses marginal predictive signal in many cases.
Dramatically reduces privacy risk.
Improves compliance with GDPR data minimization.

When NOT to reduce granularity

The system requires the exact value (medical dosing requires an exact age, not a bracket).
A legal requirement (some financial transactions require an exact timestamp).
The use case is precisely that granularity (scientific research).

In all of these cases, justify and document.

A real case: applying minimization to a RAG system

The system: a customer support RAG. It retrieves relevant docs and answers queries.

Before minimization

# System prompt (dangerous)
system_prompt = f"""
You are a customer support assistant.
Customer information:
- Name: {customer.full_name}
- Email: {customer.email}
- Phone: {customer.phone}
- Address: {customer.full_address}
- Account ID: {customer.account_id}
- Subscription: {customer.subscription}
- Last 10 tickets: {customer.recent_tickets}
- Last 100 conversations: {customer.recent_conversations}
- Payment methods: {customer.payment_methods}

Answer the customer's question.
"""

# Logging
logger.info(f"Query: {query}, prompt: {system_prompt}, response: {response}")

Problems:

Email, phone, and address in the prompt → potential leak in the response.
Sent to the OpenAI API → in their logs.
Logged locally → the email persists in the logs.
Payment methods → completely irrelevant for support, but exposed.

After minimization

# Minimal system prompt
system_prompt = f"""
You are a customer support assistant.
Customer context (minimal):
- Subscription tier: {customer.subscription_tier}
- Account age: {customer.account_age_in_months}
- Topic of last ticket: {customer.last_ticket_topic}

Answer the customer's question. Do not mention specific personal details
unless directly relevant to the question.
"""

# Anonymized logging
logger.info(
    f"Query type: {classify_query(query)}, "
    f"customer_tier: {customer.subscription_tier}, "
    f"response_length: {len(response)}"
)
# No raw query, no raw response, no email/phone.

Improvements:

No email/phone/address in the prompt.
Payment methods removed.
Logging anonymized.
The LLM isn't exposed to unnecessary PII.

Trade-offs accepted:

The LLM can't personalize the response using the customer's name.
The logs don't let you reproduce bugs exactly (mitigated with a different structured logging approach).
Some queries may require more back-and-forth.

Net result: drastically reduced privacy surface area, a modest reduction in personalization. A net win by an order of magnitude.

Real trade-offs

Trade-off 1: model accuracy vs. data volume

More data = a better model (in general). Minimization can reduce data.

When the trade-off is real: models that need billions of samples for foundation training.

When it is NOT real: for fine-tuning and small models, 100K curated > 1M dirty. Minimization actually improves quality.

Trade-off 2: removed features reduce signal

Dropping features can reduce accuracy.

When it's real: features that genuinely contribute unique signal.

When it is NOT real: features that correlate with others (multicollinearity). Removing them doesn't lose signal — only redundancy.

A concrete test: retrain without the feature, measure the accuracy delta. If the delta is < 1%, the feature was redundant.

Trade-off 3: reduced granularity loses precision

Buckets instead of raw values → less precision.

When it's real: systems that depend on exact precision (medical, legal).

When it is NOT real: behavioral/recommendation systems, where buckets are sufficient.

The decision framework

Identify a minimization candidate (data, feature, granularity).
Hypothesize the impact: how much do I lose if I remove/reduce it?
A/B test offline: compare the model with vs. without.
Decide based on the data: if the delta is small, minimize. If the delta is significant, justify and document.

Implementation: the data minimization checklist

## Data Minimization Checklist

For each data type collected/processed:

### Necessity
- [ ] Justified specific business purpose
- [ ] Cannot achieve purpose without this data
- [ ] Reviewed within last 6 months

### Granularity
- [ ] Minimum granularity acceptable for purpose
- [ ] Aggregations used where possible
- [ ] Buckets/categories instead of raw values where applicable

### Lifecycle
- [ ] Retention period defined and enforced
- [ ] Auto-purge mechanism in place
- [ ] Process for "right to deletion" requests

### Access
- [ ] Access limited to those who need it
- [ ] Logged when accessed
- [ ] Encrypted at rest and in transit

### Logging
- [ ] Logs do not contain unnecessary PII
- [ ] Logs are anonymized where possible
- [ ] Log retention defined

### LLM-specific (if applicable)
- [ ] PII not in system prompts
- [ ] PII not in user context unless necessary
- [ ] LLM API contract reviewed for data handling
- [ ] Conversations not used for training without consent

Apply this checklist before deploy and every 6 months.

Common traps

1. "Just in case"

"Let's save X in case we need it later" — the classic violation. If you can't justify a specific use now, don't collect it.

2. Logs with complete PII

Easy to do (print(user) logs the entire user object), hard to audit afterward. Implement structured logging from day 1.

3. Confusing "anonymized" with "minimized"

Anonymization is different. Minimization is about not collecting in the first place. Anonymization is second best (you collected it, then obfuscated it).

4. Forgetting caches and backups

Data in caches, in backups, in archives — it's all data too. The same rules apply. If you purge from the primary DB but the cache keeps it for 30 days, you didn't minimize.

5. PII in LLM prompts

Especially common. PMs ask to "personalize the response" → you end up passing the entire customer profile to the LLM. Always question it.

Self-check

1. How do you distinguish "necessary" from "nice-to-have" for data?

A concrete test: can you fulfill the specific declared purpose WITHOUT this data?

If the answer is yes → not necessary, remove it.

If the answer is no → it's necessary (or at least justified).

Edge case: "we could improve the result with this data but it isn't strictly required." That's nice-to-have. Under the data minimization principle: do NOT collect it unless the improvement is significant and stakeholders approve it with a full understanding of the privacy cost.

Documentation: for every data field collected, write 1-2 lines justifying its necessity. If you can't write it in a few lines, it probably isn't necessary.

2. Why is reducing granularity often better than removing features?

Removing a feature loses all the signal from that dimension.

Reducing granularity preserves the signal at the dimension level while reducing individual specificity.

Example: location.

Remove the location feature: the model knows nothing about geography. If geography matters for the prediction, accuracy drops significantly.
Reduce to country: the model still knows general geography. It only loses within-country distinctions. For many use cases, that's sufficient.

The privacy benefit:

Country level: practically impossible to re-identify an individual.
GPS level: trivial to re-identify (combined with a timestamp and other features, GPS alone almost always identifies a unique person).

Trade-off: marginal accuracy loss vs. a major privacy improvement. Frequently, reducing granularity is the right call.

3. What are the specific risks of PII in LLM system prompts?

Four:

Leakage in outputs: the LLM can mention the PII in its response, exposing it to the user or to anyone else who sees the conversation.
Logging of the prompt: system logs typically include the full prompt sent. PII persists in the logs.
API provider logs: if you use an OpenAI/Anthropic API, prompts can end up on their servers. It depends on the contract (some plans guarantee no logging, others don't).
Future training data: some providers may use conversations to train future models. If your PII was part of the prompt, it could end up in future model parameters.

Mitigations:

Minimize PII in prompts: only include what's necessary for the query.
Use opaque IDs instead of PII: customer_id instead of email.
Verify the provider's terms: that prompts aren't used for training.
Self-hosted models when privacy is critical.
Logging policies: filter PII out of logs before persisting.

4. When is it legitimate NOT to apply minimization?

Cases where minimization doesn't apply or is counterproductive:

Regulation requires precision: e.g., medical systems require the full history for safety. Minimization would create a safety risk.
Forensic/audit purposes: detail may be legally required for compliance.
Research with explicit consent: if users specifically consent to comprehensive data collection for research benefit, minimization can be relaxed.
Aggregations where minorities would become identifiable: paradoxically, extreme minimization (k=1 anonymization) can make individuals in small groups stand out. You have to balance.
Preventing critical errors: sometimes you need the detail to prevent system errors that would cause harm.

In all of these cases: document the justification. "We retain this level of detail because [specific reason]." Without a documented justification, the default is to minimize.

And always: even when you don't minimize a specific type, minimize the others. It isn't all-or-nothing.

Summary and next step

Data minimization: only collect/process what's necessary for the specific purpose.
A necessity test for every piece of data: can you fulfill the purpose without it?
Application across the AI lifecycle: training, inference, logging, conversations.
Reducing granularity is often better than removing features.
Honest trade-offs: sometimes minimization reduces model accuracy. Decide in an informed way.
The RAG case: a dramatic privacy improvement with a minor reduction in personalization.

Checkpoint: you should be able to review a system and identify at least 3 minimization opportunities.

Bridge to the next capsule: capsule 03 covers anonymization vs. pseudonymization — the two main techniques for processing PII while reducing risk. You'll learn when each applies, their limitations (re-identification attacks), and techniques like k-anonymity, l-diversity, and differential privacy at a conceptual level.

Resources

GDPR Art 5(1)(c) — Data Minimisation — the legal reference.
Privacy by Design — Principle 2 (Default) — context.
Data Minimization in ML (Microsoft Research) — research papers.

Next: 03-anonymization-pseudonymization.md — Anonymization vs. Pseudonymization.

Capsule 02 of 08 — Module 3 — AI Ethics & Compliance Guide

← Previous1. Module introduction: Privacy and Data Protection Fundamentals Next →3. Anonymization vs Pseudonymization