Module 3: Privacy and Data Protection Fundamentals

4. Consent in AI Systems

Next →

Capsule description

Consent is the central legal and ethical principle of privacy: people must authorize the use of their data. Without valid consent, processing is illegal in most jurisdictions (GDPR, CCPA, etc.).

But consent in AI is more complicated than in traditional software. Reasons:

  1. Users don't understand how AI processes their data. "Your data will be used to improve our services" → what does that mean technically?

  2. The uses are emergent: when you collected the data 2 years ago, you couldn't anticipate the uses that new LLMs make possible.

  3. Data is baked into models: once a sample is in a trained model, "revoke consent" requires retraining (expensive or impossible).

  4. Indirect data collection: data about person X can come from person Y (referrals, mentions, photos).

This capsule covers how to design meaningful consent in AI systems:

  1. The 4 elements of valid consent: freely given, specific, informed, unambiguous.
  2. Granularity: separate consent for different uses.
  3. Informed consent in practice: how to explain AI to non-experts.
  4. Revocation: how to handle withdrawal when the data is already in models.
  5. Edge cases: indirect data, public data, derived data.

The 4 elements of valid consent

GDPR Art 4(11) defines consent. The 4 required elements:

1. Freely given

The user must have a real option NOT to consent, with no penalty.

Violations:

  • "Accept or you can't use the service" (when the data isn't necessary for the service).
  • Pre-checked boxes.
  • Bundled consent ("I accept the terms and AI training").

Compliant:

  • Boxes unchecked by default.
  • A clear option to refuse without losing the core service.
  • Refusal doesn't significantly degrade UX.

2. Specific

Consent for a specific purpose, not a blanket one.

Violations:

  • "I agree that my data may be used to improve our services" (which services? which specific ones?)
  • A single checkbox for multiple unrelated purposes.

Compliant:

  • "I agree that my conversations may be used to train text generation models."
  • Separate boxes for distinct purposes.

3. Informed

The user must understand what they're consenting to.

Violations:

  • A 50-page Terms of Service.
  • Technical jargon that normal users don't understand.
  • Hiding key information in fine print.

Compliant:

  • A plain-language summary.
  • Concrete examples of how the data will be used.
  • An accessible FAQ.

4. Unambiguous

A clear action expressing consent.

Violations:

  • Continued use as consent ("by using this app, you consent...").
  • Inferred consent.
  • Silence.

Compliant:

  • An active checkbox.
  • Clicking an "I consent" button.
  • Clear separation between consent and other actions.

Granularity: separate consent per purpose

In AI, there are multiple potential uses of the same data. Each requires separate consent.

Example: a customer support chatbot

Data: conversations between the user and the AI.

Possible uses:

  1. Operate the service: handle the current query.
  2. Improve the model: fine-tune for better responses.
  3. Research: published research papers.
  4. Marketing: training models for advertising.
  5. Sell to partners: third-party applications.

The compliant approach: separate consent for each one.

☐ Use my conversations to provide customer support (REQUIRED)
☐ Use my conversations to improve the AI model (OPTIONAL)
☐ Use my (anonymized) conversations for AI safety research (OPTIONAL)
☐ Share my conversations with marketing partners (OPTIONAL)

The user can consent to the first (necessary) and refuse the others. The service should function regardless of the optional consents.

The "primary" purpose vs. "secondary" purposes

Primary purpose: what the user started the interaction for. Consent is typically implicit (using the service = consent).

Secondary purposes: anything else. Requires explicit additional consent.

Many AI platforms violate this by bundling secondary uses with the primary one.

Informed consent: how to explain AI

The challenge: most users don't understand AI technically. How do you achieve "informed" consent?

Approach 1: layered disclosure

Layer 1 (summary, all users see it):

We'll use your conversations to answer your questions better. If you allow it, we'll also use them to train our AI models.

Layer 2 (more detail, click to expand):

"Train AI models" means your conversations are included in datasets that teach our models to generate better responses. The models learn general patterns; they don't memorize specific details.

Layer 3 (technical, for those who want it):

Conversations are anonymized by removing emails, phone numbers, names, and addresses. They're stored encrypted. After fine-tuning, the data is purged from the raw store. The trained model may contain statistical traces but no individual records.

The user can choose their level of detail based on interest. Most users read layer 1, some layer 2, a few layer 3 — but the information is available.

Approach 2: concrete examples

Instead of the abstract:

❌ "Your data will be used to improve services."

✅ "If you ask 'how do I change my password?', we'll train the model to better answer similar questions from other users."

Concrete examples communicate what abstractions don't.

Approach 3: visual / interactive

An interactive demo:

"Let's see what would happen with your data:

  1. You send a message: 'Hi, I'm María, I have a problem with my order'
  2. The system anonymizes it: 'Hi, I'm [NAME], I have a problem with my order'
  3. The anonymized version goes into the training dataset
  4. Future users with similar questions get a better answer"

It demonstrates the real flow.

Revocation: the problem with trained models

The challenge

Imagine:

  1. María consents to her data being used for training (1 year ago).
  2. The model was trained with her data (included among 1M samples).
  3. The model is deployed and serving 100K queries/day.
  4. María revokes consent.

What do you do?

The options

Option 1: retrain without María's data

  • Cost: weeks of compute, $100K+.
  • Practicality: rarely feasible.

Option 2: "machine unlearning"

An active research area. Techniques that approximately "remove" a sample's influence without a full retrain.

  • Example: SISA (Sharded, Isolated, Sliced, Aggregated training).
  • Limitations: works better for some architectures than others.
  • No silver bullet yet.

Option 3: remove from the RAG / retrieval store, accept the model contains traces

  • If the system uses RAG (retrieval), remove it from the store.
  • The base model may contain traces, but the outputs no longer include María's data directly.
  • Often legally acceptable under GDPR if a "reasonable" effort was made.

Option 4: block future use without retraining

  • Document that María opted out.
  • Future training rounds exclude her data.
  • The existing model continues but there's no further use.

The realistic approach: a combination — remove from retrieval stores, opt out of future training, document if full unlearning is impossible.

What you CANNOT do

  • Ignore the request: the GDPR Right to Erasure (Art 17) is enforceable.
  • Charge a fee for revocation.
  • Delay indefinitely.

You have to respond, make a best effort, and document what you did and why not more.

Indirect data collection

Cases where you collect data about person X via person Y:

Case 1: photos with multiple people

A user uploads a selfie with a friend → your system processes the friend's face too.

Solution:

  • Notify users that photos with others require their consent.
  • Face detection + blurring for non-consenting individuals.
  • Limit retention of derivative data.

Case 2: mentions in conversations

User message: "My girlfriend Sarah said she wanted X."

Your system processes this with Sarah's name. Sarah didn't consent.

Solution:

  • PII redaction: detect names, replace with placeholders.
  • Don't persist mentions of third parties.
  • Educate users about not sharing third-party PII.

Case 3: communication apps

A user forwards an email from Bob to your AI for summarization.

Solution:

  • Process it inline, don't persist it.
  • Document that forwarded emails aren't stored.
  • Bob's data is processed only in the context of the specific request.

The general lessons

Indirect data is a harder problem. Mitigations:

  1. Inform users about their responsibility for third-party data they share.
  2. Detect and limit persistence of third-party PII.
  3. Process locally when possible (no cloud round-trip with PII).
  4. Audit periodically for unexpected third-party data accumulation.

Public data: is there consent?

A common belief: "the data is public, it can be used freely."

False under GDPR. Public data is still personal data. Processing requires a lawful basis (consent or legitimate interest, etc.).

Cases

Web scraping:

  • Scraping public LinkedIn profiles → potentially a violation.
  • Scraping public tweets → a grey area, depends on the use.
  • Scraping public government data → typically OK.

Foundation model training:

  • GPT, Claude, etc. trained on internet data → ongoing legal challenges.
  • New York Times v. OpenAI (2023): the claim that OpenAI used NYT articles without consent.
  • The outcome will shape industry standards.

Conservative best practices

  1. Use only data with a clear lawful basis (consent, documented legitimate interest).
  2. Respect robots.txt and similar signals.
  3. Don't bypass paywalls / authentication.
  4. Document data provenance carefully.
  5. Be prepared to remove data if requested.

Implementation: consent management

The consents table

CREATE TABLE user_consents (
    user_id INT,
    consent_type VARCHAR(50),  -- 'training', 'research', 'marketing', etc.
    granted_at TIMESTAMP,
    revoked_at TIMESTAMP NULL,
    consent_text TEXT,  -- exact text user agreed to
    consent_version VARCHAR(20),
    PRIMARY KEY (user_id, consent_type)
);

An API for consent management

class ConsentManager:
    def grant_consent(self, user_id, consent_type, version):
        """User grants consent."""
        self.db.execute("""
            INSERT INTO user_consents 
            (user_id, consent_type, granted_at, consent_version)
            VALUES (?, ?, NOW(), ?)
            ON DUPLICATE KEY UPDATE granted_at = NOW(), revoked_at = NULL
        """, (user_id, consent_type, version))
    
    def revoke_consent(self, user_id, consent_type):
        """User revokes consent."""
        self.db.execute("""
            UPDATE user_consents
            SET revoked_at = NOW()
            WHERE user_id = ? AND consent_type = ?
        """, (user_id, consent_type))
        
        # Trigger downstream actions
        self._handle_revocation_actions(user_id, consent_type)
    
    def has_active_consent(self, user_id, consent_type):
        """Check before processing."""
        result = self.db.fetchone("""
            SELECT 1 FROM user_consents
            WHERE user_id = ? AND consent_type = ?
            AND granted_at IS NOT NULL AND revoked_at IS NULL
        """, (user_id, consent_type))
        return result is not None
    
    def _handle_revocation_actions(self, user_id, consent_type):
        """Trigger removal/blocking based on consent type."""
        if consent_type == 'training':
            # Mark user data to exclude from future training
            self.exclude_from_future_training(user_id)
        if consent_type == 'retention':
            # Schedule data deletion
            self.schedule_deletion(user_id)

Integration: check consent before processing

async def process_for_training(user_id, conversation):
    if not consent_manager.has_active_consent(user_id, 'training'):
        return  # Skip, no consent
    
    anonymized = anonymize(conversation)
    add_to_training_set(anonymized)

The consent check is the first thing in any processing pipeline. Default deny.

Common traps

1. Bundled consent

"Accept all terms" combining 5 distinct purposes. It violates the "specific" requirement. Each purpose needs a separate checkbox.

2. "Continued use = consent"

A banner: "By continuing to use this site, you agree to AI training." Not valid. An active consent action is required.

3. Granting consent with no UI to revoke

Easy to grant, hard to revoke = it violates the spirit. Revocation must be similarly accessible.

4. Consent collected once, never refreshed

Consent given 5 years ago for "service improvements" doesn't cover new use cases (LLM training) that didn't exist then. Re-consent for materially new uses.

5. Default opt-in for AI

Pre-checked boxes are a GDPR violation. AI training participation should be opt-in, unchecked by default.

Self-check

1. What are the 4 elements of valid consent?

  1. Freely given: the user has a real option NOT to consent, with no penalty.

  2. Specific: for a specific declared purpose, not a blanket one.

  3. Informed: the user understands what they're consenting to.

  4. Unambiguous: a clear active action expressing consent (no inference, no continued use).

If one fails, the consent is invalid. Implication: processing based on invalid consent is illegal under GDPR (no lawful basis).

Practical: implement consent flows that clearly address each element. Document that you meet each one. Audit periodically.

2. Why does consent granularity matter?

Because AI has multiple possible uses of the same data:

  • Operate the service.
  • Improve the model.
  • Research.
  • Marketing.
  • Sell to partners.

Bundling these into a single consent violates GDPR's "specific" requirement. Each requires explicit separate consent.

Granular consent lets users:

  • Enable the core service (necessary).
  • Opt out of secondary uses they don't agree with.
  • Keep control over their data participation.

Implementation: separate checkboxes, separately revocable. Off by default for non-essential uses.

An additional benefit: users who trust how you handle their data grant more consents. Granularity = trust = more data overall (vs. bundling, where users may refuse everything).

3. How do you handle revocation when the data is already in a trained model?

The realistic approach is a combination:

  1. Remove from retrieval/RAG stores immediately: future queries don't use the data.

  2. Mark for exclusion in future training rounds: the next time the model is retrained, María's data is excluded.

  3. Machine unlearning if feasible: emerging techniques that approximate removal from the model. Not a full retrain, but a best effort.

  4. Document what was done and what wasn't: a paper trail showing reasonable effort. GDPR accepts a "reasonable" effort when full removal is impossible.

  5. Communicate honestly with the user: "We've removed your data from our retrieval stores and our future training will exclude it. Existing models may contain statistical traces that aren't individually identifiable, but they can't be fully removed without a full retrain."

What you CANNOT do:

  • Ignore the request.
  • Pretend that "anonymized" data after model training is no longer her data.
  • Charge for processing the request.
  • Wait indefinitely.

Bottom line: best effort + transparency + a paper trail. Most regulators accept this when the alternative is technically impossible.

4. Why is public data still personal data under GDPR?

GDPR defines personal data as "any information relating to an identified or identifiable natural person." No mention of public/private.

If the data identifies a person, it's personal data, regardless of where you got it.

Implication: scraping public LinkedIn, Twitter, etc. for AI training requires a lawful basis:

  • Consent (the individual's explicit consent).
  • Legitimate interest (documented, balanced against rights).
  • Contract necessity.
  • Legal obligation.
  • Vital interest.
  • Public task.

For most foundation model training, "consent" isn't feasible at scale (you can't get consent from every individual mentioned online). Companies typically claim "legitimate interest" — but that requires:

  • A documented assessment.
  • A balancing test (your interest vs. the individual's rights).
  • A user option to object.

Ongoing legal challenges (NYT v. OpenAI) are testing these claims. The outcome will set industry standards.

Practical recommendation:

  • Document data provenance carefully.
  • Have a process for removing specific individuals upon request.
  • Use data with a clear lawful basis when possible.
  • Be prepared for legal scrutiny as regulations evolve.

Summary and next step

  • Valid consent requires 4 elements: freely given, specific, informed, unambiguous.
  • Granularity matters: separate consent for each distinct purpose.
  • Informed consent in AI requires plain-language explanations, layered disclosure, concrete examples.
  • Revocation is challenging with trained models. Best effort + transparency + documentation.
  • Indirect data collection (third parties mentioned, photos with others) requires special handling.
  • Public data is still personal data under GDPR. A lawful basis is required.

Checkpoint: you should be able to evaluate a consent flow and identify violations of each of the 4 elements.

Bridge to the next capsule: capsule 05 covers retention and the data lifecycle: how long to keep data, automatic purge policies, handling "right to deletion" requests. Critical, because data persists longer than planned by default.

Resources

  1. GDPR Art 4(11), 6, 7 — definitions and conditions for consent.
  2. Article 29 Working Party — Consent Guidelines — interpretive guidance.
  3. California Consumer Privacy Act (CCPA) — the US comparison.
  4. Machine Unlearning research — emerging techniques.

Next: 05-retention-lifecycle.md — Retention, purge, and the right to erasure.

Capsule 04 of 08 — Module 3 — AI Ethics & Compliance Guide

← Previous3. Anonymization vs PseudonymizationNext →5. Retention and the Data Lifecycle