Módulo 3: Privacy and Data Protection Fundamentals

8. Mini-proyecto: Privacy Assessment

Siguiente →

Cerrás el módulo (y la Phase 1 de la guía) aplicando todo lo aprendido a un Privacy Assessment comprehensivo de un sistema AI real.

Es el equivalente del Ethics Impact Analysis (M1/08) pero focused on privacy specifically. Documento de 4-6 páginas que demuestra dominio del proceso y produce artifact reusable.

Especificación del entregable

Mínimo viable

  • ✅ Sistema definido: caracterización data-focused.
  • ✅ Data inventory: lista de all data types, sources, lifecycle.
  • ✅ Minimization assessment: cada data type evaluated for necessity.
  • ✅ Anonymization decisions: para cada PII, qué technique aplicar.
  • ✅ Consent flow: documentado para each purpose.
  • ✅ Retention policy: tabla con periods + automation.
  • ✅ AI-specific risks: assessment of membership inference, model inversion, prompt leakage, memorization.
  • ✅ Privacy by Design checklist: 7 principles evaluated.
  • ✅ Mitigation plan: concrete actions con owners y timelines.

Stretch goals

  • ⭐ Implementación: code at least 1 mitigation (PII filter, consent manager, etc.).
  • ⭐ Stakeholder review: legal/security feedback.
  • ⭐ GDPR pre-mapping: identify which articles each section addresses.
  • ⭐ Periodic re-evaluation calendar: schedule next review.

Elegir el sistema

Mismas opciones que M1/08 y M2/08:

Opción A: tu Capstone del path AI Engineering.

Opción B: un sistema de tu trabajo (con permission).

Opción C: case study provisto:

C1: Healthcare Triage System (recommendado para privacy focus)

Sistema AI que recibe síntomas de pacientes via chat web y clasifica urgencia. LLM-based con prompt engineering. Datos de entrenamiento: 10,000 conversaciones históricas anonimizadas. Deploy: clínica con 50K visits/año en US (HIPAA + state laws).

Particularly rich for privacy because:

  • HIPAA applies (US).
  • Health data is "sensitive" under GDPR (special category).
  • LLM provider third party.
  • Conversations contain PII potentially.

C2: Customer Support Chatbot

Sistema RAG-based para customer support. Conversations stored, indexed, used for FAQ generation. 1M users globally (US + EU + LATAM).

Privacy aspects:

  • GDPR (EU users).
  • CCPA (CA users).
  • LATAM regulations (LGPD Brazil, Argentina PDPA).
  • Cross-jurisdictional compliance challenge.

C3: Loan Approval System

Como en M1/08. Predicts default probability. Trained on 5 years internal data including financial PII.

Privacy aspects:

  • GLBA (US financial).
  • FCRA (credit reporting).
  • GDPR (EU customers).
  • Sensitive financial data → high stakes if leaked.

Estructura del documento

# Privacy Assessment: [System Name]

## 1. Executive Summary
- System overview (1 paragraph)
- Privacy risk level (low/medium/high)
- Key findings
- Critical mitigations needed

## 2. System Characterization
[See section below]

## 3. Data Inventory
[Comprehensive list]

## 4. Data Minimization Assessment
[Per data type evaluation]

## 5. Anonymization & Pseudonymization Decisions
[What technique per data type]

## 6. Consent Management
[Flows, purposes, granularity]

## 7. Retention Policy
[Periods, automation, deletion processes]

## 8. AI-Specific Privacy Risks
[Each attack assessed]

## 9. Privacy by Design Evaluation
[7 principles checklist]

## 10. Mitigation Plan
[Concrete actions]

## 11. Monitoring & Re-evaluation
[Ongoing privacy posture]

## 12. Decision Log
[Authors, reviewers, dates]

4-6 páginas. Compact pero comprehensive.

Implementación detallada

Sección 2: System Characterization

Different from Ethics Impact Analysis — focus is data flows, not just functionality.

## 2. System Characterization

**System**: Healthcare Triage Chatbot

**Functionality**: Receives patient symptoms via web chat, classifies urgency 
(immediate / 24h / week / non-urgent), recommends action.

**Data flows**:
1. User submits symptoms via web form (TLS to our server)
2. Server constructs prompt with symptoms + medical knowledge context
3. Prompt sent to LLM API (OpenAI, US region)
4. LLM response received
5. Response shown to user
6. Conversation saved (encrypted)

**Data lifecycle stages**:
- **Input**: user types symptoms in web form
- **Processing**: prompt construction, LLM call, response classification
- **Storage**: conversation stored 90 days (per medical record retention)
- **Training**: 5% of conversations (with consent) used for fine-tuning quarterly

**Jurisdictions**: US (HIPAA, state-specific medical privacy laws)

**Third parties**: 
- OpenAI (LLM provider, BAA signed for HIPAA)
- AWS (hosting, BAA signed)
- Stripe (payment, no medical data)

**User base**: US adults seeking medical guidance; ~50K patients/year

Sección 3: Data Inventory

## 3. Data Inventory

| # | Data Type | Source | Sensitivity | Storage | Retention |
|---|-----------|--------|-------------|---------|-----------|
| 1 | Patient name | User input | High (PII) | DB encrypted | Account life + 90d |
| 2 | DOB | User input | Medium | DB encrypted | Account life + 90d |
| 3 | Symptoms text | User input | Critical (PHI) | DB encrypted | 90 days |
| 4 | Triage result | System output | Medium | DB encrypted | 90 days |
| 5 | LLM prompt | Generated | Critical (PHI) | LLM API logs | 30 days (per OpenAI) |
| 6 | LLM response | Generated | High | DB + logs | 90 days |
| 7 | User account | User input | Medium | DB encrypted | Until deletion |
| 8 | Payment data | Stripe | High | Stripe (not us) | Per Stripe |
| 9 | Audit logs | System | Low | Encrypted store | 6 years |

**Total data types**: 9
**With PHI**: 4 (#1, #3, #5, #6)
**With PII**: 7 (excluding #4, #9)

Sección 4: Data Minimization Assessment

## 4. Data Minimization Assessment

For each data type:

### Patient name (#1)
- **Necessary?** YES — required for medical record creation
- **Granularity?** Full name needed (legal medical record requirement)
- **Recommendations?** None — required minimum

### DOB (#2)
- **Necessary?** YES — for age-based triage decisions
- **Granularity?** Full DOB for medical accuracy. Could be year only?
  - **Decision**: Full DOB. Medical accuracy outweighs privacy gain of bracket.

### Symptoms text (#3)
- **Necessary?** YES — core functionality
- **Granularity?** As detailed as user provides
- **Recommendations?** None — minimum for triage

### LLM prompt (#5)
- **Necessary?** YES — required to query LLM
- **Improvements**:
  - Strip patient name, DOB before sending to LLM (LLM doesn't need)
  - Only symptoms + age bracket sent
  - Pre-action: implement prompt sanitization

### Audit logs (#9)
- **Necessary?** YES — HIPAA compliance requires audit trail
- **Improvements**:
  - Reduce content: don't log full prompts/responses, only IDs
  - Implement PII redaction in logs

**Action items**:
- [ ] Implement LLM prompt sanitization (strip name, exact DOB)
- [ ] Implement log PII redaction
- [ ] Owner: ML engineer + DevOps. Timeline: 2 weeks.

Sección 5: Anonymization Decisions

## 5. Anonymization & Pseudonymization Decisions

| Data | Strategy | Reasoning |
|------|----------|-----------|
| Patient name | None (required identifier) | Medical record requires identification |
| Symptoms (in prompt to LLM) | Pseudonymize patient name; remove if mentioned | LLM doesn't need name to assess symptoms |
| Training data (5% sample) | k-anonymity (k=10) + name removal | Aggregates while preserving pattern signal |
| Logs | PII redaction | Don't need actual values for debugging |
| Aggregated stats (analytics) | Differential privacy | Public stats need DP guarantee |

**Decision**: NOT using DP for training because:
- Accuracy cost too high (medical accuracy critical)
- Sample size sufficient for k-anonymity to be meaningful
- Combined with consent + access controls, sufficient privacy

Sección 6: Consent Management

## 6. Consent Management

**Purposes** (each with separate consent):

1. **Service operation** (REQUIRED):
   - "Use my data to provide medical triage service"
   - Cannot use service without

2. **Anonymized research** (OPTIONAL):
   - "Allow my anonymized conversations to be used for AI safety research"
   - Default OFF

3. **Model improvement** (OPTIONAL):
   - "Allow my anonymized conversations to be used to improve our AI"
   - Default OFF

4. **Marketing** (OPTIONAL):
   - "Allow contact about new health features"
   - Default OFF

**Implementation**:
- Consent UI separate checkboxes, all unchecked by default except #1
- Consent recorded with timestamp + version
- Consent revocable from privacy settings
- Revocation triggers exclusion from future training (within 24h)
- Existing models contain residual data (documented to user)

**Information disclosure**:
- Plain-language privacy notice
- Layered: summary, details, technical
- FAQ accessible

Sección 7: Retention Policy

## 7. Retention Policy

| Data Type | Retention | Mechanism | Verification |
|-----------|-----------|-----------|--------------|
| Active conversations | 30 days | Daily cron job | Weekly audit query |
| Triage results | 90 days | Daily cron job | Weekly audit query |
| Account data (active) | While active + 90d post-cancel | Auto-trigger | Monthly audit |
| Backups | 90-day rolling | S3 lifecycle | Auto-managed |
| Audit logs | 6 years | Immutable storage | Compliance review |
| Training data (5% sample) | Until consent revoked OR 1 year | Quarterly job | Per quarter check |

**Right to Erasure handling**:
- API endpoint for user request
- 24h verification + processing
- Deletes from primary, marks for backup cleanup
- Excludes from future training
- Documents limitation: existing trained models contain traces
- 30-day completion timeline

**Action items**:
- [ ] Implement automated retention cron jobs
- [ ] Implement Right to Erasure API
- [ ] Document handling of model traces in privacy notice

Sección 8: AI-Specific Risks

## 8. AI-Specific Privacy Risks

### Risk 1: Membership Inference

**Concern**: Attacker queries the model to determine if specific patient was in training set.

**Severity**: Medium (revealing membership in healthcare dataset is sensitive)

**Probability**: Low (limited model access; rate limiting in place)

**Mitigations**:
- DP NOT used (accuracy trade-off too high for medical)
- Output regularization: return classification only, not confidence scores
- Rate limiting: 100 queries/hour/user
- Monitoring: anomalous query patterns

**Status**: Acceptable with mitigations.

### Risk 2: Model Inversion

**Concern**: Reconstruct training symptoms from model.

**Severity**: High (medical data reconstructed)

**Probability**: Low (LLM with much larger pre-training dilutes inversion)

**Mitigations**:
- Training data minimal in fine-tuning
- API access only (no weight access)
- Rate limited

**Status**: Acceptable but monitor research developments.

### Risk 3: Prompt Leakage

**Concern**: System prompt contains medical context; injection could leak.

**Severity**: High

**Probability**: Medium (LLMs have known injection vulnerabilities)

**Mitigations**:
- Patient PII NOT in system prompt (already implemented)
- Output filter for sensitive medical terms
- Prompt structure: clear separation of instructions and user input
- Two-LLM pattern for high-stakes queries

**Status**: Mitigated.

### Risk 4: Memorization

**Concern**: Fine-tuned model reproduces verbatim training patient data.

**Severity**: Critical

**Probability**: Medium (LLMs known to memorize)

**Mitigations**:
- Training data deduplicated
- PII redacted before training (names, DOBs, addresses, etc.)
- Periodic memorization auditing (quarterly)
- If memorization detected, retrain with stricter PII filter

**Status**: Mitigated, monitoring required.

Sección 9: Privacy by Design Evaluation

## 9. Privacy by Design Evaluation

| # | Principle | Status | Evidence |
|---|-----------|--------|----------|
| 1 | Proactive | ✅ | This Assessment + Ethics Impact Analysis pre-deployment |
| 2 | Default | ✅ | Consent OFF for non-essential; minimal data collected |
| 3 | Embedded | ✅ | Architecture review with privacy annotations; data flow diagram |
| 4 | Full functionality | ✅ | Privacy + functionality co-designed; not zero-sum |
| 5 | End-to-end security | ✅ | TLS, encryption at rest, audit logs, access controls |
| 6 | Visibility | ⚠️ | Privacy policy yes; model card pending |
| 7 | Respect for users | ✅ | Easy access, deletion, settings; clear notices |

**Action items**:
- [ ] Publish model card (#6 partial gap)
- [ ] Owner: ML lead. Timeline: 1 week.

Sección 10: Mitigation Plan

## 10. Mitigation Plan

| # | Action | Priority | Owner | Timeline |
|---|--------|----------|-------|----------|
| 1 | Implement LLM prompt sanitization | High | ML eng | 2 weeks |
| 2 | Implement log PII redaction | High | DevOps | 2 weeks |
| 3 | Automated retention cron jobs | Medium | DevOps | 4 weeks |
| 4 | Right to Erasure API | High | Backend | 4 weeks |
| 5 | Memorization auditing setup | Medium | ML eng | 6 weeks |
| 6 | Publish model card | Medium | ML lead | 1 week |
| 7 | Two-LLM pattern for high-stakes | Low | ML eng | 8 weeks |
| 8 | Privacy notice update | High | Product + legal | 2 weeks |

**Total estimated effort**: 8 weeks of distributed work.
**Critical path**: items #1, #2, #4, #6, #8 must complete before deploy.

Sección 11: Monitoring & Re-evaluation

## 11. Monitoring & Re-evaluation

**Continuous metrics**:
- Failed consent checks (alert if non-zero)
- PII detected in logs (alert if non-zero)
- Right to erasure processing time (target < 24h)
- Memorization audit results (target < 1%)
- Privacy-related user complaints (track trend)

**Re-evaluation calendar**:
- Quarterly: review metrics, update if needed
- Semi-annually: full Privacy Assessment review
- On material change: any architecture change triggers re-review

**Triggers for ad-hoc review**:
- New jurisdiction added
- New data source added
- New third-party integration
- Significant feature change
- Privacy incident (internal or industry)

Sección 12: Decision Log

## 12. Decision Log

| Date | Action | By | Notes |
|------|--------|-----|-------|
| 2026-05-08 | Privacy Assessment v1 created | [Author] | Initial assessment |
| 2026-05-15 | Reviewed by legal | [Legal lead] | No changes; approved |
| 2026-05-22 | Reviewed by security | [Security lead] | Approved with note on #2 |
| 2026-06-01 | Approved for deployment | [VP Product] | Conditional on items #1, #2, #4, #6, #8 |
| 2026-12-01 | Scheduled re-review | — | Calendar reminder |

Cómo proceder

Paso 1: bloque de tiempo (6-10 horas)

Esto es más laborioso que Bias Audit. Reservá time apropiado.

Paso 2: data inventory primero (2-3h)

Sin inventory completo, todo el resto es vago. Worth the time.

Paso 3: minimization assessment (1-2h)

For each data type, hard questions: ¿es realmente necesario? Most teams find 20%+ es overcollection.

Paso 4: technical sections (2-3h)

Anonymization, consent, retention, AI risks. Concrete decisions.

Paso 5: PbD checklist (30 min)

Evaluate against 7 principles. Identify gaps.

Paso 6: mitigation plan (1h)

Concrete actions. Owners. Timelines.

Paso 7: review

Stakeholder review (legal, security, ML lead). Iterate.

Cierre del módulo

8 cápsulas

  1. Introducción al módulo.
  2. Data minimization.
  3. Anonymization vs pseudonymization.
  4. Consent en AI.
  5. Retention y lifecycle.
  6. AI-specific privacy risks.
  7. Privacy by Design.
  8. Mini-proyecto: Privacy Assessment (esta cápsula).

Lo que tenés ahora

Tres herramientas integradas (Phase 1 completa):

  1. Ethics Impact Analysis (M1) — risks + stakeholders.
  2. Bias Audit Toolkit (M2) — fairness measurement.
  3. Privacy Assessment (M3) — privacy posture comprehensiva.

Aplicadas a un sistema, te dan vista 360° de su responsibility.

Empezamos en el siguiente módulo

Módulo 4: EU AI Act Deep Dive marca el inicio de Phase 2: Regulatory Compliance.

La transición:

"Entendés impact, bias, y privacy como fundamentos. Ahora vemos cómo la regulación más importante del mundo (EU AI Act) formaliza estos conceptos en obligaciones legales con categorías de risk, requisitos obligatorios, y multas hasta €35M."

Privacy connecta directamente: EU AI Act tiene data quality y governance requirements que se overlap con minimization. GDPR (M5) lo formaliza completamente.

Prep work: ningún. El framework conceptual de M3 te prepara naturalmente.

Recursos para el ejercicio

  1. Microsoft Privacy Threat Modeling — methodology.
  2. NIST Privacy Framework — framework completo.
  3. ICO Privacy Impact Assessment template — UK regulator template.
  4. GDPR Art 35 — Data Protection Impact Assessment — when DPIA is required.
  5. HIPAA Security Risk Assessment Tool — for healthcare.

Cápsula 08 de 08 — Módulo 3 — AI Ethics & Compliance Guide

Fin del módulo 3. Fin de Phase 1: Ethics Foundations. Continúa con el módulo 4 (EU AI Act Deep Dive).

← Anterior7. Privacy by Design en Sistemas AISiguiente →1. Introducción al módulo: EU AI Act Deep Dive