Module 5: GDPR for AI Systems

Art. 22: Automated Decision-Making in detail

Description

GDPR Art. 22 is the most relevant article for AI Engineers. This capsule goes deep: exactly when it applies, what exceptions exist, what level of explanation is required, and how to comply with it in real AI systems.

By the end you'll be able to:

Identify precisely when Art. 22 applies to your system
Apply the 3 legal exceptions correctly
Specify what information to provide to the data subject under Art. 22(3)
Implement "right to challenge" mechanisms in your product

The 3 criteria for Art. 22 to apply

Art. 22 applies only if all 3 criteria are met:

Criterion 1: A decision based SOLELY on automated processing

"Solely" means with no meaningful human intervention. A rubber-stamping human review (with no real analysis) does NOT count as intervention.

Scenario	"Solely automated"?
AI scoring + a human reviews each case with the data before deciding	NO — there's human analysis
AI scoring + a human approves without reviewing the case	YES — a rubber stamp
AI scoring + auto-approve if > threshold, human if < threshold	It depends — the auto-approve branch is "solely"
AI scoring + the AI decides + an automated email to the user	YES

Implication: to avoid Art. 22, you need meaningful human review, not a checkbox.

Criterion 2: Producing legal effects or similarly significantly affecting someone

Legal effects: it changes legal rights (denial of credit, termination of employment, visa denial)
Similarly significant: a comparable impact even if not strictly legal (discriminatory personalized pricing, exclusion from services, medical denial)

Debated cases (no clear consensus):

Individual dynamic pricing: probably "significant"
Automatic filters on social networks: probably NOT significant
Personalized recommendations: typically NOT significant

Criterion 3: It applies to a "data subject" (a natural person)

Automated decisions affecting companies (B2B) → Art. 22 does NOT apply
Decisions affecting employees (HR systems) → it DOES apply
Decisions affecting consumers → it DOES apply

The 3 exceptions in Art. 22(2)

Even when all 3 criteria are met, there are 3 legitimate exceptions:

Exception A: Necessary for a contract

"Necessary for entering into, or performance of, a contract"

It applies if: without the automated decision, the contract couldn't be formed or performed.

A valid example: auto-scoring for a loan (necessary to process the volume; without scoring it's impossible). A NOT valid example: personalized pricing (it isn't "necessary" — you could use uniform pricing).

Exception B: Authorized by law

"Authorised by Union or Member State law"

E.g.: bank fraud detection (financial regulation authorizes it). KYC/AML compliance.

Exception C: Explicit consent

"Based on the data subject's explicit consent"

"Explicit" is the key word: not a general "checkbox consent," but consent specific to this automated decision.

Requirements for explicit consent:

Informed: the user knows specifically what they're consenting to
Specific: for the concrete automated decision
Unambiguous: a clear affirmative action
Revocable: the user can withdraw consent

What information must the system provide? (Art. 22(3))

Even with an applicable exception, the system must provide:

Meaningful information about the logic involved
The significance of the decision
The envisaged consequences

Plus the data subject's rights:

The right to human intervention
The right to express their point of view
The right to contest the decision

What "meaningful information about the logic" means

This is ambiguous and controversial. The acceptable minimum:

Which factors were considered
How they were weighted (not necessarily the exact weights)
What outcomes are possible

A good example:

"Your application was evaluated automatically considering: credit history (40% of the weight), stable income (30%), employment tenure (20%), and other factors (10%). In your case, your credit history showed X result, which led to a negative decision. You can request human review with this form."

A bad example:

"Your application was rejected by our system."

A worked example: a credit scoring system

The system

An AI model that approves/rejects loans automatically.

The Art. 22 analysis

Does Art. 22 apply?

Solely automated: ✅ (no human review in the normal flow)
Legal/significant effect: ✅ (access to credit)
Data subject (a consumer): ✅

→ Yes, Art. 22 applies.

Which exception could apply?

(a) Contract necessity: ✅ arguable (a loan is a contract, and scoring is necessary)
(b) Authorized by law: partially (banking regulation)
(c) Explicit consent: possible, but it requires specific consent at loan application time

The compliance design:

1. Information at loan application time:
   "Your application will be evaluated automatically by our scoring system.
   It considers: credit history, income, tenure. You have the right to
   request human review after the decision."

2. Consent checkbox (separate from the general contract):
   [ ] I consent to automated evaluation of my loan application

3. Decision notification:
   "Your application was [approved/rejected]. The decision was based on X factors.
   Main reason: [the most significant factor]. You can:
   - Request human review → [link]
   - Appeal the decision → [link]
   - Learn more about the logic → [link]"

4. Internal processes:
   - Complete logging of inputs, model output, decision, timestamps
   - A human review queue workflow
   - Templates for responding with explanations

Common traps

Trap 1 — "We have human review" but it's a rubber stamp. The human approves 99% without reviewing. It doesn't count as meaningful intervention.

Trap 2 — Consent buried in the general T&Cs. For Art. 22, consent must be specific and separate.

Trap 3 — An invented post-hoc explanation. "The system rejected it because of X." But the system doesn't expose X — you made it up. That's deception and a violation of honesty.

Trap 4 — A "right to challenge" with no real process. An "appeal" button that goes to /dev/null. You need a real workflow, an SLA, a response process.

Trap 5 — Getting the geographic application wrong. "My system runs in the USA." If the decisions affect EU residents, GDPR applies.

Exercise

Your product: an AI-powered ATS (Applicant Tracking System) that filters résumés and shows only the top 20% to the human recruiter. The remaining 80% get an automated "you're not moving forward" email.

Does Art. 22 apply?
Which exception could you use?
Design the compliance flow (information, consent, decision communication, right to challenge)

See the solution

Yes, it applies:
- Solely automated: the rejected 80% never see human review
- Significant effect: access to a job opportunity
- Data subject: the candidate (a natural person)
Exceptions:
- (a) Contract necessity: does NOT clearly apply (the hiring process isn't a contract yet)
- (c) Consent: possible if the candidate consents when applying

The compliance flow:

1. Job posting page:
   "Your résumé will be evaluated by our AI system. The system looks for matches
    based on experience, skills, and education against the job description.
    [ ] I consent to automated processing"

2. Submission acknowledgment:
   "Your application is being evaluated. You'll get a response within 5 days."

3. If the AI rejects it:
   "We're sorry to inform you that your application is not moving forward. The evaluation
    was based on a match of skills, experience, and education against the job description.
    In your case, there wasn't a significant match in [area].

    If you think the evaluation was unfair, you can:
    - Request human review → [link, 5-day response SLA]
    - Learn more about the criteria → [link]"

4. Internal:
   - A log of the inputs (parsed CV fields), the AI score, the decision
   - A human review queue with an SLA
   - An annual bias audit of the decisions

Summary

You learned:

✅ The 3 criteria for Art. 22 to apply (solely automated, significant effect, data subject)
✅ The 3 legal exceptions (contract necessity, law, explicit consent)
✅ The mandatory information for the data subject (logic, consequences, rights)
✅ A worked case: credit scoring
✅ The traps: rubber stamping, buried consent, invented explanations

Checkpoint: if you can analyze an AI system and determine Art. 22 applicability + design the compliance, you're ready.

Next capsule

03 — The Right to Explanation for LLMs. The hardest tension: how do you meet the "right to explanation" with a fundamentally opaque LLM?

Resources

GDPR Article 22.
WP29 Guidelines on Automated Decision-Making — the official interpretation.
Bygrave, "Article 22 GDPR" — legal analysis.
Case studies of Art. 22 enforcement — real fines.

← PreviousIntroduction to GDPR for AI Systems Next →The Right to Explanation for LLMs