Module 2: Machine Learning Fundamentals

3. Supervised Learning: Learning with Labels

Next →

Description

In this lesson you'll understand Supervised Learning: the most common type of Machine Learning and the one used in most of the products you know (spam filters, image recognition, price prediction, etc.). The idea is simple: you give the system examples with correct answers (labels), and the system learns to predict those answers on new data.

This matters because when we talk in later modules about "training a model" or "classification" or "regression", you'll be looking at examples of supervised learning. Without this concept, terms like "labeled data" or "labels" would be black boxes. Here you build that foundation: what supervised learning is, what types there are (classification vs regression), and concrete examples in products.

What is Supervised Learning?

Definition: A type of Machine Learning where the system learns from examples that have correct answers (labels). The goal is for the model to learn to predict those answers on new (unseen) data.

Analogy: It's like learning with a teacher who corrects you. You show the system "This email is spam" (example + correct answer), "This email isn't spam" (example + correct answer), thousands of times. The system learns what distinguishes spam from non-spam. When a new email arrives (without a label), the model predicts "spam" or "not-spam" based on what it learned.

Basic flow:

Training data (examples + labels) → Algorithm → Model
New data (without labels) → Model → Prediction (estimated label)

Concrete example (spam filter):

  1. Training data: 10,000 emails, each manually labeled "spam" or "not-spam".
  2. Algorithm: For example, Logistic Regression, Random Forest, or a neural network.
  3. Model: After training, the model has parameters that represent "if these words appear with this frequency, the probability of spam is X".
  4. New data: A new email "Make money fast!".
  5. Prediction: The model predicts "spam" with 95% confidence.

Key point: Without the labels (without knowing which emails are spam and which aren't), the model can't learn. That's why it's called "supervised": someone (a human or an automated process) had to label the training data.

The two main types: Classification and Regression

Supervised Learning splits into two types depending on the kind of answer it predicts:

1. Classification

Definition: Predicting a discrete category or class. The answer is a label (spam/not-spam, cat/dog, approved/rejected).

Examples:

  • Spam filter: Is this email spam or not? (2 classes: spam, not-spam)
  • Image recognition: What's in this photo? (multiple classes: cat, dog, car, person, etc.)
  • Medical diagnosis: Is this tumor benign or malignant? (2 classes)
  • Sentiment analysis: Is this comment positive, negative or neutral? (3 classes)

Binary vs multiclass classification:

  • Binary: Only 2 classes (spam/not-spam, yes/no, approved/rejected).
  • Multiclass: More than 2 classes (cat/dog/horse/bird, etc.).

Analogy: It's like a multiple-choice exam: the model chooses among predefined options (the classes).

2. Regression

Definition: Predicting a continuous number. The answer is a numeric value (price, temperature, distance, time).

Examples:

  • House price: Predicting how much a house is worth based on its features (size, location, number of rooms). Answer: a number (e.g. $250,000).
  • Sales forecasting: How many units will we sell next month? Answer: a number (e.g. 1,523 units).
  • Temperature: Predicting tomorrow's temperature from historical data. Answer: a number (e.g. 22°C).
  • Arrival time: How long will it take me to reach my destination? Answer: a number (e.g. 25 minutes).

Difference from classification: In classification, the model chooses among discrete categories (spam/not-spam). In regression, the model predicts a number in a continuous range (e.g. a price between $100,000 and $1,000,000).

Analogy: It's like an open-answer exam: the model gives a specific number, not just a choice among options.

Quick comparison: Classification vs Regression

AspectClassificationRegression
OutputA category (spam/not-spam, cat/dog)A continuous number (price, temperature, time)
ExampleIs this email spam? → yes/noHow much is this house worth? → $250,000
Type of variableDiscrete (finite options)Continuous (a range of numbers)
AnalogyMultiple choiceNumeric answer
Common algorithmsLogistic Regression, Decision Trees, SVMLinear Regression, Gradient Boosting, Neural Networks

Note: Both are supervised learning because in both cases the model learns from data with correct answers (labels in classification, numeric values in regression).

How Supervised Learning works (conceptual flow)

  1. Collect labeled data: You need examples (inputs) and correct answers (outputs/labels). E.g. emails with the label "spam" or "not-spam".

  2. Split the data: You separate the data into two sets:

    • Training set: 70-80% of the data. It's used to train the model.
    • Test set: 20-30% of the data. It's used to evaluate how well the model works on unseen data.

  3. Train the model: The algorithm adjusts the model's internal parameters to minimize errors on the training set. E.g. "If the model predicts 'spam' and the real label is 'not-spam', adjust the parameters to reduce that error."

  4. Evaluate the model: You use the test set (data not seen during training) to measure how well the model predicts. Common metrics: accuracy, precision, recall, F1-score (for classification); MAE, RMSE (for regression).

  5. Use the model (inference): If the model works well on the test set, you put it into production. Now the model receives new data (without labels) and predicts the answer.

Visual flow:

Labeled data
    ↓
70% Training + 30% Test
    ↓
Training → Algorithm → Model
    ↓
Evaluate model with Test set → Does it work well?
    ↓ Yes
Production → New data → Model → Prediction

Examples of Supervised Learning in products

1. Spam Filter (Gmail, Outlook)

  • Type: Binary classification (spam / not-spam).
  • Training data: Millions of emails labeled manually or by users ("Mark as spam").
  • Model: It learns which words, subject patterns, senders, etc. are indicative of spam.
  • Use: When a new email arrives, the model predicts whether it's spam and filters it automatically.

2. Image Recognition (Google Photos, Facebook)

  • Type: Multiclass classification (cat, dog, person, car, etc.).
  • Training data: Millions of labeled photos (e.g. ImageNet: 14 million images across 20,000 categories).
  • Model: It learns which visual patterns (edges, shapes, colors, textures) distinguish a cat from a dog, a car from a person, etc.
  • Use: When you upload a photo, the model predicts what's in it and lets you search for "cats" in your library.

3. Price Prediction (Zillow, Airbnb)

  • Type: Regression (predicting a price in $).
  • Training data: Thousands of houses/apartments with features (size, location, number of rooms, etc.) and sale/rental price.
  • Model: It learns which features influence the price (e.g. more square meters → higher price; better location → higher price).
  • Use: When you enter the features of a new house, the model predicts how much it should be worth.

4. Speech Recognition (Siri, Alexa, Google Assistant)

  • Type: Sequential classification (converting audio into text, where each audio fragment is classified as a word or phoneme).
  • Training data: Millions of audio recordings labeled with the corresponding text.
  • Model: It learns which audio patterns (frequencies, tones, durations) correspond to which words.
  • Use: When you speak, the model predicts what you said (it converts audio into text).

5. Machine Translation (Google Translate, DeepL)

  • Type: Sequential classification (predicting the next word in the target language, given a sentence in the source language).
  • Training data: Millions of sentence pairs (English-Spanish, English-French, etc.).
  • Model: It learns which words in English correspond to which words in Spanish, and how to structure sentences grammatically.
  • Use: When you write a sentence in English, the model predicts the translation in Spanish.

All of these are supervised learning because the model learned from data with correct answers (labels on images, text for audio, translated sentences, etc.).

Why this matters for an AI Engineer

As an AI Engineer, you don't train supervised models from scratch in most cases (that's done by ML Engineering teams or by providers like OpenAI, Google, Meta). But understanding supervised learning helps you:

  1. Understand what a model can do: If the model was trained on labeled data (e.g. photos of cats and dogs), it can classify new photos of cats and dogs. If it wasn't trained with photos of horses, it won't recognize them well. That helps you reason about limits.

  2. Evaluate the model's quality: When you use a model via API (e.g. an image classification API), you can ask yourself: "What data was it trained on? How good is the accuracy on test data?" That helps you decide whether the model works for your use case.

  3. Fine-tuning: If you need to adjust a pre-trained model with your own data (e.g. fine-tuning GPT with your company's data), you're doing supervised learning (you give it examples with correct answers). Understanding the concept helps you design that process.

  4. Communicate with product and business: Being able to explain "the model needs labeled data to learn" or "to improve the model we need more examples with correct answers" lets you justify decisions and expectations.

Limitations and challenges of Supervised Learning

1. You need labeled data (and a lot of it)

Problem: Labeling data is expensive and slow. Someone (a human or an automated process) has to review each example and assign the correct label. For complex problems (e.g. recognizing 1,000 categories of objects in photos), you need millions of labeled examples.

Example: ImageNet (the reference dataset for vision) has 14 million manually labeled images. That required years of work and thousands of people.

2. The model only learns from what it saw

Problem: If the model didn't see examples of a category during training, it won't recognize it. If the training data has biases (e.g. only photos of cats against a white background), the model may not work well with different backgrounds.

Example: A model trained with photos of dogs taken during the day may not recognize dogs in nighttime photos.

3. Overfitting

Problem: The model can fit the training data too closely and lose its ability to generalize. It memorizes the examples instead of learning general patterns.

Solution: Use more data, regularization techniques, or cross-validation to detect overfitting.

4. Label quality

Problem: If the labels are wrong (human errors, ambiguity, biases), the model learns those errors. "Garbage in, garbage out."

Example: If in the training data some spam emails are labeled as "not-spam" by mistake, the model will learn incorrectly.

Exercises

Exercise 1: Classify problems

Classify each problem as classification or regression. Justify in one sentence.

  1. Predicting whether a customer will buy a product (yes/no).
  2. Estimating how many units of a product will sell next month.
  3. Detecting whether a transaction is fraudulent (yes/no).
  4. Predicting the price of a used car based on model, year and mileage.
  5. Classifying emails into categories (work, personal, spam).
See solution
  1. Binary classification. The answer is a category (yes/no).
  2. Regression. The answer is a continuous number (e.g. 1,523 units).
  3. Binary classification. The answer is a category (yes/no).
  4. Regression. The answer is a continuous number (a price in $).
  5. Multiclass classification. The answer is a category among several options (work, personal, spam).

Exercise 2: Supervised vs other types

Why are these problems supervised learning? Write the reason in one sentence.

  1. Spam filter.
  2. Image recognition (cat/dog).
  3. House price prediction.
See solution
  1. Spam filter: The model learns from emails labeled as spam or not-spam (data with correct answers).
  2. Image recognition: The model learns from photos labeled "cat" or "dog" (data with correct answers).
  3. Price prediction: The model learns from houses with known prices (data with correct answers = the price).

In every case, the model learns from data with correct answers → supervised learning.

Exercise 3: Identify the label

For each problem, identify what the label is (the correct answer the model must learn to predict).

  1. Classifying emails as spam or not-spam.
  2. Predicting the price of a house.
  3. Recognizing handwritten digits (0-9).
See solution
  1. Label: "spam" or "not-spam" (a category).
  2. Label: A price in $ (a continuous number).
  3. Label: A digit (0, 1, 2, ..., 9) (a category).

Exercise 4: Limitations

What's the main limitation of supervised learning? Write 2-3 sentences.

See answer guide

Possible guide: The main limitation is that you need labeled data, and labeling data is expensive, slow and requires human effort (or automated processes that can introduce errors). Besides, the model only learns from what it saw: if the training data doesn't cover a case, the model won't handle it well. For example, a model trained only with photos of cats indoors may not recognize cats outdoors.

Exercise 5: Why it matters for AI Engineering

Why does an AI Engineer need to understand supervised learning, even if they don't train models from scratch? Write 2 reasons.

See answer guide

Possible guide:

  1. Understanding the model's limits: Knowing that the model learned from labeled data helps you reason about what it can and can't do (e.g. if it was trained with photos of cats and dogs, it won't recognize horses).
  2. Fine-tuning and evaluation: If you need to adjust a pre-trained model with your own data (fine-tuning), you're doing supervised learning (you give it examples with correct answers). Understanding the concept helps you design that process and evaluate the model's quality (accuracy, precision, recall).

Summary

In one sentence: Supervised Learning is the type of ML where the model learns from data with correct answers (labels), in order to then predict those answers on new data.

Key points:

  • Two types: Classification (predicting categories: spam/not-spam, cat/dog) and Regression (predicting numbers: price, temperature).
  • Flow: Labeled data → Train the model → Evaluate with a test set → Use in production (inference).
  • Examples in products: Spam filter, image recognition, price prediction, speech recognition, translation.
  • Limitations: You need labeled data (expensive); the model only learns from what it saw; risk of overfitting; label quality is critical.
  • Why it matters for an AI Engineer: Understanding model limits, fine-tuning, evaluation, communication with product/business.
  • Key point: Without labels, there's no supervised learning. The quality of the labels determines the quality of the model.

Connection with the rest of the module

In the following lessons you'll see other types of ML:

  • Lesson 04 (Unsupervised Learning): ML without labels (the model looks for patterns on its own, e.g. grouping customers into segments).
  • Lesson 05 (Reinforcement Learning): ML with rewards (the agent learns by trial and error, e.g. games).

And the most important lesson:

  • Lesson 06 (Training vs Inference): The distinction between "training a supervised model" (what OpenAI, Google, Meta do) and "using the already-trained model" (what you do when you call an API). That distinction is critical for AI Engineering.

Additional resources

  1. Google ML Crash Course: Supervised Learning — Official introduction to supervised learning with interactive examples. In English.

  2. Coursera: Supervised Machine Learning (Andrew Ng) — The classic course covering classification and regression in detail. In English with subtitles.

  3. Scikit-learn: Supervised Learning — Documentation for the most widely used ML library in Python; it covers classification and regression algorithms. In English.

  4. StatQuest: Logistic Regression — A video explaining a common classification algorithm. In English. Visual and clear.

  5. 3Blue1Brown: Neural Networks (supervised) — A video explaining how neural networks (a type of supervised model) learn. In English. It goes deeper in Module 3.

  6. ImageNet — The reference dataset for supervised learning in vision (14M labeled images). Useful for understanding the scale of data required.

← Previous2. What is Machine Learning?Next →4. Unsupervised Learning: Finding Patterns without Labels