Module 3: Similarity and Distance
1. Module Introduction: Similarity and Distance - Measuring Closeness in the Space
Overview
This is the module where you formalize how to measure closeness between vectors. So far you know what vectors are (Module 1) and how they form structured spaces (Module 2). Now you'll see how to quantify "how close" two vectors are, what "similarity" means, and when to use each metric.
This module is the mathematical heart of semantic search: "searching for nearby vectors" requires a precise definition of "nearby". You'll see three main metrics (Euclidean, Manhattan, cosine) and you'll learn that cosine is superior in high dimensions for embeddings.
Approach: 60% conceptual theory with simple formulas, 40% numerical exercises. There's NO code, but there are manual calculations (a calculator is fine). When you finish, you'll understand why almost every semantic search system uses cosine similarity.
Estimated time: 2-2.5 hours
Module Objectives
By completing this module, you'll be able to:
- Compute Euclidean distance (a straight line between points)
- Compute Manhattan distance (a path along a grid)
- Compute cosine similarity (the angle between vectors)
- Compare metrics (when to use each one)
- Justify why cosine is better in high dimensions
- Apply metrics to semantic search (measuring relevance)
- Interpret similarity values (what 0.85 vs 0.92 means)
Key competency gained: You'll understand that semantic search isn't just "searching for nearby vectors" but "searching for vectors with high cosine similarity" (similar direction, magnitude irrelevant).
Module roadmap: the 7 capsules
| # | Capsule | What you'll see | Duration |
|---|---|---|---|
| 01 | Module introduction | Why measure similarity, an overview of the metrics | 15 min |
| 02 | Euclidean distance | Straight line, formula, 2D/3D/1536D examples | 20 min |
| 03 | Manhattan and others | Manhattan distance, Minkowski, use cases | 20 min |
| 04 | Cosine similarity | The angle between vectors, why it's used in AI | 30 min |
| 05 | Comparing metrics | Euclidean vs cosine, trade-offs | 25 min |
| 06 | Metrics in semantic search | How they're used in production | 20 min |
| 07 | Capstone exercise | Compute metrics, compare, decide | 30 min |
Total: ~2.5 hours
Connection to Modules 1-2
Module 1: Individual vectors (what they are, operations)
Module 2: Vector spaces (structure, subspaces)
Module 3: Measuring closeness (distance, similarity)
The logical flow:
- You have vectors (Module 1) ✅
- They live in a structured space (Module 2) ✅
- How do you know whether two vectors are "close"? (Module 3) ← You are here
The central problem
In semantic search:
Query: "domestic animal"
Documents:
- Doc 1: Vector [0.23, -0.45, ..., -0.34]
- Doc 2: Vector [0.25, -0.43, ..., -0.32]
- Doc 3: Vector [9.34, 5.21, ..., 7.56]
Question: Which document is more relevant (the "closest" to the query)?
You need a closeness metric to answer that question quantitatively.
The three main metrics
1. Euclidean distance
What it measures: A straight line between two points.
When to use it: When magnitude matters (e.g. physical distances).
In AI: Used, but not optimal in high dimensions (all the distances become similar).
2. Manhattan distance
What it measures: A path along a grid (the sum of the differences on each axis).
When to use it: When movement is only horizontal/vertical (e.g. city streets).
In AI: Rare with embeddings (Euclidean or cosine are more common).
3. Cosine similarity
What it measures: The angle between vectors (direction, not magnitude).
When to use it: When only direction matters (e.g. normalized embeddings).
In AI: The standard metric for semantic search. Almost every system uses cosine.
Why cosine is superior in high dimensions
The problem with Euclidean in high dimensions:
In 1536D, all the distances look similar (the curse of dimensionality). It's hard to tell "close" from "far".
The advantage of cosine:
Cosine measures the angle (direction), not the distance. In high dimensions, angles are preserved better than distances.
Result: Cosine gives more discriminative values (0.92 vs 0.45 is a clear difference).
Visualizing the concept
In 2D (simplified):
↑
|
•B | •A (A and B have similar directions, high cosine)
\|/
•────→
|
|
•C (C has a very different direction, low cosine)
Euclidean distance: A and C can be at similar distances from the origin.
Cosine similarity: A and B have a high cosine (a small angle). A and C have a low cosine (a large angle).
Direct application to semantic search
Query: "dog"
Doc 1: "The dog is a domestic animal" → Nearby vector (cosine 0.95)
Doc 2: "The hound is loyal" → Nearby vector (cosine 0.93)
Doc 3: "Cars have wheels" → Distant vector (cosine 0.12)
Ranking by cosine:
- Doc 1 (0.95) — Most relevant
- Doc 2 (0.93) — Very relevant
- Doc 3 (0.12) — Not relevant
The system returns Doc 1 and Doc 2.
What you will NOT see in this module
To keep the focus conceptual:
- ❌ Python/NumPy code: You won't implement the metrics (that's for later guides)
- ❌ Optimizations: You won't see SIMD, GPU, approximations (that's Module 4)
- ❌ Exotic metrics: Only Euclidean, Manhattan, cosine (the 3 most used)
Success criteria
You'll know you completed the module if you can:
- ✅ Compute the Euclidean distance between two 2D/3D vectors
- ✅ Compute the cosine similarity between two 2D/3D vectors
- ✅ Explain why cosine is better than Euclidean in high dimensions
- ✅ Interpret cosine values (0.95 = very similar, 0.3 = barely similar)
- ✅ Decide which metric to use in a given use case
- ✅ Apply metrics to semantic search (query → vectors → rank by cosine)
Quick test: If you can explain "Why does semantic search use cosine instead of Euclidean?" without hesitating, you're ready for Module 4.
Upcoming capsules
Capsule 02: Euclidean distance (formula, examples, when to use it)
Capsule 03: Manhattan and other metrics
Capsule 04: Cosine similarity (the key metric for AI)
Capsule 05: Comparison (Euclidean vs cosine)
Capsule 06: Metrics in real semantic search
Capsule 07: Capstone exercise
Next capsule: 02-euclidean-distance.md — Straight line, formula, calculations.