Module 3: Similarity and Distance
4. Cosine Similarity: The Key Metric for AI
Overview
Cosine similarity measures the angle between two vectors, not the distance. It's the standard metric in semantic search because only the direction matters (the kind of concept), not the magnitude (frequency, length).
Formula
cos(A, B) = (A · B) / (||A|| × ||B||)
Where:
- A · B = the dot product = a₁×b₁ + a₂×b₂ + ... + aₙ×bₙ
- ||A|| = the magnitude of A = √(a₁² + a₂² + ... + aₙ²)
If A and B are normalized (magnitude = 1):
cos(A, B) = A · B (a simple dot product)
2D example
A = [3, 4]
B = [6, 8]
Dot product: 3×6 + 4×8 = 18 + 32 = 50
||A|| = √(9 + 16) = 5
||B|| = √(36 + 64) = 10
cos(A, B) = 50 / (5 × 10) = 50/50 = 1.0
Interpretation: A cosine of 1 → the vectors have the same direction (they're parallel).
Cosine values
cos = 1.0 → Identical (0° angle)
cos = 0.9 → Very similar
cos = 0.5 → Moderately similar
cos = 0.0 → Orthogonal (perpendicular, 90°)
cos = -1.0 → Opposite (180° angle)
Why cosine in AI
Reason 1: It ignores magnitude
[3, 4] (magnitude 5) and [6, 8] (magnitude 10)
→ Same direction → cosine = 1.0
In embeddings, magnitude can be noise. Cosine removes it.
Reason 2: It's robust in high dimensions
In 1536D:
- Euclidean distances → all similar (hard to distinguish)
- Angles (cosine) → more discriminative (easy to distinguish)
Reason 3: Interpretable values
cosine > 0.9 → Very similar (synonyms)
cosine 0.7-0.9 → Similar (related concepts)
cosine < 0.5 → Barely similar
Example in semantic search
Query: "dog" → [0.23, -0.45, ..., -0.34]
Doc 1: "cat" → [0.25, -0.43, ..., -0.32] → cosine = 0.95 ✅
Doc 2: "car" → [9.34, 5.21, ..., 7.56] → cosine = 0.12 ❌
Doc 1 has a high cosine → Relevant.
Comparison: Euclidean vs Cosine
↑
B • | • A (A and B: same direction, different magnitudes)
\|/
•────→
Euclidean: distance(A, B) = large (different magnitudes)
Cosine: cos(A, B) = 1.0 (same direction)
For embeddings: Cosine is better (only direction matters).
Next capsule: 05-metrics-comparison.md — Trade-offs between the metrics.