Cosine Similarity: The Standard Metric

Why cosine is the standard

Cosine similarity measures the angle between vectors, not their magnitude. This is ideal for embeddings, where semantics live in the direction, not in the length of the vector.

Formula:

cos_sim(A, B) = (A · B) / (||A|| × ||B||)

numpy implementation:

import numpy as np

def cosine_similarity(a, b):
    """Cosine similarity between 2 vectors"""
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Example
a = np.array([1, 2, 3])
b = np.array([2, 4, 6])

sim = cosine_similarity(a, b)
print(f"Cosine: {sim:.4f}")  # 1.0 (same direction)

---

Properties

1. Range: [-1, 1]

   • 1.0 = Identical
   • 0.0 = Orthogonal
   • -1.0 = Opposite

2. Scale-invariant:

a = [1, 2, 3]
b = [10, 20, 30]  # 10x larger

cosine_similarity(a, b)  # 1.0 (same direction)

3. Automatic normalization:

# It doesn't matter if embeddings have different magnitudes
# Cosine only considers direction

---

Batch cosine similarity

from sklearn.metrics.pairwise import cosine_similarity

# Multiple vectors
X = np.array([[1, 2, 3], [4, 5, 6]])
Y = np.array([[1, 2, 3], [7, 8, 9]])

# Similarity matrix
sims = cosine_similarity(X, Y)
print(sims)
# [[1.0,    0.9594],
#  [0.9746, 0.9982]]

---

Summary

• ✅ Standard for embeddings
• ✅ Invariant to scale
• ✅ Range: [-1, 1]
• ✅ Use: Semantic search (default)

---

Module 5 - Capsule 02