Dot Product & Other Metrics

Dot Product

Formula:

dot(A, B) = Σ(A_i × B_i)

Key insight: If embeddings are normalized (||A|| = ||B|| = 1), then:

dot(A, B) = cosine_similarity(A, B)

Advantage: 3x faster than cosine (no runtime normalization required).

Implementation:

import numpy as np

# Normalize embeddings (do this once)
a = np.array([3, 4])
a_norm = a / np.linalg.norm(a)  # [0.6, 0.8]

b = np.array([4, 3])
b_norm = b / np.linalg.norm(b)  # [0.8, 0.6]

# Dot product (fast)
dot = np.dot(a_norm, b_norm)
print(f"Dot: {dot:.4f}")  # 0.96

# Equivalent to cosine
cos = np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
print(f"Cosine: {cos:.4f}")  # 0.96 (same)

---

Manhattan Distance

Formula:

manhattan(A, B) = Σ|A_i - B_i|

When to use it:

• High dimensionality (curse of dimensionality)
• Faster than Euclidean (no sqrt required)

def manhattan_distance(a, b):
    return np.sum(np.abs(a - b))

a = np.array([1, 2, 3])
b = np.array([4, 5, 6])

dist = manhattan_distance(a, b)
print(f"Manhattan: {dist}")  # 9

---

Hamming Distance

For binary vectors (0/1):

def hamming_distance(a, b):
    return np.sum(a != b)

a = np.array([1, 0, 1, 0])
b = np.array([1, 1, 1, 0])

dist = hamming_distance(a, b)
print(f"Hamming: {dist}")  # 1 (1 bit different)

---

Summary

Metric	Speed	When to use
Dot Product	⚡ Fast	Normalized embeddings
Manhattan	⚡ Fast	High dimensionality
Hamming	⚡ Fast	Binary vectors

---

Module 5 - Capsule 04