Performance Benchmarks

Benchmark Setup

import numpy as np
import time

# Generate random embeddings
n = 10000
dim = 1536
embeddings = np.random.randn(n, dim)
query = np.random.randn(dim)

# Normalize (for a fair comparison)
embeddings_norm = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True)
query_norm = query / np.linalg.norm(query)

---

Speed Comparison

# Cosine similarity
start = time.time()
cos_sims = np.dot(embeddings, query) / (np.linalg.norm(embeddings, axis=1) * np.linalg.norm(query))
time_cosine = time.time() - start

# Dot product (normalized embeddings)
start = time.time()
dot_sims = np.dot(embeddings_norm, query_norm)
time_dot = time.time() - start

# Euclidean
start = time.time()
euc_dists = np.linalg.norm(embeddings - query, axis=1)
time_euclidean = time.time() - start

print(f"Cosine: {time_cosine:.4f}s")
print(f"Dot Product: {time_dot:.4f}s")
print(f"Euclidean: {time_euclidean:.4f}s")
print(f"\nSpeedup (Dot vs Cosine): {time_cosine/time_dot:.1f}x")

Typical output (timings vary by machine):

Cosine: 0.0234s
Dot Product: 0.0078s
Euclidean: 0.0156s

Speedup (Dot vs Cosine): 3.0x

---

Key Insights

1. Dot product is faster than cosine (no runtime normalization)
2. Pre-normalizing embeddings → use dot product
3. Exact timings depend on hardware, BLAS, and memory bandwidth

---

Production Pattern

class FastVectorSearch:
    """Optimized search with dot product"""
    
    def __init__(self, embeddings):
        # Normalize once (upfront cost)
        self.embeddings = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True)
    
    def search(self, query, k=5):
        # Normalize the query
        query_norm = query / np.linalg.norm(query)
        
        # Dot product (fast!)
        scores = np.dot(self.embeddings, query_norm)
        
        # Top-K
        top_k = np.argsort(scores)[::-1][:k]
        return top_k, scores[top_k]

---

Summary

• ✅ Dot product: faster (no runtime normalization)
• ✅ Pre-normalize: Upfront cost, runtime savings
• ✅ Production: Always use dot product when you can

---

Module 5 - Capsule 05