Dimensionality Reduction

Why reduce dimensions

# OpenAI embeddings: 1536 dims
# Problem: Hard to visualize

# Solution: Reduce to 2-3 dims for visualization
# WARNING: For visualization only, NOT for production

---

PCA (Principal Component Analysis)

from sklearn.decomposition import PCA
import matplotlib.pyplot as plt

# Embeddings (N × 1536)
embeddings = np.array([...])

# Reduce to 2D
pca = PCA(n_components=2)
embeddings_2d = pca.fit_transform(embeddings)

# Plot
plt.scatter(embeddings_2d[:, 0], embeddings_2d[:, 1])
plt.xlabel('PC1')
plt.ylabel('PC2')
plt.title('Embeddings in 2D (PCA)')
plt.show()

---

UMAP (better than PCA)

import umap

# UMAP preserves local structure better than PCA
reducer = umap.UMAP(n_components=2)
embeddings_2d = reducer.fit_transform(embeddings)

# Plot (clearer clusters than PCA)
plt.scatter(embeddings_2d[:, 0], embeddings_2d[:, 1])

---

Limitations

❌ Do NOT use for production:

• Information loss (~90%)
• For visualization/exploration only

✅ Use for:

• Visualizing clusters
• Debugging (spotting outliers)
• Presentations

---

Summary

• ✅ PCA: Fast, linear
• ✅ UMAP: Better structure preservation
• ❌ Production: Do NOT reduce dims (info loss)
• ✅ Visualization: OK

---

Module 6 - Capsule 05