Module 2: Vector Spaces

4. Subspaces and Projections: Regions and Visualization

Capsule overview

In this capsule you're going to understand subspaces (regions of the space with special properties) and projections (how to reduce 1536D to 2D for visualization). These concepts are fundamental for interpreting embedding visualizations and understanding semantic clustering.

What is a subspace?

A subspace is a "region" of the space that is, in itself, a vector space.

Analogy: If 3D space is a room, a subspace can be:

A wall (a 2D space inside 3D)
The floor (a 2D space inside 3D)
A straight line (a 1D space inside 3D)

Examples of subspaces

In 2D:

A 1D subspace (a line):

All the vectors of the form [t, t] where t is any number

Examples: [1,1], [2,2], [-3,-3], [0,0]

Visualization:

      ↑ Y
      |
    3 |     •[3,3]
      |    ╱
    2 |   •[2,2]
      |  ╱
    1 | •[1,1]
      |╱
    0 •─────────→ X
      0  1  2  3

It's a subspace because:

If you add two vectors of the form [t,t], the result is still of the form [t,t] ✅
If you scale [t,t], the result is still [s×t, s×t] ✅

In 3D:

A 2D subspace (a plane):

All the vectors of the form [x, y, 0] (the XY plane)

Examples: [2,3,0], [-1,5,0], [0,0,0]

It's a subspace (the XY plane inside 3D space).

Subspaces in semantic spaces

In the 1536D embedding space, there are semantic subspaces:

The "animals" subspace:

All the vectors of words related to animals
"dog", "cat", "lion", "tiger" are in this region

The "vehicles" subspace:

All the vectors of words related to transportation
"car", "airplane", "boat" are in this region

Use in AI: Clustering exploits subspaces. If you search for "animal", you find vectors in the animal subspace.

Projections: From 1536D to 2D

A projection is a transformation that reduces dimensionality:

1536D vector → Projection → 2D vector

Goal: Visualizing high dimensions in 2D/3D (which is drawable).

Projection techniques

1. PCA (Principal Component Analysis)

What it does: Finds the 2-3 most important axes and projects onto them.

Advantage: Fast, deterministic.

Disadvantage: Linear (it can lose non-linear structure).

2. t-SNE (t-Distributed Stochastic Neighbor Embedding)

What it does: Projects while preserving local closeness (points close together in 1536D stay close in 2D).

Advantage: It preserves clusters (groupings).

Disadvantage: It doesn't preserve global distances (distant points can end up close together).

3. UMAP (Uniform Manifold Approximation and Projection)

What it does: Similar to t-SNE but faster, and it preserves more global structure.

Advantage: A balance between speed and quality.

Disadvantage: It isn't perfect (it's still an approximation).

What is lost and what is preserved

In a 1536D → 2D projection:

Preserved (approximately):

Local closeness (nearby points stay nearby)
Clusters (groupings)
The intuition of "similarity"

Lost:

Exact distances
The information from 1534 dimensions
Complex global relationships

Conclusion: Projections are useful for visualizing intuition, not for computing mathematically.

Interpreting visualizations

When you see a t-SNE diagram of embeddings:

What you see:

Clusters (animals together, vehicles together)
Relative closeness

What you do NOT see:

Exact distances (1536D compressed into 2D)
The complete structure of the original space

How to interpret it:

Points close together in 2D → probably close together in 1536D ✅
Points far apart in 2D → far apart in 1536D? Maybe ⚠️

Summary

Key points:

Subspace: A region of the space that is, in itself, a vector space
Semantic subspaces: Themed regions (animals, vehicles, emotions)
Projection: Reducing dimensionality (1536D → 2D) to visualize
Techniques: PCA, t-SNE, UMAP (each with trade-offs)
Preserves: Local closeness, clusters, intuition
Loses: Exact distances, the information from the removed dimensions

Next capsule: 05-normalization-and-transformations.md — Why magnitude = 1, operations that preserve meaning.

← Previous3. Bases and Dimensions: The Axes of the Universe Next →5. Normalization and Transformations