Module 2: Vector Spaces
3. Bases and Dimensions: The Axes of the Universe
Capsule overview
In this capsule you're going to understand what bases are (the "axes" of the space) and what dimensionality means (how many axes there are). If the vector space is the "universe" where vectors live, the bases are the "coordinate system" you use to locate each point.
When you finish, you'll understand why OpenAI's embedding space has exactly 1536 dimensions (1536 independent axes), what "linear independence" means without heavy algebra, and why more dimensions = more capacity to capture nuance.
What is a basis?
A basis is a set of vectors that:
- Are independent (none can be built as a combination of the others)
- Span the whole space (any vector can be expressed as a combination of them)
Analogy: The X and Y axes in 2D are a basis. Any point [3, 2] is expressed as "3 steps along X, 2 steps along Y".
Standard bases in 2D and 3D
The standard basis in 2D:
e₁ = [1, 0] (the pure X axis)
e₂ = [0, 1] (the pure Y axis)
Any vector is expressed as a combination:
[3, 2] = 3×[1,0] + 2×[0,1] = 3×e₁ + 2×e₂
The standard basis in 3D:
e₁ = [1, 0, 0] (the X axis)
e₂ = [0, 1, 0] (the Y axis)
e₃ = [0, 0, 1] (the Z axis)
Dimensionality: How many axes?
The dimension of a space is the number of vectors in a basis:
- 2D: 2 basis vectors → 2 dimensions
- 3D: 3 basis vectors → 3 dimensions
- 1536D: 1536 basis vectors → 1536 dimensions
In OpenAI embeddings (1536D):
- There are 1536 independent "pure directions"
- Each embedding is a combination of those 1536 directions
- More dimensions = more capacity to distinguish concepts
Linear independence (a simple concept)
Vectors are independent if none of them is a combination of the others.
2D example - Independent:
v₁ = [1, 0]
v₂ = [0, 1]
You can't build v₂ as a multiple of v₁ → Independent ✅
2D example - Dependent:
v₁ = [1, 0]
v₂ = [2, 0]
v₂ = 2×v₁ → Dependent ❌ (v₂ contributes no new direction)
Why more dimensions = more capacity
In 2D: You can only capture 2 aspects (e.g. size, speed)
In 1536D: You can capture 1536 aspects simultaneously:
- Axis 1: Animalness
- Axis 2: Size
- Axis 3: Speed
- ...
- Axis 1536: An abstract aspect
Result: Subtly different concepts (a large dog vs a small dog) have distinguishable positions.
Summary
Key points:
- Basis: A set of independent vectors that span the space
- Dimension: The number of vectors in the basis (2D → 2 axes, 1536D → 1536 axes)
- Independence: No basis vector can be built as a combination of the others
- More dimensions: More capacity to capture nuance
Next capsule: 04-subspaces-and-projections.md — Regions of the space, dimensionality reduction (t-SNE, UMAP).