Module 2: Vector Spaces
7. Capstone Exercise: Designing a Semantic Vector Space
Exercise overview
This is the capstone exercise for Module 2. Here you're going to apply everything: designing a 2D vector space for concepts, identifying subspaces (themed regions), projecting concepts, and reflecting on how it scales to 1536D.
Part 1: Designing the space
Task 1.1: Define the axes
Choose 2 conceptual axes for your 2D space:
Option A:
- X axis: Concrete ↔ Abstract
- Y axis: Positive ↔ Negative (emotional valence)
Option B:
- X axis: Natural ↔ Artificial
- Y axis: Small ↔ Large (size)
Pick one or design your own.
Task 1.2: Place 10 concepts
Assign coordinates [x, y] to these concepts:
- love
- dog
- computer
- sadness
- mountain
- smartphone
- joy
- ant
- building
- tree
Example (with Option A):
love = [-3, 8] (abstract, very positive)
dog = [5, 6] (concrete, positive)
computer = [7, 0] (concrete, neutral)
...
Part 2: Identifying subspaces
Task 2.1: Find clusters
Which concepts are close together? Group them:
Example:
- Cluster 1 (Emotions): love, sadness, joy
- Cluster 2 (Technology): computer, smartphone
- Cluster 3 (Nature): mountain, tree, dog, ant
Task 2.2: Define a subspace
Identify a "region" of the space that is a subspace.
Example: All the concepts with Y > 5 (positive concepts) form a themed subspace.
Part 3: Vector operations
Task 3.1: Compute a sum
Vector("dog") + Vector("joy") = ?
What concept would you expect to be near that result?
Task 3.2: Semantic direction
Vector("building") - Vector("ant") = ?
What "direction" does that subtraction capture? (size: small → large)
Part 4: Conceptual projection
Task 4.1: Add a 3rd dimension
Imagine you add a Z axis: "Living ↔ Inanimate".
How would the coordinates change for:
- dog
- computer
- tree
Task 4.2: Project 3D → 2D
If you remove the Z axis (projecting to 2D), what information do you lose?
Part 5: Reflecting on 1536D
Question 1:
If your 2D space captures 2 aspects (concrete-abstract, positive-negative), which 10 aspects would you capture in a 10D space for concepts?
Question 2:
Why does 1536D let you distinguish "large dog" from "small dog" better than 2D?
Question 3:
How would you use this space for semantic search? (Describe the process: query → vector → search for nearby ones)
Exercise summary
What you did:
- ✅ You designed a 2D vector space with conceptual axes
- ✅ You placed 10 concepts as vectors
- ✅ You identified clusters (themed subspaces)
- ✅ You computed operations (addition, subtraction = directions)
- ✅ You reflected on projections (3D → 2D)
- ✅ You connected it to 1536D (how it scales)
The intuition, consolidated:
"A vector space isn't a random set of points. It's a structured universe where regions capture topics (clusters), directions capture relationships (gender, size), and geometry reflects meaning."
Module 2 conclusion
Congratulations on completing Module 2: Vector Spaces. 🎉
What you achieved:
- ✅ You understand what a vector space is (a structured universe)
- ✅ You know what bases and dimensions are (axes, 1536D)
- ✅ You can identify subspaces (themed regions)
- ✅ You understand projections (t-SNE, UMAP conceptually)
- ✅ You know why you normalize (it removes magnitude noise)
- ✅ You see how LLMs organize meaning (clusters, directions)
- ✅ You applied all of it conceptually
The module's key intuition:
The embedding space is a structured geometric universe. Semantic search navigates that universe using distances and angles.
Next module: Module 3: Similarity and Distance — Formalizing how to measure closeness (Euclidean vs cosine).