7. Capstone Exercise: Mapping Concepts to Vectors

Exercise overview

This is the capstone exercise for Module 1. Here you're going to apply everything you learned: representing concepts as 2D vectors (simplified so you can visualize them), computing operations (addition, subtraction, magnitude), measuring distances, and verifying that "geometric closeness = semantic similarity".

This exercise is 100% conceptual: there's no code to write. You'll work with pencil and paper (or mentally), drawing vectors, computing simple operations, and reflecting on how geometry captures meaning. The goal is for you to internalize the intuition that nearby vectors = similar concepts.

When you finish, you'll have closed the module's conceptual loop: you know what a vector is (Capsule 02), how to visualize it (Capsule 03), how it works in high dimensions (Capsule 04), how to operate on it (Capsule 05), why it's used in AI (Capsule 06), and now you apply it by hand. This intuition is the foundation of Modules 2-8 and of everything ahead of you on your path as an AI Engineer.

Exercise context

Imagine you're designing a semantic search system for a library of concepts. Each concept (word) is represented as a 2D vector (simplified; in reality it would be 1536D). Your task is to:

1. Assign coordinates to 8 concepts
2. Draw the vectors on a Cartesian plane
3. Compute operations (addition, subtraction, distance)
4. Verify that nearby vectors = similar concepts
5. Simulate a semantic search

Important: This exercise uses 2D for simplicity. In reality, embeddings are 1536D, but the intuition is exactly the same.

Part 1: Assigning vectors to concepts

Concepts to map:

1. dog
2. cat
3. tiger
4. car
5. bicycle
6. airplane
7. apple
8. orange

The axes of the 2D space:

To keep it simple, we use 2 conceptual axes:

• X axis (horizontal): "Animalness" → "Mechanicalness"

  • Negative X → Animals / living beings
  • Positive X → Machines / objects

• Y axis (vertical): "Domesticity" → "Speed"

  • Positive Y → Domestic / slow
  • Negative Y → Wild / fast

Note: These axes are arbitrary simplifications used for illustration. In real embeddings (1536D), the axes are learned by the model and don't have clear labels.

Task 1.1: Assign coordinates

For each concept, estimate coordinates [x, y] based on:

• X: How "animal" vs "machine" it is
• Y: How "domestic" vs "fast" it is

Value guide:

• X: -10 (very animal) → 0 (neutral) → +10 (very machine)
• Y: -10 (very wild/fast) → 0 (neutral) → +10 (very domestic/slow)

Example (worked out):

1. dog      = [-8, 7]   (domestic animal)
2. cat      = [-7, 6]   (domestic animal, slightly less so than dog)
3. tiger    = [-9, -5]  (wild animal, fast)
4. car      = [6, -3]   (machine, fast)
5. bicycle  = [5, 4]    (machine, slow, everyday)
6. airplane = [8, -7]   (machine, very fast)
7. apple    = [0, 8]    (neutral animal-machine, everyday/slow object)
8. orange   = [1, 8]    (neutral animal-machine, everyday/slow object)

Note: These coordinates are arbitrary (chosen for illustration). In real embeddings, the model learns optimal coordinates automatically. What matters is that similar concepts have nearby coordinates.

Task 1.2: Draw the vectors

Draw a Cartesian plane on paper and mark the 8 points:

        Y ↑ (Domestic/Slow)
        10|
          |
        8 | •apple  •orange
          |
        6 |  •cat
          |
        4 | •bicycle   •dog
          |
        2 |
          |
        0 |───────────────────────→ X (Animalness → Mechanicalness)
       -2 |        •car
          |
       -4 |
          |
       -6 | •tiger
          |
       -8 |        •airplane
          |
      -10 |
        -10    -5    0    5   10

Expected observations:

• Cluster 1 (Domestic animals): "dog" and "cat" close together (upper-left quadrant)
• Cluster 2 (Vehicles): "car", "bicycle", "airplane" on the right (positive X)
• Cluster 3 (Fruits): "apple" and "orange" very close together (top, center)
• Isolated: "tiger" far from the domestic ones (a wild animal)

Reflection questions:

• Did the similar concepts end up close together on the plane?
• Did the different concepts end up far apart?

Part 2: Computing distances

Task 2.1: Euclidean distance

Compute the Euclidean distance between pairs of vectors:

Formula:

distance([x₁, y₁], [x₂, y₂]) = √((x₂ - x₁)² + (y₂ - y₁)²)

Pairs to compute:

a) distance(dog, cat)
b) distance(dog, tiger)
c) distance(car, bicycle)
d) distance(apple, orange)
e) distance(dog, car)

dog = [-8, 7]
cat = [-7, 6]

distance = √((-7 - (-8))² + (6 - 7)²)
         = √(1² + (-1)²)
         = √(1 + 1)
         = √2 ≈ 1.41

dog = [-8, 7]
tiger = [-9, -5]

distance = √((-9 - (-8))² + (-5 - 7)²)
         = √((-1)² + (-12)²)
         = √(1 + 144)
         = √145 ≈ 12.04

car = [6, -3]
bicycle = [5, 4]

distance = √((5 - 6)² + (4 - (-3))²)
         = √((-1)² + 7²)
         = √(1 + 49)
         = √50 ≈ 7.07

apple = [0, 8]
orange = [1, 8]

distance = √((1 - 0)² + (8 - 8)²)
         = √(1² + 0²)
         = √1 = 1

dog = [-8, 7]
car = [6, -3]

distance = √((6 - (-8))² + (-3 - 7)²)
         = √(14² + (-10)²)
         = √(196 + 100)
         = √296 ≈ 17.20

Task 2.2: Rank by similarity

Given the concept "dog", order the other concepts from closest to farthest (use the distances you computed and estimate the missing ones):

List to order: cat, tiger, car, bicycle, airplane, apple, orange

Part 3: Vector operations

Task 3.1: Vector addition

Compute:

a) dog + cat
b) What concept would you expect to be near that result?

dog = [-8, 7]
cat = [-7, 6]

dog + cat = [-8 + (-7), 7 + 6] = [-15, 13]

Task 3.2: Vector subtraction

Compute:

a) car - bicycle
b) What conceptual "direction" does that subtraction capture?

car = [6, -3]
bicycle = [5, 4]

car - bicycle = [6 - 5, -3 - 4] = [1, -7]

airplane + (car - bicycle) = airplane + [1, -7]
                           = [8, -7] + [1, -7]
                           = [9, -14]

Task 3.3: Scaling

Compute:

a) 2 × apple
b) What does it mean geometrically? Does the concept change?

apple = [0, 8]

2 × apple = [2 × 0, 2 × 8] = [0, 16]

Part 4: Simulating semantic search

Task 4.1: Semantic search

Context: You have a "database" with 8 documents (the 8 concepts from the exercise), each represented by its vector.

The user's query: "domestic animal"

Step 1: Assign a vector to the query "domestic animal"

Guide: The query "domestic animal" should have coordinates close to the "dog" and "cat" cluster.

Proposal: query = [-7.5, 6.5] (roughly the average of dog and cat)

Step 2: Compute the distance from the query to each document:

Use the Euclidean distance formula:

distance(query, doc) = √((doc_x - query_x)² + (doc_y - query_y)²)

Documents:

1. dog = [-8, 7]
2. cat = [-7, 6]
3. tiger = [-9, -5]
4. car = [6, -3]
5. bicycle = [5, 4]
6. airplane = [8, -7]
7. apple = [0, 8]
8. orange = [1, 8]

query = [-7.5, 6.5]

distance(query, dog) = √((-8 - (-7.5))² + (7 - 6.5)²) = √(0.25 + 0.25) = √0.5 ≈ 0.71
distance(query, cat) = √((-7 - (-7.5))² + (6 - 6.5)²) = √(0.25 + 0.25) = √0.5 ≈ 0.71
distance(query, tiger) = √((-9 - (-7.5))² + (-5 - 6.5)²) = √(2.25 + 132.25) ≈ 11.60
distance(query, car) = √((6 - (-7.5))² + (-3 - 6.5)²) = √(182.25 + 90.25) ≈ 16.51
distance(query, bicycle) = √((5 - (-7.5))² + (4 - 6.5)²) = √(156.25 + 6.25) ≈ 12.75
distance(query, airplane) = √((8 - (-7.5))² + (-7 - 6.5)²) = √(240.25 + 182.25) ≈ 20.56
distance(query, apple) = √((0 - (-7.5))² + (8 - 6.5)²) = √(56.25 + 2.25) ≈ 7.65
distance(query, orange) = √((1 - (-7.5))² + (8 - 6.5)²) = √(72.25 + 2.25) ≈ 8.63

Task 4.2: Contrast with keyword search

Question: If you used keyword search (exact word matching), which documents would you return for the query "domestic animal"?

Part 5: Reflection and the connection to RAG

Task 5.1: How would this scale to 1536D?

Question: In this exercise you used 2D (2 axes). In reality, embeddings are 1536D. What changes and what stays the same?

Task 5.2: The connection to RAG

Question: How is what you did in Task 4.1 (semantic search) used in RAG?

Context (from semantic search):
- Doc 1: "The dog is a very loyal domestic animal"
- Doc 2: "The cat is an independent domestic animal"

Query: "Which animals are pets?"

"The animals that are pets include dogs and cats.
Dogs are known for their loyalty, while cats
tend to be more independent."

Part 6: Self-assessment

Final reflection questions:

Answer each one in 1-2 sentences (for yourself; there are no single "correct" answers):

1. Why represent concepts as vectors instead of as strings?

2. What does it mean geometrically for two vectors to be "close"?

3. How does the operation king - man + woman ≈ queen work conceptually?

4. How does semantic search differ from keyword search?

5. Why do real embeddings use 1536D instead of 2D or 3D?

Exercise summary

What you did:

1. ✅ You assigned 2D coordinates to 8 concepts (simulating embeddings)
2. ✅ You drew vectors and verified that similar concepts ended up close together
3. ✅ You computed Euclidean distances (measuring similarity)
4. ✅ You applied operations (addition, subtraction, scaling)
5. ✅ You simulated semantic search (searching for vectors near a query)
6. ✅ You compared semantic search vs keyword search
7. ✅ You reflected on how this scales to 1536D and how it's used in RAG

The key intuition, consolidated:

Nearby vectors = similar concepts. All the magic of semantic search, RAG and embeddings comes down to that geometric intuition. Now that you've lived it by hand in 2D, you understand what happens "under the hood" when you use OpenAI embeddings or Pinecone in 1536D.

Connection to Module 2

In Module 2 (Vector Spaces), you'll extend this intuition:

• Vector space: The "container" where all the vectors live (2D, 1536D, etc.)
• Bases and dimensions: How the axes structure the space
• Subspaces: Regions of the space with special properties (e.g. all the animals)
• Projections: How to reduce 1536D to 2D (t-SNE, UMAP)
• Normalization: Why you make all vectors have magnitude 1

In Module 3, you'll see formal similarity metrics (Euclidean distance, cosine) and when to use each. What you computed by hand in this exercise (Euclidean distance) is one of those metrics.

In Module 4, you'll apply semantic search in real contexts (RAG, document search).

Everything builds on the intuition you consolidated here: vectors in a space, geometric closeness = semantic similarity.

Deliverable (optional)

If you want to consolidate the learning further, save your work:

1. A diagram with the 8 vectors drawn (a photo or a digital drawing)
2. The distance calculations (on paper or in a document)
3. The semantic search ranking (an ordered list)
4. Written reflections (your answers to the 5 self-assessment questions)

It isn't mandatory, but doing it on paper and keeping it will help you remember the intuition when you see embedding code or diagrams in the future.

Extension exercise (optional)

If you want to go deeper:

Extension 1: Add more concepts

Add 3 new concepts to your 2D space:

• "lion"
• "boat"
• "banana"

Assign coordinates, draw them, and verify that they land in the expected regions (lion near tiger, boat near the vehicles, banana near the fruits).

Extension 2: Compute cosine similarity

Instead of Euclidean distance, compute the cosine similarity between "dog" and "cat".

Formula (simplified):

cosine(A, B) = (A · B) / (||A|| × ||B||)

Where:

• A · B = the dot product = a_x × b_x + a_y × b_y
• ||A|| = the magnitude of A = √(a_x² + a_y²)

Compute it:

dog = [-8, 7]
cat = [-7, 6]

Dot product: (-8) × (-7) + 7 × 6 = 56 + 42 = 98
||dog|| = √(64 + 49) = √113 ≈ 10.63
||cat|| = √(49 + 36) = √85 ≈ 9.22

cosine = 98 / (10.63 × 9.22) ≈ 98 / 98.01 ≈ 1.00

Interpretation: A cosine of 1 means "perfectly similar in direction" (a 0° angle). "dog" and "cat" point in nearly identical directions.

You'll see cosine in detail in Module 3.

Extension 3: Simulate a second query

Query: "fast vehicle"

Assign a vector to the query (e.g. [7, -5] — a fast machine), compute the distances, and verify that "car" and "airplane" appear at the top of the ranking.

Additional resources

1. Google Colab: "Visualizing Word Embeddings" — An interactive notebook where you can play with real embeddings and visualize them in 2D with t-SNE. In English.

2. Embedding Projector (TensorFlow) — A web tool for visualizing interactive 3D embeddings. You can rotate, zoom, and search for words. In English.

3. Jay Alammar: "The Illustrated Word2vec" — A blog with diagrams on embedding operations. It complements this exercise with visualizations. In English.

4. Pinecone: "Semantic Search Tutorial" — A hands-on tutorial on how to implement semantic search with embeddings. For after you complete the guide. In English.

5. OpenAI Embeddings Playground — Documentation with code examples for generating embeddings and doing semantic search. In English.

6. 3Blue1Brown: "Dot products and duality" — A 14-minute video on the dot product (the basis of cosine). Excellent visualizations. In English.

Module 1 conclusion

Congratulations on completing Module 1: Introduction to Vectors. 🎉

What you achieved:

1. ✅ You understand what a vector is (direction + magnitude)
2. ✅ You can visualize vectors in 2D and 3D
3. ✅ You understand that high-dimensional vectors (1536D) work the same way
4. ✅ You know how to operate on vectors (addition, subtraction, scaling)
5. ✅ You understand why AI uses vectors (the distributional hypothesis, semantic search)
6. ✅ You applied all of it by hand (the capstone exercise)

The module's key intuition:

Vectors let you represent meaning geometrically. Similar words/concepts → nearby vectors. Semantic search = searching for nearby vectors.

Next step: Module 2 (Vector Spaces) — You'll see how vectors "live" in spaces with structure (bases, dimensions, subspaces), how to project high dimensions to 2D/3D, and how the geometry of the space affects operations and search.

Everything you learned here is the foundation. In Module 2 you go deeper into the geometry; in Module 3 you formalize similarity (cosine, distance); in Module 4 you apply semantic search; in Modules 5-8 you connect it to RAG and AI Engineering.

Keep going! The intuition you consolidated here is 80% of the conceptual understanding of semantic search and RAG. The rest is applying these ideas to more complex contexts.

End of Module 1! 🚀