2. What Is a Vector?

Capsule overview

In this capsule you're going to build the fundamental definition: what a vector is. We're not going to bury you in formulas or formal linear algebra; we're going to give you the clear geometric intuition: a vector is an entity that has direction and magnitude, and is represented by numerical coordinates.

This is the foundation of everything. Without understanding what a vector is, you won't be able to understand vector spaces (Module 2), similarity (Module 3), search (Module 4), embeddings (Guide #6) or RAG (Module 7). Here we build the fundamental block.

When you finish this capsule, you'll be able to explain what a vector is to a non-technical person using real-world analogies (arrows, displacements, coordinates on a map), and you'll understand that [3, 2] isn't just "three and two" but a point in space with geometric meaning.

What is a vector? Intuitive definition

A vector is a mathematical entity that represents:

1. A direction (where it points)
2. A magnitude (how far or how "strong")

You can think of it as:

• An arrow in space (it points in a direction, it has a length)
• A displacement from one point to another (moving from the origin to a position)
• A point in space with coordinates (a location relative to the origin)

Real-world example:

Imagine you're in a city with a grid of streets. You tell someone: "Walk 3 blocks east and 2 blocks north". That's a vector: it has direction (northeast, a combination of east + north) and magnitude (how far to walk).

In mathematics, that displacement is written as:

Vector v = [3, 2]

Where:

• 3 = coordinate on the X axis (horizontal, east)
• 2 = coordinate on the Y axis (vertical, north)

Visual representation: The arrow

The most intuitive way to think about vectors is as arrows:

      ↑ Y
      |
    2 • ────→ (3, 2)
      |    /
    1 |   /
      |  /
    0 |─────────→ X
      0  1  2  3

The arrow goes from the origin (0, 0) to the point (3, 2).

Components of the vector:

• Direction: Northeast (a combination of +X and +Y)
• Magnitude: The length of the arrow (the distance from the origin to the point)
• Coordinates: [3, 2] — the numerical representation

Analogy: If the origin is your house and the point (3, 2) is a store, the vector [3, 2] represents "how to get there": walk 3 blocks east, 2 north. The arrow shows you the direct route (direction + distance).

Direction: Where does the vector point?

The direction of a vector is "where it points", regardless of how long it is.

Examples:

Vector A = [3, 2]     → Points northeast
Vector B = [6, 4]     → Points northeast (same direction as A, twice as long)
Vector C = [-3, -2]   → Points southwest (opposite direction to A)
Vector D = [2, 3]     → Points northeast, but at a different angle than A

Two vectors have the same direction if one is a multiple of the other:

• [3, 2] and [6, 4] point in the same direction (the second is 2× the first)
• [3, 2] and [2, 3] do NOT have the same direction (different angles)

Why it matters in AI: When two words have related meaning (e.g. "king" and "monarch"), their vectors (embeddings) point in similar directions. Direction captures "what kind of concept it is"; magnitude can vary.

Magnitude: How "long" is the vector?

The magnitude of a vector is its "length" or "size", regardless of direction.

Conceptual formula (no need to memorize it):

For a 2D vector [x, y], the magnitude is the distance from the origin:

Magnitude = √(x² + y²)

Example:

Vector [3, 2]
Magnitude = √(3² + 2²) = √(9 + 4) = √13 ≈ 3.6

Interpretation: If you walk 3 blocks east and 2 north, you've covered roughly 3.6 blocks in a straight line from where you started.

Visualization:

      ↑
    2 • ────→ (3, 2)
      | ╱
    1 |╱  ← Magnitude (hypotenuse)
      |
    0 •─────────→
      0  1  2  3

The magnitude is the length of the arrow (the hypotenuse of the triangle).

Why it matters in AI: In semantic search, we sometimes normalize vectors so they all have magnitude 1 (only direction matters). In other cases, magnitude can capture "intensity" or "frequency". Don't worry about the details now; just understand that magnitude = the length of the arrow.

Numerical representation: Coordinates

A vector is represented with coordinates: numbers that indicate a position on each axis.

In 2D (2 dimensions):

Vector v = [x, y]

Example: [3, 2]
- x = 3 (coordinate on the X axis)
- y = 2 (coordinate on the Y axis)

In 3D (3 dimensions):

Vector v = [x, y, z]

Example: [3, 2, -1]
- x = 3 (coordinate on the X axis)
- y = 2 (coordinate on the Y axis)
- z = -1 (coordinate on the Z axis)

In N dimensions:

Vector v = [x₁, x₂, x₃, ..., xₙ]

Example in 5D: [3, 2, -1, 0, 4]

Interpretation: Each number is the position on one axis. In 2D you have 2 axes (X, Y); in 3D you have 3 axes (X, Y, Z); in 1536D you have 1536 axes (you can't draw them, but mathematically they work the same).

Why it matters in AI: The embeddings you'll see in Guide #6 are high-dimensional vectors. For example, OpenAI's text-embedding-3-small embedding has 1536 dimensions: it's an array of 1536 numbers. Each number is the coordinate on one axis of the 1536D space. You can't visualize it, but conceptually it's the same as [3, 2] in 2D; just with many more axes.

Vector as an arrow vs vector as a point

There are two equivalent ways to think about vectors:

1. Vector as an arrow (displacement)

The arrow goes from the origin (0, 0) to the point (x, y).

      ↑
    2 • ────→ Destination point (3, 2)
      | ╱
    1 |╱  ← Arrow (vector)
      |
    0 •─────────→ Origin (0, 0)
      0  1  2  3

Interpretation: "Move 3 units along X, 2 units along Y"

When it's useful: For thinking about operations (addition, subtraction), about displacements, about motion.

2. Vector as a point (position)

The vector [3, 2] can also be thought of as a point in space: the location you arrive at if you apply the displacement.

      ↑
    2 •(3,2) ← Point in space
      |
    1 |
      |
    0 •─────────→
      0  1  2  3

Interpretation: "A point located at x=3, y=2"

When it's useful: For thinking about embeddings (each word is a point in space), about vector spaces, about closeness (nearby points = similar concepts).

Both perspectives are correct and useful:

• Arrow → Emphasizes direction and magnitude
• Point → Emphasizes position and closeness

In AI, we usually think of embeddings as points in high-dimensional space (similar words are nearby points). But the arrow intuition helps for understanding operations (adding vectors = combining arrows).

Concrete example: The vector [3, 2]

Let's fully analyze the vector [3, 2]:

Coordinates: x = 3, y = 2

Direction: Northeast (an angle of roughly 33.7° from the X axis)

Magnitude: √(3² + 2²) = √13 ≈ 3.6

As an arrow: An arrow from (0, 0) to (3, 2)

As a point: A point located at x=3, y=2 on the plane

Visualization:

Y ↑
  3 |
  2 |        •(3,2) ← Destination point
  1 |       ╱
  0 |•─────────────→ X
    0  1  2  3  4
    ↑
    Origin

Practical interpretation: If you're at the origin and you walk 3 steps east and 2 north, you arrive at the point (3, 2). The vector [3, 2] captures that entire displacement: direction (northeast) and distance (3.6 steps in a straight line).

Vector vs number (scalar)

It's important to distinguish a vector from a simple number (called a "scalar"):

Why it matters:

• A scalar has no direction (the number 5 doesn't "point" anywhere)
• A vector does have direction (the vector [3, 2] points northeast)

Practical example:

• Temperature = scalar (just a number: 25°C)
• Wind = vector (direction: northeast, magnitude: 15 km/h)

In AI:

• A word represented as an embedding = vector (e.g. [0.5, -0.3, 0.8, ...])
• The price of a product = scalar (just a number: $19.99)

Vectors in different dimensions

Vectors can have any number of dimensions:

1D (one dimension):

Vector: [5]

It's simply a number on a line. It isn't usually called a "vector" in geometry (it's just a point on a line), but mathematically it is a 1D vector.

2D (two dimensions):

Vector: [3, 2]

A point on a plane (x, y). You can draw it on paper easily.

3D (three dimensions):

Vector: [3, 2, -1]

A point in space (x, y, z). You can picture it (an object in a room: horizontal position, vertical position and depth).

High dimensions (100D, 1536D, 3072D):

Vector: [0.23, -0.45, 0.12, 0.78, ..., -0.34]  (1536 numbers)

You can't draw it, but mathematically it works the same: each number is the coordinate on one axis. In AI, text embeddings are high-dimensional vectors.

Why high dimensions: More dimensions = more capacity to capture nuance and relationships. A 2D vector can only capture 2 aspects; a 1536D vector can capture 1536 different aspects of meaning.

We'll look at this in detail in Capsule 04 (Vectors in high dimensions). For now, understand that the concept is the same: coordinates on axes, direction and magnitude. Just with many more axes.

Real-world analogies

To lock in the intuition, here are some useful analogies:

Analogy 1: GPS and maps

When you use GPS, your location is represented with coordinates:

Latitude: 19.4326  (North-South)
Longitude: -99.1332  (East-West)

That's a 2D vector: [19.4326, -99.1332]

Direction: Where you are relative to the reference point (the Equator and the Greenwich meridian)

Magnitude: How far you are from the origin (0, 0)

Analogy 2: Movement instructions

"Walk 5 steps forward, then 3 steps to the right" = Vector [5, 3]

Direction: A combination of forward + right (diagonal)

Magnitude: Total straight-line distance ≈ 5.8 steps

Analogy 3: Wind

Wind has:

• Direction: Northeast, southwest, etc.
• Magnitude: Speed (15 km/h)

That can be represented as a vector: direction + intensity.

Analogy 4: Force in physics

If you push a box:

• Direction: Where you're pushing
• Magnitude: How hard

That's a force vector: direction + intensity.

In AI: A word like "dog" is represented as a vector (an embedding). The vector has:

• Direction: What "kind of concept" it is (animal, pet, etc.)
• Magnitude: How "present" or "frequent" it is in the training corpus

Similar words ("dog", "cat") have vectors with similar directions (nearby points in space).

Vectors as arrays of numbers

In programming, a vector is represented as an array (a list of numbers):

Python (illustrative example, you will NOT implement it here):

# 2D vector
v1 = [3, 2]

# 3D vector
v2 = [3, 2, -1]

# 5D vector
v3 = [0.5, -0.3, 0.8, 0.1, -0.2]

# High-dimensional vector (1536D) - embedding example
embedding = [0.234, -0.456, 0.123, ..., -0.789]  # 1536 numbers

Important: Even though they look like simple lists of numbers, every vector has a geometric interpretation:

• Each number = a coordinate on an axis
• The full set = a point in N-dimensional space
• It has direction and magnitude

It isn't just "an array of numbers". It's a representation of position, direction and magnitude in a mathematical space.

Fundamental properties of vectors

1. Vectors have an origin

By convention, the origin is (0, 0) in 2D, (0, 0, 0) in 3D, etc.

The vector [3, 2] goes from the origin to the point (3, 2).

2. Vectors can be scaled

Multiplying a vector by a number (a scalar) changes its magnitude but not its direction:

Original vector: [3, 2]
Scalar: 2

Scaled vector: [6, 4]  (same direction, twice as long)

3. Vectors can be added

Adding two vectors combines their displacements:

Vector A: [3, 2]   (3 east, 2 north)
Vector B: [1, 1]   (1 east, 1 north)

A + B = [4, 3]     (4 east, 3 north — the combined total)

We'll look at operations in detail in Capsule 05.

4. Vectors can be subtracted

Subtracting vectors gives you the "difference" or "displacement between points":

Vector A: [5, 3]
Vector B: [2, 1]

A - B = [3, 2]  (the difference: from B to A)

Also in Capsule 05.

The zero vector: The origin

The zero vector is special:

Zero vector in 2D: [0, 0]
Zero vector in 3D: [0, 0, 0]
Zero vector in ND: [0, 0, ..., 0]

Properties:

• It has no direction (or its direction is undefined)
• Magnitude = 0 (it has no length)
• It's the reference point (the origin)

In AI: The zero vector almost never appears in real embeddings (models learn non-zero representations), but conceptually it's important as a reference point.

Negative vectors

Coordinates can be negative:

Vector A: [3, 2]    (northeast)
Vector B: [-3, -2]  (southwest — opposite to A)
Vector C: [3, -2]   (southeast)
Vector D: [-3, 2]   (northwest)

Visualization:

      ↑ Y
      |
    2 | D(-3,2) •       • A(3,2)
      |         |       |
    0 |─────────•───────────→ X
      |         |       |
   -2 | C(3,-2) •       • B(-3,-2)
      |

Interpretation: Negative coordinates mean "in the opposite direction along the axis":

• x < 0 → West (left)
• y < 0 → South (down)
• z < 0 → Back (if Z is depth)

In AI: Embeddings can have negative coordinates. For example, a 1536D embedding can be [0.23, -0.45, 0.12, -0.78, ...]. Negative doesn't mean "bad"; it only means a position on the axis (it can be on the negative side of the axis, like "west" vs "east").

Mathematical notation (optional)

In formal linear algebra, vectors are written with special notation:

Column vector:

    ⎡ 3 ⎤
v = ⎢ 2 ⎥
    ⎣   ⎦

Row vector:

v = [3  2]

In this guide we use simple notation:

v = [3, 2]

It's more readable and conceptually equivalent. Don't worry about the formal notation; just understand that a vector is an ordered set of numbers with a geometric interpretation.

Visual summary: Everything in one diagram

           Y ↑
             |
           2 |        •(3,2) ← Point (vector as position)
             |       ╱|
           1 |      ╱ |
             |     ╱  | y=2
           0 |•───────────→ X
             0   x=3
             ↑
             Origin

Elements:
- Origin: (0, 0)
- Destination point: (3, 2)
- Arrow: Vector [3, 2]
- Coordinates: x=3, y=2
- Direction: Northeast (~33.7°)
- Magnitude: √13 ≈ 3.6 units

Exercises

Exercise 1: Identify the components

Given the vector [4, -2]:

a) What are the coordinates?
b) Which direction does it point? (northeast, southwest, etc.)
c) Is the magnitude greater or less than 5?

  Y ↑
    |
  0 |•───────→ X
    |     ╲
 -2 |      •(4,-2)
    0  1  2  3  4

Exercise 2: Vector vs scalar

Which of these are vectors and which are scalars?

a) [5]
b) 5
c) [0, 0]
d) [-3, 2, 1]
e) 3.14

Exercise 3: Draw vectors

Draw on paper (or mentally) these 2D vectors:

a) [2, 3]
b) [-2, 1]
c) [0, -2]

For each one:

• Mark the origin (0, 0)
• Draw the arrow from the origin to the point
• Identify the direction (northeast, southwest, etc.)

  3 •(2,3) ← Northeast
    |╱
  2 |
    |
  1 |
    |
  0 •───────→
    0  1  2  3

  1 •(-2,1) ← Northwest
    |╲
  0 •───────→
    -2 -1  0  1

  0 •───────→
    |
 -1 |
    |
 -2 •(0,-2) ← South (straight down)

Exercise 4: Comparing magnitudes

Without calculating exactly, order these vectors from smallest to largest magnitude:

a) [1, 1]
b) [3, 4]
c) [0, 2]
d) [-5, 0]

Exercise 5: Same direction

Which of these vectors have the same direction as [2, 1]?

a) [4, 2]
b) [2, 2]
c) [6, 3]
d) [-2, -1]

Exercise 6: Interpretation in AI

An embedding of the word "dog" is the vector:

[0.5, -0.3, 0.8, 0.1, -0.2]  (simplified to 5D for illustration)

a) What does each number represent?
b) Why is it useful to represent "dog" as a vector instead of just as the word "dog"?
c) What does it mean that the embedding of "cat" is [0.52, -0.28, 0.75, 0.12, -0.18]?

Conceptual troubleshooting

Problem 1: "I don't understand what the 'direction' of a vector is"

Symptom: The concept of "direction" feels abstract to you.

Solution: Think of a physical arrow: it points somewhere (north, northeast, up). That's direction. In 2D, direction is measured with an angle (0° = east, 90° = north, etc.). But you don't need to calculate angles; just understand that [3, 2] points "northeast" and [-3, -2] points "southwest" (the opposite).

Analogy: If someone tells you "walk northeast", that's a direction. It doesn't matter whether you walk 1 meter or 100 meters; the direction (northeast) is the same.

Problem 2: "Why does the vector [3, 2] start at (0, 0)?"

Symptom: You don't understand why the vector goes from the origin.

Solution: By convention, vectors are represented starting from the origin (0, 0). That doesn't mean only vectors from the origin exist; it means any displacement can be represented as a vector from the origin to a point. If you want to represent "going from A to B", you calculate B - A and that gives you a vector.

Example: To go from (1, 1) to (4, 3):

• Displacement: [4-1, 3-1] = [3, 2]
• You represent that displacement as the vector [3, 2] from the origin

Problem 3: "I don't know what 'magnitude' is"

Symptom: The concept of magnitude confuses you.

Solution: Magnitude = the length of the arrow = the distance from the origin to the point. It's a positive number that measures "how far" you are from the origin.

Intuitive calculation: If the vector is [3, 0] (only on the X axis), the magnitude is 3 (the direct distance). If the vector is [3, 4], the magnitude is 5 (Pythagorean: 3² + 4² = 25, √25 = 5). You don't need to calculate exact magnitudes; just understand that it's "the length of the arrow".

Problem 4: "Are vectors in AI different from vectors in geometry?"

Symptom: You think an "embedding" is something different from a "mathematical vector".

Solution: They're the same thing. An embedding is an ordinary vector with many dimensions (e.g. 1536D). There's no mathematical difference. The only difference is the context of use:

• In geometry: you use vectors to represent position, force, velocity
• In AI: you use vectors to represent meaning (words, phrases, documents)

But mathematically, both are arrays of numbers with direction and magnitude.

Connection to semantic search and RAG

Now that you know what a vector is, you can understand the full semantic search flow:

1. Document → Vector (embedding)

A document like "The dog is a domestic animal" is converted into a high-dimensional vector:

doc_vector = [0.23, -0.45, 0.12, ..., -0.34]  (1536 numbers)

2. Query → Vector (embedding)

Your question "Which animals are pets?" is also converted into a vector:

query_vector = [0.25, -0.43, 0.10, ..., -0.32]  (1536 numbers)

3. Compare vectors (similarity)

We measure how close the vectors are (cosine similarity, which you'll see in Module 3):

Similarity(doc_vector, query_vector) = 0.92  (very high → similar)

4. Result: Nearby documents are relevant

The document whose vector is closest to the query is the most relevant one.

This is semantic search: convert text → vectors, compare vectors, return the closest ones.

This is the heart of RAG: the "retrieval" step uses semantic search to find relevant documents based on vector closeness.

Without understanding what a vector is, this flow would be magic. With this capsule, you already have the first block: a vector is a numerical representation with direction and magnitude. In the following capsules you'll see how to visualize, compare and search vectors.

Summary

In one sentence: A vector is a geometric entity with direction and magnitude, represented by numerical coordinates (e.g. [3, 2] in 2D, [3, 2, -1] in 3D, [...] in 1536D).

Key points:

• Direction: Where the vector points (e.g. northeast, south)
• Magnitude: How long the vector is (the distance from the origin)
• Coordinates: The numerical representation ([x, y] in 2D, [x, y, z] in 3D, etc.)
• Arrow vs point: Both perspectives are correct (arrow = displacement, point = position)
• Vector ≠ scalar: A vector has direction; a scalar is just a number
• High dimensions: Vectors can have 1536D, 3072D; same concept, more axes
• In AI: Embeddings are vectors that represent meaning (words, documents)
• Semantic search: Compares vectors to find similar documents
• RAG: Uses semantic search (vector comparison) for retrieval

Additional resources

1. 3Blue1Brown: "Vectors, what even are they?" — A 10-minute video with excellent visualizations of vectors as arrows and as points. A perfect complement to this capsule. In English.

2. Khan Academy: "Vector Introduction" — A series of short videos on basic vectors: definition, notation, visualization. In English with subtitles.

3. Better Explained: "Vector Calculus" — A blog with intuitive explanations of vectors and operations. The "Understanding Vectors" section is useful. In English.

4. Immersive Math: Chapter 1 - Introduction — An interactive book where you can rotate and manipulate 3D vectors in the browser. Excellent for visualization. In English.

5. Jay Alammar: "Visualizing A Neural Machine Translation Model" — A blog with diagrams on how vectors represent words in language models. For after you complete the module. In English.

6. Wikipedia: "Euclidean Vector" — The complete formal definition with mathematical notation. Useful if you want to go deeper into linear algebra afterward. In English (also available in other languages).

Next capsule: 03-vectors-in-2d-and-3d.md — Detailed visualization of vectors in planes and three-dimensional spaces.