Module 1: What Are Embeddings?

Mini-Project: Your First Embedding - Similarity Calculator

Next →

Project overview

In this mini-project you'll build your first working system using embeddings: a Similarity Calculator that lets you search for documents similar to a query using OpenAI embeddings.

This project consolidates ALL the concepts from Module 1: the definition of embeddings, vector properties, vector spaces, use cases, comparison with keywords, and architecture. By the end you'll have a production-ready CLI tool that you can expand in future modules.

Estimated duration: 30-40 minutes

Project objectives

By completing this project, you'll have:

  • ✅ Configured the OpenAI API correctly
  • ✅ Generated embeddings for multiple documents
  • ✅ Calculated cosine similarity with numpy
  • ✅ Implemented a top-K most-similar search
  • ✅ Created a working CLI with error handling
  • ✅ Cached embeddings (optimization)

Technical specifications

Required features:

  1. Indexing: Load 10+ documents and generate embeddings
  2. Query: The user enters a query, the system returns the top-3 most similar
  3. Scoring: Show the cosine similarity score for each result
  4. Cache: Save embeddings to a file to reuse
  5. Error handling: Handle API errors, missing files, etc.

Tech stack:

  • Python 3.10+
  • OpenAI API (text-embedding-3-small)
  • NumPy (vector calculations)
  • python-dotenv (environment variables)
  • JSON (simple persistence)

Step 1: Project setup

1.1: File structure

embeddings-similarity-calculator/
├── .env                    # API key (do NOT commit)
├── .gitignore             # Ignore .env
├── requirements.txt       # Dependencies
├── documents.txt          # Corpus (10 documents)
├── embeddings_cache.json  # Saved embeddings
└── similarity_calculator.py  # Main script

1.2: Create the project folder

mkdir embeddings-similarity-calculator
cd embeddings-similarity-calculator

1.3: Create a virtualenv (recommended)

# Create virtualenv
python -m venv venv

# Activate
# macOS/Linux:
source venv/bin/activate

# Windows:
venv\Scripts\activate

1.4: Install dependencies

Create requirements.txt:

openai==1.12.0
numpy==1.26.3
python-dotenv==1.0.0

Install:

pip install -r requirements.txt

1.5: Configure the OpenAI API key

Create .env:

# .env
OPENAI_API_KEY=sk-proj-your-api-key-here

Create .gitignore:

# .gitignore
.env
venv/
__pycache__/
*.pyc
embeddings_cache.json

⚠️ IMPORTANT: NEVER commit .env with your API key.

Step 2: Create the document corpus

Create documents.txt:

Python is a high-level programming language
JavaScript is the main language for web development
TypeScript adds static types to JavaScript
React is a library for building user interfaces
Vue.js is a progressive framework for building UIs
Django is a Python web framework for the backend
FastAPI is a modern, fast framework for APIs in Python
Node.js lets you run JavaScript on the server
Docker is a platform for containerizing applications
Kubernetes orchestrates containers in production

Each line = 1 document.

Step 3: Implement the Similarity Calculator

Create similarity_calculator.py:

"""
Similarity Calculator - Module 1 Mini-Project
A semantic search system using OpenAI embeddings
"""

import os
import json
import numpy as np
from openai import OpenAI
from dotenv import load_dotenv
from typing import List, Dict, Tuple

# Load environment variables
load_dotenv()

# Configuration
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
EMBEDDING_MODEL = "text-embedding-3-small"
DOCUMENTS_FILE = "documents.txt"
CACHE_FILE = "embeddings_cache.json"


class SimilarityCalculator:
    """
    Semantic search system with embeddings
    """
    
    def __init__(self):
        """Initialize the OpenAI client and data structures"""
        if not OPENAI_API_KEY:
            raise ValueError("OPENAI_API_KEY not found in .env")
        
        self.client = OpenAI(api_key=OPENAI_API_KEY)
        self.documents: List[str] = []
        self.embeddings: Dict[int, List[float]] = {}
    
    def load_documents(self, filepath: str) -> None:
        """
        Load documents from a file
        
        Args:
            filepath: Path to the documents file
        """
        try:
            with open(filepath, 'r', encoding='utf-8') as f:
                self.documents = [line.strip() for line in f if line.strip()]
            
            print(f"✅ Loaded {len(self.documents)} documents")
        
        except FileNotFoundError:
            raise FileNotFoundError(f"File {filepath} not found")
        except Exception as e:
            raise Exception(f"Error loading documents: {e}")
    
    def get_embedding(self, text: str) -> List[float]:
        """
        Generate an embedding for a text
        
        Args:
            text: Text to convert into an embedding
        
        Returns:
            List of floats (embedding vector)
        """
        try:
            response = self.client.embeddings.create(
                model=EMBEDDING_MODEL,
                input=text
            )
            return response.data[0].embedding
        
        except Exception as e:
            print(f"❌ Error generating embedding: {e}")
            raise
    
    def generate_embeddings(self, use_cache: bool = True) -> None:
        """
        Generate embeddings for all documents
        
        Args:
            use_cache: If True, try to load from cache first
        """
        # Try to load from cache
        if use_cache and self.load_cache():
            print("✅ Embeddings loaded from cache")
            return
        
        print(f"🔄 Generating embeddings for {len(self.documents)} documents...")
        
        for idx, doc in enumerate(self.documents):
            try:
                embedding = self.get_embedding(doc)
                self.embeddings[idx] = embedding
                print(f"  [{idx+1}/{len(self.documents)}] Generated")
            
            except Exception as e:
                print(f"  ❌ Error on document {idx}: {e}")
        
        # Save to cache
        self.save_cache()
        print("✅ Embeddings generated and saved to cache")
    
    def save_cache(self) -> None:
        """Save embeddings to a JSON file"""
        try:
            cache_data = {
                "documents": self.documents,
                "embeddings": {str(k): v for k, v in self.embeddings.items()}
            }
            
            with open(CACHE_FILE, 'w', encoding='utf-8') as f:
                json.dump(cache_data, f)
        
        except Exception as e:
            print(f"⚠️ Error saving cache: {e}")
    
    def load_cache(self) -> bool:
        """
        Load embeddings from the cache
        
        Returns:
            True if loaded successfully, False otherwise
        """
        try:
            if not os.path.exists(CACHE_FILE):
                return False
            
            with open(CACHE_FILE, 'r', encoding='utf-8') as f:
                cache_data = json.load(f)
            
            # Check that the documents match
            if cache_data["documents"] != self.documents:
                print("⚠️ Documents changed, regenerating embeddings...")
                return False
            
            # Load embeddings
            self.embeddings = {
                int(k): v for k, v in cache_data["embeddings"].items()
            }
            
            return True
        
        except Exception as e:
            print(f"⚠️ Error loading cache: {e}")
            return False
    
    @staticmethod
    def cosine_similarity(vec_a: List[float], vec_b: List[float]) -> float:
        """
        Calculate cosine similarity between two vectors
        
        Args:
            vec_a: First vector
            vec_b: Second vector
        
        Returns:
            Similarity (0.0 to 1.0)
        """
        vec_a = np.array(vec_a)
        vec_b = np.array(vec_b)
        
        dot_product = np.dot(vec_a, vec_b)
        norm_a = np.linalg.norm(vec_a)
        norm_b = np.linalg.norm(vec_b)
        
        return dot_product / (norm_a * norm_b)
    
    def search(self, query: str, top_k: int = 3) -> List[Tuple[int, str, float]]:
        """
        Search for the documents most similar to the query
        
        Args:
            query: The user's query
            top_k: Number of results to return
        
        Returns:
            List of tuples (doc_id, document, similarity)
        """
        # Generate the query embedding
        print(f"\n🔍 Searching: '{query}'")
        query_embedding = self.get_embedding(query)
        
        # Calculate similarities
        similarities = []
        for doc_id, doc_embedding in self.embeddings.items():
            sim = self.cosine_similarity(query_embedding, doc_embedding)
            similarities.append((doc_id, self.documents[doc_id], sim))
        
        # Sort by similarity (descending)
        similarities.sort(key=lambda x: x[2], reverse=True)
        
        # Return top-K
        return similarities[:top_k]
    
    def print_results(self, results: List[Tuple[int, str, float]]) -> None:
        """
        Print formatted results
        
        Args:
            results: List of results [(doc_id, doc, similarity)]
        """
        print("\n📊 Results:")
        print("=" * 80)
        
        for rank, (doc_id, doc, similarity) in enumerate(results, 1):
            # Visual progress bar
            bar_length = int(similarity * 50)  # 50 chars max
            bar = "█" * bar_length + "░" * (50 - bar_length)
            
            print(f"\n{rank}. [Doc {doc_id}] Similarity: {similarity:.4f}")
            print(f"   {bar} {similarity:.1%}")
            print(f"   \"{doc}\"")
        
        print("\n" + "=" * 80)


def main():
    """Main function - interactive CLI"""
    print("=" * 80)
    print("🚀 SIMILARITY CALCULATOR - Embeddings Deep Dive Guide")
    print("=" * 80)
    
    try:
        # Initialize
        calculator = SimilarityCalculator()
        
        # Load documents
        calculator.load_documents(DOCUMENTS_FILE)
        
        # Generate embeddings (uses cache if it exists)
        calculator.generate_embeddings(use_cache=True)
        
        # Interactive loop
        print("\n" + "=" * 80)
        print("💬 Enter your query (or 'exit' to quit)")
        print("=" * 80)
        
        while True:
            query = input("\nQuery: ").strip()
            
            if query.lower() in ['exit', 'quit']:
                print("\n👋 See you later!")
                break
            
            if not query:
                print("⚠️ Empty query. Try again.")
                continue
            
            # Search
            results = calculator.search(query, top_k=3)
            
            # Show results
            calculator.print_results(results)
    
    except KeyboardInterrupt:
        print("\n\n👋 Interrupted. See you later!")
    
    except Exception as e:
        print(f"\n❌ Error: {e}")
        import traceback
        traceback.print_exc()


if __name__ == "__main__":
    main()

Step 4: Run the project

First run (without cache):

python similarity_calculator.py

Expected output:

================================================================================
🚀 SIMILARITY CALCULATOR - Embeddings Deep Dive Guide
================================================================================
✅ Loaded 10 documents
🔄 Generating embeddings for 10 documents...
  [1/10] Generated
  [2/10] Generated
  [3/10] Generated
  [4/10] Generated
  [5/10] Generated
  [6/10] Generated
  [7/10] Generated
  [8/10] Generated
  [9/10] Generated
  [10/10] Generated
✅ Embeddings generated and saved to cache

================================================================================
💬 Enter your query (or 'exit' to quit)
================================================================================

Query: python for backend

🔍 Searching: 'python for backend'

📊 Results:
================================================================================

1. [Doc 5] Similarity: 0.8923
   ████████████████████████████████████████████░░░░░░ 89.2%
   "Django is a Python web framework for the backend"

2. [Doc 6] Similarity: 0.8756
   ███████████████████████████████████████████░░░░░░░ 87.6%
   "FastAPI is a modern, fast framework for APIs in Python"

3. [Doc 0] Similarity: 0.8234
   █████████████████████████████████████████░░░░░░░░░ 82.3%
   "Python is a high-level programming language"

================================================================================

Query: containers

🔍 Searching: 'containers'

📊 Results:
================================================================================

1. [Doc 8] Similarity: 0.9012
   █████████████████████████████████████████████░░░░░ 90.1%
   "Docker is a platform for containerizing applications"

2. [Doc 9] Similarity: 0.8678
   ███████████████████████████████████████████░░░░░░░ 86.8%
   "Kubernetes orchestrates containers in production"

3. [Doc 7] Similarity: 0.6823
   ██████████████████████████████████░░░░░░░░░░░░░░░░ 68.2%
   "Node.js lets you run JavaScript on the server"

================================================================================

Query: exit
👋 See you later!

Second run (with cache):

python similarity_calculator.py

Output:

================================================================================
🚀 SIMILARITY CALCULATOR - Embeddings Deep Dive Guide
================================================================================
✅ Loaded 10 documents
✅ Embeddings loaded from cache

================================================================================
💬 Enter your query (or 'exit' to quit)
================================================================================

Query:

Note: This time it did NOT generate embeddings (it loaded from embeddings_cache.json). Saves time and API costs.

Step 5: Manual testing

Test queries:

Query: framework for interfaces
# Should return: React, Vue.js, (related)

Query: typed language
# Should return: TypeScript, (related)

Query: run on the server
# Should return: Node.js, FastAPI, Django

Query: orchestration
# Should return: Kubernetes, Docker

Query: modern web development
# Should return: React, Vue.js, JavaScript

Technical validations

Functionality checklist:

  • The system loads documents correctly
  • Generates embeddings (1536 dims each)
  • Calculates cosine similarity correctly
  • Returns the top-3 sorted by similarity
  • Saves the cache to JSON
  • Loads the cache on subsequent runs
  • Handles errors (invalid API key, missing file)
  • The interactive CLI works
  • Scores are in the range [0.0, 1.0]
  • Results are semantically correct

Optional improvements (Extra Credit)

Level 1: Additional features

# 1. Normalize embeddings
def normalize_embedding(self, embedding: List[float]) -> List[float]:
    """Normalize an embedding to magnitude 1.0"""
    embedding_array = np.array(embedding)
    return (embedding_array / np.linalg.norm(embedding_array)).tolist()

# 2. Show statistics
def print_stats(self):
    """Print corpus statistics"""
    print(f"\n📈 Statistics:")
    print(f"  Documents: {len(self.documents)}")
    print(f"  Embeddings: {len(self.embeddings)}")
    print(f"  Dimensions: 1536")
    print(f"  Model: {EMBEDDING_MODEL}")

Level 2: Compare with BM25

# Add a simple keyword search
def keyword_search(self, query: str, top_k: int = 3):
    """Simplified BM25 search (keyword matching)"""
    query_words = set(query.lower().split())
    
    scores = []
    for idx, doc in enumerate(self.documents):
        doc_words = set(doc.lower().split())
        matches = len(query_words.intersection(doc_words))
        scores.append((idx, doc, matches))
    
    scores.sort(key=lambda x: x[2], reverse=True)
    return scores[:top_k]

# Compare both methods:
semantic_results = calculator.search(query, top_k=3)
keyword_results = calculator.keyword_search(query, top_k=3)

print("\n🔹 Semantic Search:")
calculator.print_results(semantic_results)

print("\n🔹 Keyword Search:")
# print keyword results...

Level 3: Hybrid search

def hybrid_search(self, query: str, alpha: float = 0.5, top_k: int = 3):
    """
    Hybrid search: Semantic + Keyword
    
    Args:
        query: The user's query
        alpha: Weight of keyword (1-alpha = weight of semantic)
        top_k: Number of results
    """
    # Semantic scores
    query_emb = self.get_embedding(query)
    semantic_scores = {}
    for doc_id, doc_emb in self.embeddings.items():
        sim = self.cosine_similarity(query_emb, doc_emb)
        semantic_scores[doc_id] = (sim + 1) / 2  # Normalize to [0, 1]
    
    # Keyword scores
    query_words = set(query.lower().split())
    keyword_scores = {}
    for doc_id, doc in enumerate(self.documents):
        doc_words = set(doc.lower().split())
        matches = len(query_words.intersection(doc_words))
        keyword_scores[doc_id] = matches
    
    # Normalize keyword scores
    max_keyword = max(keyword_scores.values()) if keyword_scores else 1
    keyword_normalized = {
        k: v / max_keyword for k, v in keyword_scores.items()
    }
    
    # Hybrid scores
    hybrid_scores = []
    for doc_id in range(len(self.documents)):
        semantic_score = semantic_scores.get(doc_id, 0)
        keyword_score = keyword_normalized.get(doc_id, 0)
        
        hybrid_score = alpha * keyword_score + (1 - alpha) * semantic_score
        
        hybrid_scores.append((doc_id, self.documents[doc_id], hybrid_score))
    
    # Sort and return top-K
    hybrid_scores.sort(key=lambda x: x[2], reverse=True)
    return hybrid_scores[:top_k]

Troubleshooting

Problem 1: "OPENAI_API_KEY not found"

Cause: .env doesn't exist or doesn't have the key.

Solution:

# Create .env:
echo "OPENAI_API_KEY=sk-proj-your-key-here" > .env

# Verify that it's loading:
python -c "from dotenv import load_dotenv; import os; load_dotenv(); print(os.getenv('OPENAI_API_KEY'))"

Problem 2: "FileNotFoundError: documents.txt"

Cause: The file doesn't exist.

Solution:

# Create documents.txt with the content from Step 2

Problem 3: API rate limit error

Cause: Too many calls to the OpenAI API.

Solution:

# Add a delay between calls:
import time

for idx, doc in enumerate(self.documents):
    embedding = self.get_embedding(doc)
    self.embeddings[idx] = embedding
    time.sleep(0.1)  # 100ms delay

Problem 4: All similarity scores ~0.60-0.70

Cause: A very heterogeneous corpus or a very generic query.

Solution: Normal. Scores >0.80 indicate high similarity. Scores 0.60-0.70 are "somewhat related".

Success criteria

Functionality (70 points):

  • (15 pts) The system loads documents
  • (20 pts) Generates embeddings correctly
  • (15 pts) Calculates cosine similarity
  • (10 pts) Returns top-3 sorted
  • (10 pts) Cache works

Code (20 points):

  • (5 pts) Clean, organized code
  • (5 pts) Consistent type hints
  • (5 pts) Docstrings on functions
  • (5 pts) Robust error handling

Usability (10 points):

  • (5 pts) Intuitive CLI
  • (5 pts) Readable, formatted output

Project summary

You have built:

  • ✅ A working semantic search system
  • ✅ Integration with the OpenAI API
  • ✅ Cosine similarity calculation with numpy
  • ✅ Embedding cache (optimization)
  • ✅ An interactive CLI

Skills acquired:

  • Consuming embedding APIs
  • Vector manipulation with numpy
  • Data persistence (JSON)
  • Robust error handling
  • Semantic search architecture

This project is the foundation for:

  • Module 2: Deep technical architecture
  • Module 4: Semantic search from scratch with numpy
  • Module 6: Embeddings in production (batch, cache)
  • Module 8: Complete integrator project

Additional resources

  1. OpenAI Embeddings API Docs - Official documentation
  2. NumPy Docs - NumPy for vector calculations
  3. Python dotenv - Environment variables
  4. JSON in Python - Serialization
  5. Cosine Similarity Explained - The math

Next steps

Module 2: How Do Embeddings Work?

You'll go deeper into:

  • Transformer encoders (detailed architecture)
  • Tokenization (BPE, WordPiece)
  • Self-attention mechanisms
  • Pooling strategies (real code)
  • Advanced OpenAI API (batching, rate limiting)

This project lays the foundation. The following modules build on it.

Congratulations! 🎉

You've completed Module 1: What Are Embeddings?

What you accomplished:

  • ✅ Understood what embeddings are and why they're critical
  • ✅ Learned vector properties and cosine similarity
  • ✅ Understood high-dimensional vector spaces
  • ✅ Identified real use cases (RAG, search, recommendations)
  • ✅ Compared embeddings vs keyword search
  • ✅ Learned the high-level architecture of Transformers
  • ✅ Built your first working system

You're ready for Module 2: Deep embedding architecture. 🚀

Module 1 completed - Embeddings Deep Dive Guide From fundamental concepts to practical implementation

← PreviousArchitecture Overview: How Embeddings Are GeneratedNext →Welcome to Module 2: How Do Embeddings Work?