Module 3: Embedding Models Compared

OpenAI Embeddings Models: Deep Dive Comparison

Capsule overview

OpenAI offers 2 main embedding models in 2026: text-embedding-3-small and text-embedding-3-large. Although both work well, they have critical differences in performance, cost, dimensions, and optimal use cases.

In this capsule you'll learn the technical differences between the two models, how they compare on benchmarks (MTEB), when to use each one depending on your use case, and how to configure dimensions to optimize storage/cost. You'll also see code to compare both models empirically.

By the end, you'll be able to choose the right OpenAI model for your project.

Available models (2026)

OpenAI Embeddings Models:

Model	Default dims	Config dims	MTEB Score	Cost/1M tokens	Release
`text-embedding-3-small`	1536	512-1536	~62	$0.020	2024
`text-embedding-3-large`	3072	256-3072	~64	$0.130	2024
`text-embedding-ada-002` (legacy)	1536	1536	~61	$0.100	2022

Recommendation: Use 3-small or 3-large (ada-002 is legacy).

text-embedding-3-small

Characteristics:

Model: text-embedding-3-small
Dimensions: 1536 (default), configurable 512-1536
MTEB Score: ~62
Cost: $0.020 / 1M tokens
Max tokens: 8,191

Performance (MTEB):

Benchmark         | Score | Rank
------------------|-------|------
Retrieval         | 0.55  | Good
Classification    | 0.68  | Very good
Clustering        | 0.47  | Good
Semantic Similarity| 0.82 | Excellent
Reranking         | 0.60  | Good

MTEB Average: 0.62

Interpretation: Good for general semantic search, excellent for similarity tasks.

When to use 3-small:

✅ 1. Fast prototyping

# Low cost = fast iteration
# $0.02/1M tokens vs $0.13/1M tokens (6.5x cheaper)

✅ 2. Limited budget

# Startup with $100/month:
# 3-small: 5,000M tokens (5B)
# 3-large: 769M tokens

✅ 3. Non-critical semantic search

# FAQ search, documentation search
# Does not require maximum precision

✅ 4. High volume (millions of queries)

# 10M queries/month:
# 3-small: $200
# 3-large: $1,300

Usage code:

from openai import OpenAI
import os
from dotenv import load_dotenv

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Generate an embedding with 3-small
response = client.embeddings.create(
    model="text-embedding-3-small",
    input="Python is a programming language"
)

embedding = response.data[0].embedding
print(f"Model: text-embedding-3-small")
print(f"Dimensions: {len(embedding)}")
print(f"First 5 dims: {embedding[:5]}")

Output:

Model: text-embedding-3-small
Dimensions: 1536
First 5 dims: [0.0234, -0.0123, 0.0456, -0.0189, 0.0267]

text-embedding-3-large

Characteristics:

Model: text-embedding-3-large
Dimensions: 3072 (default), configurable 256-3072
MTEB Score: ~64
Cost: $0.130 / 1M tokens
Max tokens: 8,191

Performance (MTEB):

Benchmark         | Score | Rank
------------------|-------|------
Retrieval         | 0.60  | Excellent
Classification    | 0.70  | Excellent
Clustering        | 0.52  | Very good
Semantic Similarity| 0.84 | Excellent
Reranking         | 0.64  | Very good

MTEB Average: 0.64

Improvement vs 3-small: +2 MTEB points (~3% better).

When to use 3-large:

✅ 1. Production-ready RAG

# Correct context is critical
# +3% precision = better queries

✅ 2. Critical domain (legal, medical)

# Errors are costly
# Worth paying 6.5x more

✅ 3. Low-to-medium volume

# <1M queries/month:
# Manageable total cost ($130/month)

✅ 4. Benchmarking other models

# Use 3-large as the "gold standard"
# Compare open-source vs this one

Usage code:

# Generate an embedding with 3-large
response = client.embeddings.create(
    model="text-embedding-3-large",
    input="Python is a programming language"
)

embedding = response.data[0].embedding
print(f"Model: text-embedding-3-large")
print(f"Dimensions: {len(embedding)}")

Output:

Model: text-embedding-3-large
Dimensions: 3072

Direct comparison: 3-small vs 3-large

Empirical experiment:

import numpy as np
from openai import OpenAI
import os
from dotenv import load_dotenv

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

def cosine_similarity(vec_a, vec_b):
    """Cosine similarity"""
    return np.dot(vec_a, vec_b) / (np.linalg.norm(vec_a) * np.linalg.norm(vec_b))

def compare_models(query, docs):
    """Compare 3-small vs 3-large on retrieval"""
    results = {}
    
    for model in ["text-embedding-3-small", "text-embedding-3-large"]:
        # Embed query
        query_emb = client.embeddings.create(
            model=model,
            input=query
        ).data[0].embedding
        
        # Embed docs
        doc_embs = []
        for doc in docs:
            emb = client.embeddings.create(
                model=model,
                input=doc
            ).data[0].embedding
            doc_embs.append(emb)
        
        # Calculate similarities
        sims = [cosine_similarity(query_emb, doc_emb) for doc_emb in doc_embs]
        
        # Rank docs
        ranked = sorted(zip(docs, sims), key=lambda x: x[1], reverse=True)
        
        results[model] = ranked
    
    return results

# Test
query = "How do I install Python?"

docs = [
    "To install Python, download the installer from python.org",
    "JavaScript is a web programming language",
    "Python requires pip to install packages"
]

results = compare_models(query, docs)

print("Query:", query)
print("\n=== text-embedding-3-small ===")
for doc, sim in results["text-embedding-3-small"]:
    print(f"{sim:.4f} | {doc[:50]}...")

print("\n=== text-embedding-3-large ===")
for doc, sim in results["text-embedding-3-large"]:
    print(f"{sim:.4f} | {doc[:50]}...")

Expected output:

Query: How do I install Python?

=== text-embedding-3-small ===
0.8234 | To install Python, download the installer from...
0.7123 | Python requires pip to install packages...
0.5432 | JavaScript is a web programming language...

=== text-embedding-3-large ===
0.8567 | To install Python, download the installer from...  ← Better score
0.7421 | Python requires pip to install packages...
0.5123 | JavaScript is a web programming language...

Observation: 3-large discriminates better (higher score for the correct doc, lower for the irrelevant one).

Configurable dimensions

Why reduce dimensions:

# Storage:
# 1M docs × 3072 dims × 4 bytes = 12 GB (3-large)
# 1M docs × 1536 dims × 4 bytes = 6 GB (3-small)
# 1M docs × 512 dims × 4 bytes = 2 GB (3-small reduced)

# Speedup:
# 512 dims → 3x faster in search vs 1536 dims

Code with reduced dimensions:

# text-embedding-3-small with 512 dims
response = client.embeddings.create(
    model="text-embedding-3-small",
    input="Python is popular",
    dimensions=512  # Reduce from 1536 → 512
)

embedding = response.data[0].embedding
print(f"Dimensions: {len(embedding)}")  # 512

Trade-off: Precision vs Storage

# OpenAI internal benchmark (retrieval@10):

Model: text-embedding-3-small
- 1536 dims (default): 100% precision (baseline)
- 1024 dims:           98% precision
- 512 dims:            95% precision
- 256 dims:            90% precision

Recommendation: 512-1024 dims = sweet spot

Detailed cost analysis

Scenario 1: RAG system (100K queries/month)

# Assumptions:
# - Query: 50 tokens on average
# - Docs retrieved: 5 docs × 200 tokens = 1000 tokens
# - Total per query: 1050 tokens

tokens_per_month = 100_000 * 1050  # 105M tokens

# 3-small:
cost_small = (tokens_per_month / 1_000_000) * 0.020
print(f"3-small: ${cost_small:.2f}/month")  # $2.10/month

# 3-large:
cost_large = (tokens_per_month / 1_000_000) * 0.130
print(f"3-large: ${cost_large:.2f}/month")  # $13.65/month

# Difference: $11.55/month (~6.5x)

Decision: If budget <$50/month → 3-small, if >$50/month and RAG is critical → 3-large.

Scenario 2: E-commerce search (1M queries/month)

# Assumptions:
# - Query: 30 tokens on average
# - Products: 100K (embed once)
# - Queries: 1M/month

# Initial embedding of products (once):
initial_tokens = 100_000 * 50  # 5M tokens
initial_cost_small = (initial_tokens / 1_000_000) * 0.020  # $0.10
initial_cost_large = (initial_tokens / 1_000_000) * 0.130  # $0.65

# Monthly queries:
query_tokens = 1_000_000 * 30  # 30M tokens
query_cost_small = (query_tokens / 1_000_000) * 0.020  # $0.60/month
query_cost_large = (query_tokens / 1_000_000) * 0.130  # $3.90/month

# Monthly total (after setup):
print(f"3-small: ${query_cost_small:.2f}/month")  # $0.60/month
print(f"3-large: ${query_cost_large:.2f}/month")  # $3.90/month

Decision: For e-commerce, 3-small is probably enough (significant savings).

Decision matrix

When to choose each model:

Criterion               | 3-small | 3-large
------------------------|---------|--------
Budget <$100/month      | ✅      | ❌
RAG production-critical | ❌      | ✅
Prototype/MVP           | ✅      | ❌
Critical domain (legal) | ❌      | ✅
High volume (>1M/month) | ✅      | ❌
Maximum precision       | ❌      | ✅

Benchmarking in code

Automatic comparison:

import time

def benchmark_model(model_name, texts):
    """Benchmark latency and cost"""
    start = time.time()
    
    response = client.embeddings.create(
        model=model_name,
        input=texts
    )
    
    latency = time.time() - start
    
    # Estimate cost (assuming 50 tokens/text)
    tokens = len(texts) * 50
    cost_per_1m = 0.020 if "small" in model_name else 0.130
    cost = (tokens / 1_000_000) * cost_per_1m
    
    return {
        'model': model_name,
        'latency_ms': latency * 1000,
        'cost_usd': cost,
        'texts_processed': len(texts)
    }

# Test
texts = ["Python is popular"] * 100

result_small = benchmark_model("text-embedding-3-small", texts)
result_large = benchmark_model("text-embedding-3-large", texts)

print("=== Benchmark Results ===")
print(f"3-small: {result_small['latency_ms']:.0f}ms, ${result_small['cost_usd']:.6f}")
print(f"3-large: {result_large['latency_ms']:.0f}ms, ${result_large['cost_usd']:.6f}")

Typical output:

=== Benchmark Results ===
3-small: 324ms, $0.000100
3-large: 412ms, $0.000650

Observation: 3-large is ~25% slower (more dims).

Summary

What you learned:

✅ 2 main models: 3-small (cheap, good) and 3-large (expensive, excellent)
✅ MTEB scores: 62 vs 64 (~3% improvement)
✅ Costs: $0.02 vs $0.13 (6.5x difference)
✅ Dimensions: Configurable (reduce storage)
✅ Use cases: Prototype vs production

Key concepts:

3-small → Prototype, high volume, limited budget
3-large → Critical RAG, important domain, low volume
Reduced dimensions → Storage savings (~3x)

Additional resources

OpenAI Embeddings Guide - Official
MTEB Leaderboard - Rankings
OpenAI Pricing - Up-to-date costs
Embeddings Comparison - Docs

In the next capsule

Capsule 03: Open-Source Embeddings Overview

You'll learn:

Sentence-BERT (SBERT)
BGE Models (BAAI)
Instructor Embeddings
E5 Models
How to use HuggingFace Sentence-Transformers

From the OpenAI API to self-hosted open-source.

Module 3 - Embeddings Deep Dive Guide OpenAI embeddings: choosing between small and large

← PreviousIntroduction to Module 3: Embedding Models Compared Next →Open-Source Embeddings: Self-Hosted Alternatives