Module 2: OpenAI API - Introduction

Pricing, Rate Limits, and Cost Optimization

Inicia sesion para marcar tu progreso

Capsule overview

You already know how to use the OpenAI API. Now you need to understand:

How cost is calculated (input/output tokens)
Rate limits (how many requests you can make)
How to optimize costs (without sacrificing quality)

This capsule gives you tools to keep your bill under control.

Time: 25 minutes
Difficulty: Medium

💰 OpenAI Pricing (February 2026)

Models and prices:

Model	Input ($/1M tokens)	Output ($/1M tokens)	Context Window
gpt-4-turbo	$10.00	$30.00	128k
gpt-4	$30.00	$60.00	8k
gpt-3.5-turbo	$0.50	$1.50	16k

Updated: Always check at https://openai.com/pricing

How you're billed:

Formula:

Cost = (Input tokens × Input price) + (Output tokens × Output price)

Example:

Prompt: 100 tokens (input)
Response: 200 tokens (output)
Model: GPT-3.5-turbo

Calculation:

Input:  100 tokens × $0.50/1M = $0.00005
Output: 200 tokens × $1.50/1M = $0.00030
Total:  $0.00035 per request

For 1000 requests: $0.35

🔢 Counting Tokens

Method 1: From the response (exact)

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Hello"}]
)

usage = response.usage
print(f"Input tokens:  {usage.prompt_tokens}")
print(f"Output tokens: {usage.completion_tokens}")
print(f"Total tokens:  {usage.total_tokens}")

Output:

Input tokens:  8
Output tokens: 12
Total tokens:  20

Method 2: Estimating with tiktoken (before the request)

Useful for calculating cost BEFORE sending.

Installation:

pip install tiktoken

Code:

import tiktoken

def count_tokens(text: str, model: str = "gpt-3.5-turbo") -> int:
    """Count the tokens in a piece of text."""
    encoding = tiktoken.encoding_for_model(model)
    return len(encoding.encode(text))

# Test
prompt = "Explain what Python is in 50 words"
tokens = count_tokens(prompt)
print(f"Prompt has {tokens} tokens")

# Estimate cost (assuming 100 output tokens)
input_cost = tokens * 0.50 / 1_000_000
output_cost = 100 * 1.50 / 1_000_000
total_cost = input_cost + output_cost
print(f"Estimated cost: ${total_cost:.6f}")

Rules of thumb (approximate):

1 token ≈ 4 characters (English)
1 token ≈ 0.75 words (English)
Spanish: 1.2-1.5x more tokens (longer words)

Example:

"Hello world" = 2 tokens (English)
"Hola mundo" = 3 tokens (Spanish)

📊 Calculating Your App's Cost

Real example: FAQ Chatbot

Specs:

1000 users/day
3 average queries/user
50 average input tokens
100 average output tokens

Monthly calculation:

# Volume
users_per_day = 1000
queries_per_user = 3
days_per_month = 30
total_queries = users_per_day * queries_per_user * days_per_month  # 90,000

# Tokens
input_tokens_per_query = 50
output_tokens_per_query = 100

total_input_tokens = total_queries * input_tokens_per_query    # 4.5M
total_output_tokens = total_queries * output_tokens_per_query  # 9M

# Cost (GPT-3.5-turbo)
input_cost = (total_input_tokens / 1_000_000) * 0.50   # $2.25
output_cost = (total_output_tokens / 1_000_000) * 1.50  # $13.50

total_monthly_cost = input_cost + output_cost  # $15.75/month

Result: ~$16/month for 90k queries

🚦 Rate Limits

What they are:

Limits on requests/tokens per minute to prevent abuse.

Tiers (based on how much you've spent):

Tier	Requirement	GPT-3.5 (RPM)	GPT-4 (RPM)	TPM
Free	$0 spent	60	3	40k
Tier 1	$5+ spent	500	500	200k
Tier 2	$50+ spent	5000	5000	2M

RPM: Requests per minute
TPM: Tokens per minute

See your limits: https://platform.openai.com/account/rate-limits

Error when you exceed them:

openai.RateLimitError: Rate limit reached for requests

Solution: Implement retries (capsule 07).

💡 Cost Optimization Strategies

1. Use GPT-3.5 when it's enough

Rule: GPT-4 only if you NEED maximum quality.

Example:

Simple FAQ → GPT-3.5 ✅
Complex legal analysis → GPT-4 ✅
Classification → GPT-3.5 ✅

Savings: 20x (GPT-4 is 20x more expensive)

2. Minimize context (history)

❌ Bad:

messages = last_50_messages  # 5000 tokens

✅ Good:

messages = last_5_messages   # 500 tokens

Savings: 10x fewer input tokens

3. Use `max_tokens` to limit output

❌ Bad:

max_tokens=None  # GPT decides (may generate 1000+ tokens)

✅ Good:

max_tokens=150  # Enough for an FAQ

Savings: 6.6x fewer output tokens

4. Cache common responses

import json

# In-memory cache (simple)
cache = {}

def ask_with_cache(prompt: str) -> str:
    """Look in the cache before calling the API."""
    
    # Check cache
    if prompt in cache:
        print("[CACHE HIT]")
        return cache[prompt]
    
    # Cache miss → Call the API
    print("[CACHE MISS - API call]")
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}]
    )
    
    answer = response.choices[0].message.content
    
    # Save in cache
    cache[prompt] = answer
    
    return answer

# Test
print(ask_with_cache("What is Python?"))  # MISS (calls the API)
print(ask_with_cache("What is Python?"))  # HIT (does not call the API)

Savings: 100% for repeated queries

5. Batching (grouped requests)

If you process many similar texts:

# ❌ Bad: 100 requests
for text in texts:
    response = client.chat.completions.create(...)

# ✅ Good: 1 request with a batch
batch_prompt = "\n\n".join([f"Text {i}: {text}" for i, text in enumerate(texts)])
response = client.chat.completions.create(
    messages=[{"role": "user", "content": f"Classify these texts:\n{batch_prompt}"}]
)

Savings: Request overhead (less time, similar cost)

6. Summarization of long conversations

Instead of sending 50 messages:

# Every 20 messages, summarize
if len(messages) > 20:
    summary = summarize(messages[:-5])  # Summarize the old ones
    messages = [system_message, summary] + messages[-5:]  # Keep the last 5

Savings: 80% input tokens in long conversations

📈 Usage Monitoring

OpenAI Dashboard:

Go to: https://platform.openai.com/usage
You'll see charts for:
- Daily cost
- Requests per model
- Tokens consumed

Set up alerts:

Soft limit: $5/month (email alert)
Hard limit: $10/month (stops requests)

Logging in code:

import json
from datetime import datetime

def log_usage(prompt: str, response):
    """Save usage to a JSON file."""
    
    log_entry = {
        "timestamp": datetime.now().isoformat(),
        "model": response.model,
        "prompt_tokens": response.usage.prompt_tokens,
        "completion_tokens": response.usage.completion_tokens,
        "total_tokens": response.usage.total_tokens,
        "prompt_preview": prompt[:50],  # First 50 chars
    }
    
    # Append to file
    with open("usage_log.json", "a") as f:
        f.write(json.dumps(log_entry) + "\n")

# Usage
response = client.chat.completions.create(...)
log_usage(prompt, response)

Analysis afterwards:

import json

total_tokens = 0
with open("usage_log.json") as f:
    for line in f:
        entry = json.loads(line)
        total_tokens += entry["total_tokens"]

cost = (total_tokens / 1_000_000) * 1.0  # Assuming an avg of $1/1M
print(f"Total cost: ${cost:.2f}")

🧮 Cost Calculator (Tool)

def calculate_cost(
    input_tokens: int,
    output_tokens: int,
    model: str = "gpt-3.5-turbo"
) -> dict:
    """Calculate the cost of a request."""
    
    pricing = {
        "gpt-3.5-turbo": {"input": 0.50, "output": 1.50},
        "gpt-4-turbo": {"input": 10.00, "output": 30.00},
        "gpt-4": {"input": 30.00, "output": 60.00},
    }
    
    if model not in pricing:
        raise ValueError(f"Unknown model: {model}")
    
    prices = pricing[model]
    
    input_cost = (input_tokens / 1_000_000) * prices["input"]
    output_cost = (output_tokens / 1_000_000) * prices["output"]
    total_cost = input_cost + output_cost
    
    return {
        "input_tokens": input_tokens,
        "output_tokens": output_tokens,
        "input_cost": input_cost,
        "output_cost": output_cost,
        "total_cost": total_cost,
        "model": model
    }

# Test
result = calculate_cost(100, 200, "gpt-3.5-turbo")
print(f"Total cost: ${result['total_cost']:.6f}")

📊 Summary

Key concepts:

Pricing:
- Input tokens × Input price
- Output tokens × Output price
- GPT-3.5: $0.50/$1.50 per 1M tokens
Rate limits:
- Free tier: 60 RPM (GPT-3.5)
- Tier 1: 500 RPM ($5+ spent)
- Implement retries to handle limits
Optimization:
- Use GPT-3.5 when it's enough
- Minimize context/history
- Cache common responses
- Limit output with max_tokens
Monitoring:
- OpenAI Dashboard (daily usage)
- Custom logs (detailed analysis)
- Alerts (soft/hard limits)

🔗 Additional resources

OpenAI Pricing - Updated
Rate Limits - Official docs
Tiktoken - Token counter
Usage Dashboard - Monitoring

➡️ Next step

Next capsule: 07-error-handling-and-retries.md

You'll learn to handle production errors:

Rate limit errors (429)
Timeouts
API errors (500, 503)
Exponential backoff
Retry strategies

Time: 25 minutes

Estimated time: 25 minutes
Next: 07-error-handling-and-retries.md

← PreviousAdvanced OpenAI API Parameters Next →Error Handling and Retry Strategies

Pricing, Rate Limits, and Cost Optimization

Capsule overview

💰 OpenAI Pricing (February 2026)

Models and prices:

How you're billed:

🔢 Counting Tokens

Method 1: From the response (exact)

Method 2: Estimating with tiktoken (before the request)

Rules of thumb (approximate):

📊 Calculating Your App's Cost

Real example: FAQ Chatbot

🚦 Rate Limits

What they are:

Error when you exceed them:

💡 Cost Optimization Strategies

1. Use GPT-3.5 when it's enough

2. Minimize context (history)

3. Use max_tokens to limit output

4. Cache common responses

5. Batching (grouped requests)

6. Summarization of long conversations

📈 Usage Monitoring

OpenAI Dashboard:

Logging in code:

🧮 Cost Calculator (Tool)

📊 Summary

Key concepts:

🔗 Additional resources

➡️ Next step

3. Use `max_tokens` to limit output