Hidden Costs in AI Systems

Capsule overview

The OpenAI invoice says "$2,400/month." You look at the input and output tokens, run the numbers, and they add up to $1,500. Where do the other $900 come from? From costs that don't show up when you think in terms of "price per token": embeddings that pile up silently, context windows stuffed with tokens the model ignores, retries that double your spend with no added value, and development costs nobody budgets for.

This capsule teaches you to spot those hidden costs. They aren't mistakes or bugs — they're structural features of AI systems that most engineers ignore because they never appear on the pricing page. They account for somewhere between 25% and 40% of a typical invoice.

By the end, you'll have a "hidden cost detector" that analyzes the architecture of an AI system and points out where the money is leaking.

The real breakdown: where every dollar comes from

def show_real_cost_breakdown():
    """Breakdown of a $2,400/month invoice in an AI system with RAG."""
    breakdown = {
        "Output tokens (completions)":        {"amount": 900,  "visible": True},
        "Input tokens (prompts + context)":   {"amount": 600,  "visible": True},
        "Embeddings (RAG indexing + queries)":{"amount": 480, "visible": False},
        "Retries (rate limits + errors)":     {"amount": 300, "visible": False},
        "Storage (logs, cache, vectors)":     {"amount": 120, "visible": False},
    }

    total = sum(d["amount"] for d in breakdown.values())
    hidden = sum(d["amount"] for d in breakdown.values() if not d["visible"])

    print(f"{'Component':<42} {'Amount':>8} {'%':>6} {'Type'}")
    print("-" * 68)
    for comp, data in breakdown.items():
        pct = data["amount"] / total * 100
        kind = "HIDDEN" if not data["visible"] else "Visible"
        print(f"{comp:<42} ${data['amount']:>6,} {pct:>5.1f}% {kind}")
    print("-" * 68)
    print(f"{'TOTAL':<42} ${total:>6,} 100.0%")
    print(f"\nHidden costs: ${hidden:,}/month ({hidden/total*100:.1f}% of the total)")

show_real_cost_breakdown()

# Expected output:
Component                                   Amount      %  Type
--------------------------------------------------------------------
Output tokens (completions)                $   900  37.5%  Visible
Input tokens (prompts + context)           $   600  25.0%  Visible
Embeddings (RAG indexing + queries)        $   480  20.0%  HIDDEN
Retries (rate limits + errors)             $   300  12.5%  HIDDEN
Storage (logs, cache, vectors)             $   120   5.0%  HIDDEN
--------------------------------------------------------------------
TOTAL                                      $ 2,400 100.0%

Hidden costs: $900/month (37.5% of the total)

37.5% of the invoice is made up of costs you don't see when you only think about "input and output tokens."

Hidden cost 1: Embeddings in RAG pipelines

Embeddings look cheap: $0.02 per million tokens with text-embedding-3-small. But in a RAG pipeline they pile up through two channels: the initial indexing of the corpus, and converting every user query into an embedding for similarity search.

def calculate_embedding_costs(
    num_documents: int,
    avg_tokens_per_doc: int,
    queries_per_day: int,
    avg_query_tokens: int = 50,
    price_per_1m: float = 0.02,
    reindex_frequency_days: int = 30
) -> dict:
    """Calculate the real cost of embeddings in a RAG pipeline."""
    total_doc_tokens = num_documents * avg_tokens_per_doc
    indexing_cost = (total_doc_tokens / 1_000_000) * price_per_1m
    reindexes_per_month = 30 / reindex_frequency_days
    monthly_indexing = indexing_cost * reindexes_per_month

    monthly_query = (queries_per_day * avg_query_tokens / 1_000_000) * price_per_1m * 30
    return {
        "indexing_per_run": round(indexing_cost, 4),
        "indexing_monthly": round(monthly_indexing, 4),
        "queries_monthly": round(monthly_query, 4),
        "total_monthly": round(monthly_indexing + monthly_query, 4),
    }

costs = calculate_embedding_costs(
    num_documents=10_000, avg_tokens_per_doc=2_000,
    queries_per_day=5_000, reindex_frequency_days=7
)
print(f"Indexing/month: ${costs['indexing_monthly']:.4f}")
print(f"Queries/month:  ${costs['queries_monthly']:.4f}")
print(f"TOTAL/month:    ${costs['total_monthly']:.4f}")

# Expected output:
Indexing/month: $1.7143
Queries/month:  $0.1500
TOTAL/month:    $1.8643

Only $1.86? Where are the $480?

That example uses 10K documents with the small model. In real production, the numbers scale dramatically:

scenarios = [
    ("Startup (10K docs)",    10_000,   2_000,   5_000, 30),
    ("Scale-up (100K docs)", 100_000,   3_000,  50_000,  7),
    ("Enterprise (1M docs)", 1_000_000, 3_000, 200_000,  3),
]

for price_label, price in [("small $0.02", 0.02), ("large $0.13", 0.13)]:
    print(f"\n=== text-embedding-3-{price_label}/1M ===")
    for name, docs, tpd, qpd, reindex in scenarios:
        c = calculate_embedding_costs(docs, tpd, qpd, price_per_1m=price, reindex_frequency_days=reindex)
        print(f"  {name:<25} ${c['total_monthly']:>10.2f}/month")

# Expected output:

=== text-embedding-3-small $0.02/1M ===
  Startup (10K docs)        $       0.55/month
  Scale-up (100K docs)      $      27.21/month
  Enterprise (1M docs)      $     606.00/month

=== text-embedding-3-large $0.13/1M ===
  Startup (10K docs)        $       3.58/month
  Scale-up (100K docs)      $     176.89/month
  Enterprise (1M docs)      $    3939.00/month

With frequent re-indexing and text-embedding-3-large at enterprise scale, embeddings can dominate the invoice. The trap: every re-index pays for all the tokens all over again. The fix (incremental indexing) comes in later modules.

Hidden cost 2: Wasted context window

You send 3,000 tokens of RAG context to get back a 10-token answer. Those 3,000 tokens cost money even though the model only uses a fraction of them to answer.

def analyze_context_waste(
    context_tokens: int,
    other_input_tokens: int,
    output_tokens: int,
    useful_context_ratio: float = 0.3,
    input_price_per_1m: float = 2.50,
    output_price_per_1m: float = 10.00
) -> dict:
    """Analyze context window waste."""
    total_input = context_tokens + other_input_tokens
    input_cost = (total_input / 1_000_000) * input_price_per_1m
    output_cost = (output_tokens / 1_000_000) * output_price_per_1m
    total_cost = input_cost + output_cost

    wasted = int(context_tokens * (1 - useful_context_ratio))
    wasted_cost = (wasted / 1_000_000) * input_price_per_1m

    return {
        "total_cost": round(total_cost, 8),
        "wasted_tokens": wasted,
        "wasted_cost": round(wasted_cost, 8),
        "waste_pct": round(wasted_cost / total_cost * 100, 1),
    }

# GPT-4o: 3K RAG context, 550 other input tokens, 200 output
result = analyze_context_waste(3_000, 550, 200, useful_context_ratio=0.3)
print(f"Total cost/request:  ${result['total_cost']:.6f}")
print(f"Wasted tokens:       {result['wasted_tokens']:,}")
print(f"Wasted cost:         ${result['wasted_cost']:.6f} ({result['waste_pct']}% of the request)")

# Expected output:
Total cost/request:  $0.010875
Wasted tokens:       2,100
Wasted cost:         $0.005250 (48.3% of the request)

At monthly scale

requests_day = 10_000
monthly_waste = result["wasted_cost"] * requests_day * 30
print(f"10K requests/day → ${monthly_waste:,.2f}/month wasted on irrelevant context")

# Expected output:
10K requests/day → $1,575.00/month wasted on irrelevant context

$1,575/month in tokens the model doesn't even need. The fixes (reranking, chunk filtering) are covered in modules 3 and 5-6.

Hidden cost 3: Retries from rate limiting

When a request fails because of a rate limit (HTTP 429), the standard logic retries it. Every retry costs the same as the original request but delivers no additional value.

import random

def simulate_retry_costs(
    total_requests: int,
    cost_per_request: float,
    failure_rate: float = 0.07,
    max_retries: int = 3
) -> dict:
    """Simulate the extra cost from retries."""
    random.seed(42)
    total_api_calls = 0

    for _ in range(total_requests):
        for attempt in range(max_retries + 1):
            total_api_calls += 1
            if random.random() >= failure_rate:
                break

    extra = total_api_calls - total_requests
    overhead = extra * cost_per_request
    return {
        "total_api_calls": total_api_calls,
        "extra_calls": extra,
        "intended_cost": round(total_requests * cost_per_request, 2),
        "actual_cost": round(total_api_calls * cost_per_request, 2),
        "overhead": round(overhead, 2),
        "overhead_pct": round(extra / total_requests * 100, 1),
    }

result = simulate_retry_costs(10_000, 0.01, failure_rate=0.07)
print(f"Requests: {result['total_api_calls']:,} actual vs {10_000:,} planned (+{result['extra_calls']:,})")
print(f"Cost: ${result['actual_cost']:.2f} actual vs ${result['intended_cost']:.2f} planned")
print(f"Overhead: ${result['overhead']:.2f}/day ({result['overhead_pct']}%)")
print(f"Monthly overhead: ${result['overhead'] * 30:.2f}")

# Expected output:
Requests: 10,707 actual vs 10,000 planned (+707)
Cost: $107.07 actual vs $100.00 planned
Overhead: $7.07/day (7.1%)
Monthly overhead: $212.10

$212/month in duplicated requests that deliver no value. The most effective fix isn't a better retry strategy — it's cutting requests with caching (modules 4-6).

Hidden cost 4: Development and testing costs

Every API call during development costs money. "I'm just testing" with GPT-4o can get surprisingly expensive.

def estimate_dev_costs(
    developers: int,
    calls_per_dev_day: int,
    avg_input: int, avg_output: int,
    model_prices: dict,
    working_days: int = 22
) -> dict:
    cost_per_call = (
        (avg_input / 1_000_000) * model_prices["input"]
        + (avg_output / 1_000_000) * model_prices["output"]
    )
    monthly = cost_per_call * developers * calls_per_dev_day * working_days
    return {"cost_per_call": round(cost_per_call, 6), "monthly": round(monthly, 2)}

gpt4o = estimate_dev_costs(3, 100, 1_000, 500, {"input": 2.50, "output": 10.00})
mini = estimate_dev_costs(3, 100, 1_000, 500, {"input": 0.15, "output": 0.60})

print(f"3 devs × 100 calls/day:")
print(f"  With GPT-4o:      ${gpt4o['monthly']:.2f}/month")
print(f"  With GPT-4o-mini: ${mini['monthly']:.2f}/month")
print(f"  Savings:          ${gpt4o['monthly'] - mini['monthly']:.2f}/month")

# Expected output:
3 devs × 100 calls/day:
  With GPT-4o:      $49.50/month
  With GPT-4o-mini: $2.97/month
  Savings:          $46.53/month

The golden rule: use the cheapest model for development and testing. Only use the production model for final validation.

Hidden cost 5: Fine-tuning — amortized costs

Fine-tuning has two costs many people ignore: the training cost, and an inference premium (fine-tuned models cost more per token than the base ones).

def calculate_finetuning_costs(
    training_tokens: int, epochs: int,
    requests_day: int, avg_in: int, avg_out: int,
    months: int = 12
) -> dict:
    # GPT-4o-mini fine-tuning pricing
    train_price = 3.00     # per 1M training tokens
    ft_input = 0.30        # 2x base
    ft_output = 1.20       # 2x base
    base_input = 0.15
    base_output = 0.60

    training_cost = (training_tokens * epochs / 1_000_000) * train_price
    ft_monthly = ((avg_in/1e6)*ft_input + (avg_out/1e6)*ft_output) * requests_day * 30
    base_monthly = ((avg_in/1e6)*base_input + (avg_out/1e6)*base_output) * requests_day * 30

    return {
        "training_cost": round(training_cost, 2),
        "ft_monthly_inference": round(ft_monthly, 2),
        "base_monthly_inference": round(base_monthly, 2),
        "premium_monthly": round(ft_monthly - base_monthly + training_cost/months, 2),
    }

ft = calculate_finetuning_costs(500_000, 3, 5_000, 500, 200)
print(f"Training (one-off):    ${ft['training_cost']:.2f}")
print(f"FT inference/month:    ${ft['ft_monthly_inference']:.2f}")
print(f"Base inference/month:  ${ft['base_monthly_inference']:.2f}")
print(f"Total premium/month:   ${ft['premium_monthly']:.2f}")

# Expected output:
Training (one-off):    $4.50
FT inference/month:    $58.50
Base inference/month:  $29.25
Total premium/month:   $29.62

Fine-tuning doubles your inference cost. It's only worth it if the quality improvement justifies that premium.

Hidden cost 6: Streaming and disconnections

Streaming doesn't cost more in tokens, but if the connection drops mid-stream, you lose the partial response and pay again on the retry.

def streaming_overhead(requests_day: int, cost_per_req: float, disconnect_rate: float = 0.02):
    disconnects = int(requests_day * disconnect_rate)
    # You pay for the lost partial tokens + the full retry
    daily_overhead = disconnects * cost_per_req * 1.7
    return {"disconnects_day": disconnects, "overhead_monthly": round(daily_overhead * 30, 2)}

s = streaming_overhead(10_000, 0.01, 0.02)
print(f"Disconnects/day:   {s['disconnects_day']}")
print(f"Monthly overhead:  ${s['overhead_monthly']:.2f}")

# Expected output:
Disconnects/day:   200
Monthly overhead:  $102.00

Hidden cost detector: the integrating function

from dataclasses import dataclass

@dataclass
class SystemConfig:
    name: str
    model: str
    requests_per_day: int
    avg_input_tokens: int
    avg_output_tokens: int
    has_rag: bool = False
    rag_documents: int = 0
    rag_tokens_per_doc: int = 0
    rag_reindex_days: int = 30
    rag_context_tokens: int = 0
    useful_context_ratio: float = 0.5
    failure_rate: float = 0.04
    uses_streaming: bool = False
    disconnect_rate: float = 0.02
    developers: int = 0
    dev_calls_per_day: int = 0

@dataclass
class CostAlert:
    category: str
    severity: str
    monthly_cost: float
    recommendation: str

def detect_hidden_costs(cfg: SystemConfig) -> list[CostAlert]:
    """Analyze a system and detect hidden costs."""
    alerts = []
    prices = {
        "gpt-4o": (2.50, 10.00), "gpt-4o-mini": (0.15, 0.60),
        "claude-3.5-sonnet": (3.00, 15.00), "claude-3-haiku": (0.25, 1.25),
    }
    inp_price, out_price = prices.get(cfg.model, (2.50, 10.00))
    base_cost = (cfg.avg_input_tokens/1e6)*inp_price + (cfg.avg_output_tokens/1e6)*out_price

    # 1. RAG embeddings
    if cfg.has_rag and cfg.rag_documents > 0:
        c = calculate_embedding_costs(
            cfg.rag_documents, cfg.rag_tokens_per_doc,
            cfg.requests_per_day, reindex_frequency_days=cfg.rag_reindex_days
        )
        sev = "high" if c["total_monthly"] > 100 else "medium" if c["total_monthly"] > 10 else "low"
        alerts.append(CostAlert("Embeddings (RAG)", sev, c["total_monthly"],
                                "Incremental indexing, use embedding-3-small"))

    # 2. Context waste
    if cfg.rag_context_tokens > 0:
        wasted = cfg.rag_context_tokens * (1 - cfg.useful_context_ratio)
        waste_monthly = (wasted/1e6) * inp_price * cfg.requests_per_day * 30
        sev = "high" if waste_monthly > 500 else "medium" if waste_monthly > 50 else "low"
        alerts.append(CostAlert("Context window waste", sev, round(waste_monthly, 2),
                                "Implement reranking, reduce chunks"))

    # 3. Retries
    if cfg.failure_rate > 0:
        retry_monthly = base_cost * cfg.requests_per_day * cfg.failure_rate * 1.1 * 30
        sev = "high" if retry_monthly > 200 else "medium" if retry_monthly > 20 else "low"
        alerts.append(CostAlert("Retries", sev, round(retry_monthly, 2),
                                "Caching to cut total requests"))

    # 4. Streaming
    if cfg.uses_streaming and cfg.disconnect_rate > 0:
        s_monthly = base_cost * cfg.requests_per_day * cfg.disconnect_rate * 1.7 * 30
        sev = "medium" if s_monthly > 50 else "low"
        alerts.append(CostAlert("Streaming disconnects", sev, round(s_monthly, 2),
                                "Save partial tokens"))

    # 5. Dev costs
    if cfg.developers > 0:
        dev_monthly = base_cost * cfg.developers * cfg.dev_calls_per_day * 22
        sev = "medium" if dev_monthly > 30 else "low"
        alerts.append(CostAlert("Development/Testing", sev, round(dev_monthly, 2),
                                "Use a cheap model for dev"))

    alerts.sort(key=lambda a: a.monthly_cost, reverse=True)
    return alerts

# Analyze a real system
system = SystemConfig(
    name="Customer Support Bot", model="gpt-4o",
    requests_per_day=10_000, avg_input_tokens=1_500, avg_output_tokens=500,
    has_rag=True, rag_documents=50_000, rag_tokens_per_doc=2_000,
    rag_reindex_days=7, rag_context_tokens=3_000, useful_context_ratio=0.3,
    failure_rate=0.07, uses_streaming=True, disconnect_rate=0.03,
    developers=3, dev_calls_per_day=80,
)

alerts = detect_hidden_costs(system)
print(f"=== Detector: {system.name} ===\n")
total = 0
for a in alerts:
    icon = {"high": "🔴", "medium": "🟡", "low": "🟢"}[a.severity]
    print(f"{icon} {a.category:<28} ${a.monthly_cost:>10,.2f}/month → {a.recommendation}")
    total += a.monthly_cost
print(f"\nTOTAL hidden costs: ${total:,.2f}/month")

# Expected output:
=== Detector: Customer Support Bot ===

🔴 Context window waste          $  1,575.00/month → Implement reranking, reduce chunks
🔴 Retries                       $    202.13/month → Caching to cut total requests
🟡 Streaming disconnects         $    133.88/month → Save partial tokens
🟡 Development/Testing           $     46.20/month → Use a cheap model for dev
🟢 Embeddings (RAG)              $      8.87/month → Incremental indexing, use embedding-3-small

TOTAL hidden costs: $1,966.08/month

Comparison: Visible vs hidden costs

Connection to the project

The hidden cost detector plugs straight into the Cost Breakdown Calculator. Your final calculator (capsule 08) must include visible AND hidden costs, automatically detect which ones apply based on the architecture, and generate alerts with recommendations. Without this, your baseline would underestimate real spend.

Troubleshooting

Problem 1: "I don't know what percentage of my RAG context is useful"

Cause: There are no relevance metrics for the retrieved context. Fix: Compare answers with full context vs reduced context (top 1-2 chunks vs top 5). If quality doesn't drop, you're sending unnecessary context.

Problem 2: "I have no visibility into how many retries my system makes"

Cause: The retry logic in the client library isn't logged. Fix: Wrap your API calls in a wrapper that counts attempts and logs each one.

Problem 3: "Embedding costs look irrelevant ($0.02/1M)"

Cause: Thinking about unit price without calculating total volume. Fix: Calculate docs × tokens_per_doc × price × reindexes/month. The numbers are surprising at scale.

Problem 4: "My developers use GPT-4o for everything"

Cause: No model usage policy for development. Fix: An environment variable DEV_MODEL=gpt-4o-mini. The difference is 16x in cost.

Exercises

Exercise 1: Identify hidden costs in an architecture (Easy)

Read this description: "Chatbot with RAG. GPT-4o for everything. 20K docs re-indexed weekly. Top 10 chunks as context (~5,000 tokens). 3 developers testing with GPT-4o. Streaming enabled. 8% of requests fail and get retried." List every hidden cost.

Exercise 2: Calculate embeddings for RAG (Easy)

100,000 documents of 1,500 tokens. You re-index every 3 days. 20,000 queries/day. Calculate the monthly cost with text-embedding-3-small and text-embedding-3-large.

for name, price in [("small", 0.02), ("large", 0.13)]:
    c = calculate_embedding_costs(100_000, 1_500, 20_000, price_per_1m=price, reindex_frequency_days=3)
    print(f"text-embedding-3-{name}: ${c['total_monthly']:.2f}/month")

# Expected output:
text-embedding-3-small: $30.60/month
text-embedding-3-large: $198.90/month

Exercise 3: Estimate retry cost (Medium)

50,000 requests/day to GPT-4o-mini (600 in + 200 out tokens). 10% failure rate. Calculate: actual API calls, monthly overhead, and percentage of total cost.

cost_per_req = (600/1e6)*0.15 + (200/1e6)*0.60  # $0.00021
r = simulate_retry_costs(50_000, cost_per_req, failure_rate=0.10)
print(f"Actual API calls: {r['total_api_calls']:,} (vs 50,000 planned)")
print(f"Monthly overhead: ${r['overhead'] * 30:.2f}")
print(f"Percentage: {r['overhead_pct']}%")

Exercise 4: Context optimization — how much can you save (Medium)

GPT-4o, 15,000 requests/day. Currently 4,000 tokens of RAG context with 25% useful. If you cut it down to just the 1,000 useful tokens, how much do you save annually?

other_input = 500  # system prompt + query
output = 300

# Current
current = ((4_000+other_input)/1e6)*2.50 + (output/1e6)*10.00
# Optimized
optimized = ((1_000+other_input)/1e6)*2.50 + (output/1e6)*10.00

monthly_savings = (current - optimized) * 15_000 * 30
print(f"Current/req:     ${current:.6f}")
print(f"Optimized/req:   ${optimized:.6f}")
print(f"Monthly savings: ${monthly_savings:,.2f}")
print(f"Annual savings:  ${monthly_savings * 12:,.2f}")

# Expected output:
Current/req:     $0.014250
Optimized/req:   $0.006750
Monthly savings: $3,375.00
Annual savings:  $40,500.00

Exercise 5: Full audit (Hard)

Configure detect_hidden_costs() for: "E-commerce AI assistant. Claude 3.5 Sonnet. 30K req/day. 2,000 in + 1,000 out tokens. RAG with 200K docs (1,500 tokens), re-indexed daily. 8,000 context tokens, 40% useful. 4% failure rate. Streaming with 4% disconnects. 5 devs, 120 calls/day."

ecom = SystemConfig(
    name="E-commerce Assistant", model="claude-3.5-sonnet",
    requests_per_day=30_000, avg_input_tokens=2_000, avg_output_tokens=1_000,
    has_rag=True, rag_documents=200_000, rag_tokens_per_doc=1_500,
    rag_reindex_days=1, rag_context_tokens=8_000, useful_context_ratio=0.4,
    failure_rate=0.04, uses_streaming=True, disconnect_rate=0.04,
    developers=5, dev_calls_per_day=120,
)

# Visible costs
visible = ((2_000/1e6)*3.00 + (1_000/1e6)*15.00) * 30_000 * 30
alerts = detect_hidden_costs(ecom)
hidden = sum(a.monthly_cost for a in alerts)

print(f"Visible:  ${visible:,.2f}/month")
print(f"Hidden:   ${hidden:,.2f}/month")
print(f"TOTAL:    ${visible + hidden:,.2f}/month")
print(f"% hidden: {hidden/(visible+hidden)*100:.1f}%")

Exercise 6: Prioritized reduction plan (Hard)

Using the results from Exercise 5, order the reduction actions by impact/difficulty and estimate the total savings possible.

plan = [
    ("Cut context 8K→3K (reranking)",          "60% of context waste",      "Medium", 1),
    ("Exact + semantic caching",                "40% of all requests",       "Medium", 2),
    ("Incremental embedding indexing",          "80% of indexing cost",      "Low",    3),
    ("GPT-4o-mini for development",             "90% of dev costs",          "Low",    4),
    ("Save partial streaming tokens",           "50% of streaming overhead", "Medium", 5),
]

print(f"{'#':>2} {'Action':<45} {'Estimated savings':<25} {'Difficulty'}")
print("-" * 85)
for action, savings, diff, prio in plan:
    print(f"{prio:>2}. {action:<45} {savings:<25} {diff}")
print(f"\nOrder: highest impact + lowest difficulty first")

Summary

• ✅ Hidden costs account for 25-40% of a typical AI invoice
• ✅ RAG embeddings pile up with frequent re-indexing and document volume
• ✅ Wasted context window is the biggest hidden cost in RAG systems
• ✅ Retries from rate limiting double costs without delivering value — 10% failure ≈ 10% overhead
• ✅ Development costs: "just testing" with GPT-4o costs 16x more than with mini
• ✅ Fine-tuning carries a 2x inference premium that few people budget for
• ✅ The hidden cost detector analyzes architecture and flags money leaks automatically

Next capsule: Cost breakdown by operation type — exact formulas for completions, embeddings and fine-tuning.

Additional resources

1. OpenAI Rate Limits - Rate limits and how to handle them
2. OpenAI Embeddings Guide - Embeddings guide with pricing
3. OpenAI Fine-Tuning Pricing - Training and inference costs
4. Anthropic Rate Limits - Limits at Anthropic
5. Cohere Rerank - Reranking to cut unnecessary context
6. LangChain Cost Tracking - Token tracking in LangChain
7. Redis for AI Caching - Caching patterns to cut API calls
8. OpenAI Cookbook: Token Counting - Token counting and estimation

Created: March 2026 Version: 1.0