2. Zero-Shot CoT: "Let's Think Step by Step"

Overview

The simplest CoT technique: adding "Let's think step by step" (or variants in Spanish/English) at the end of the prompt. It requires no examples. In this capsule you'll learn the most effective variants, how they work internally, benchmarks on real datasets, and how to extract the final answer from the generated reasoning.

Estimated time: 60-75 minutes

What Is Zero-Shot CoT and Why Does It Work?

Zero-Shot means you provide no examples in the prompt. You just add an instruction that switches on the model's explicit reasoning mode.

The phrase "Let's think step by step" was discovered by Kojima et al. (2022) almost by accident during prompt experiments. What was surprising is that a single sentence added at the end dramatically improved accuracy across multiple benchmarks with no other change.

Why does it work? There are two main theories:

1. Pattern-activation theory: The model has seen millions of educational texts where phrases like "step by step" precede detailed, correct explanations. By generating that phrase, it activates that statistical pattern.

2. Extended working-memory theory: By forcing the model to write intermediate steps, those steps stay in the context (the transformer's "window") and act as external memory, reducing calculation errors.

Base Implementation

from openai import OpenAI

client = OpenAI()  # Requires OPENAI_API_KEY in your environment variables

def solve_with_cot(question: str, language: str = "en") -> str:
    """
    Solves a question using Zero-Shot CoT.
    
    Args:
        question: The problem to solve
        language: "es" for Spanish, "en" for English
    
    Returns:
        The model's answer, reasoning included
    """
    if language == "es":
        cot_instruction = "Piensa paso a paso."
    else:
        cot_instruction = "Let's think step by step."
    
    prompt = f"{question}\n\n{cot_instruction}"
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        temperature=0,      # 0 for maximum consistency in the reasoning
        max_tokens=800      # Enough for reasoning + answer
    )
    return response.choices[0].message.content


# Basic usage example
if __name__ == "__main__":
    problem = """
    A kilo of apples costs $3.50. If I buy 2.5 kilos and pay with a $20 
    bill, how much change do I get?
    """
    
    result = solve_with_cot(problem)
    print(result)

Expected output:

To solve this I need to:

Step 1: Calculate the cost of 2.5 kilos of apples
- Price per kilo: $3.50
- Kilos bought: 2.5
- Total cost: 3.50 × 2.5 = $8.75

Step 2: Calculate the change
- Bill: $20.00
- Cost: $8.75
- Change: 20.00 - 8.75 = $11.25

Therefore, you get $11.25 in change.

Effective Variants of Zero-Shot CoT

Not every trigger phrase is equally effective. Here are the tested variants with their characteristics:

COT_VARIANTS = {
    # English variants (the originals)
    "classic_en": "Let's think step by step.",
    "careful_en": "Let's work through this carefully.",
    "reasoned_en": "Reason through this step by step.",
    "detailed_en": "Think step by step and explain each step.",
    
    # Spanish variants
    "classic_es": "Piensa paso a paso.",
    "detailed_es": "Razona paso a paso, explicando cada etapa.",
    "verified_es": "Piensa paso a paso y verifica tu respuesta al final.",
    "structured_es": "Paso a paso:",
    "careful_es": "Analiza esto cuidadosamente, paso a paso.",
    
    # Domain-specialized variants
    "math": "Show every math operation step by step.",
    "logic": "Identify the premises and reason toward the conclusion step by step.",
    "code": "Trace the execution of the code step by step.",
    "decision": "Evaluate each relevant factor step by step before deciding.",
}

def test_variants(problem: str, variants: list[str]) -> dict:
    """Tests multiple CoT variants on the same problem."""
    results = {}
    
    for name, variant in variants:
        prompt = f"{problem}\n\n{variant}"
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": prompt}],
            temperature=0,
            max_tokens=600
        )
        results[name] = response.choices[0].message.content
    
    return results

Variant Comparison Table

The Two-Step Protocol

Kojima et al. also proposed an important variation: the two-step protocol (two-stage prompting):

Step 1: Get the reasoning

step1_prompt = f"{question}\n\nLet's think step by step."
reasoning = get_answer(step1_prompt)

Step 2: Extract the final answer

step2_prompt = f"""
{question}

{reasoning}

Therefore, the final answer is:
"""
final_answer = get_answer(step2_prompt)

This protocol is especially useful when the reasoning is long and the final answer can get lost in the text.

def zero_shot_cot_two_steps(question: str) -> dict:
    """
    Implements the two-step protocol from Kojima et al.
    
    Returns:
        dict with 'reasoning' and 'final_answer'
    """
    # Step 1: Generate the reasoning
    reasoning_prompt = f"{question}\n\nLet's think step by step."
    
    reasoning_response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": reasoning_prompt}],
        temperature=0,
        max_tokens=600
    )
    reasoning = reasoning_response.choices[0].message.content
    
    # Step 2: Extract the final answer
    extraction_prompt = f"""
{question}

{reasoning}

Therefore, the final answer is:
"""
    
    answer_response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": extraction_prompt}],
        temperature=0,
        max_tokens=100
    )
    final_answer = answer_response.choices[0].message.content
    
    return {
        "reasoning": reasoning,
        "final_answer": final_answer,
        "total_tokens": (
            reasoning_response.usage.total_tokens + 
            answer_response.usage.total_tokens
        )
    }


# Usage example
if __name__ == "__main__":
    problem = """
    Alice, Bob and Carol work together on a project. 
    Alice completes 1/3 of the work per day. Bob completes 1/4 per day.
    Carol completes 1/6 per day. How many days do they need working together?
    """
    
    result = zero_shot_cot_two_steps(problem)
    print("=== REASONING ===")
    print(result["reasoning"])
    print("\n=== FINAL ANSWER ===")
    print(result["final_answer"])
    print(f"\nTokens used: {result['total_tokens']}")

Extracting the Final Answer

When you use Zero-Shot CoT, the model mixes reasoning and answer together. You need to pull out just the final answer:

import re
from openai import OpenAI

client = OpenAI()

def extract_numeric_answer(cot_output: str) -> str | None:
    """
    Extracts the final number from a CoT answer.
    
    Looks for patterns such as:
    - "Por lo tanto, X"
    - "La respuesta es X"
    - "Therefore, X"
    - "= X" at the end
    """
    patterns = [
        r'(?:por lo tanto|therefore|la respuesta (?:final )?es|the answer is)[,:]?\s*([-\d,.$€%\.]+)',
        r'(?:resultado|result|total)[:\s]+\$?([-\d,\.]+)',
        r'=\s*([-\d,\.]+)\s*$',
        r'(?:final answer|respuesta final)[:\s]+\$?([-\d,\.]+)',
    ]
    
    output_lower = cot_output.lower()
    
    for pattern in patterns:
        match = re.search(pattern, output_lower, re.IGNORECASE | re.MULTILINE)
        if match:
            return match.group(1).strip()
    
    # Fallback: the last number in the text
    numbers = re.findall(r'\b\d+(?:[.,]\d+)?\b', cot_output)
    return numbers[-1] if numbers else None


def extract_boolean_answer(cot_output: str) -> str | None:
    """
    Extracts a yes/no or valid/invalid answer from a CoT response.
    """
    output_lower = cot_output.lower()
    
    # Search the last part of the text (where the conclusion usually lives)
    last_part = output_lower[-300:]
    
    affirmative_patterns = ['válido', 'correcto', 'sí', 'verdadero', 'true', 'valid', 'yes']
    negative_patterns = ['inválido', 'incorrecto', 'no', 'falso', 'false', 'invalid']
    
    for pattern in affirmative_patterns:
        if pattern in last_part:
            return "YES/VALID"
    
    for pattern in negative_patterns:
        if pattern in last_part:
            return "NO/INVALID"
    
    return None


def solve_with_extraction(question: str, type: str = "numeric") -> dict:
    """
    Solves with CoT and extracts the answer cleanly.
    
    Args:
        question: The problem
        type: "numeric", "boolean", or "text"
    
    Returns:
        dict with 'reasoning', 'extracted_answer', 'raw_answer'
    """
    prompt = f"{question}\n\nLet's think step by step. At the end, write 'Final answer: [X]'"
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        temperature=0,
        max_tokens=700
    )
    
    output = response.choices[0].message.content
    
    if type == "numeric":
        answer = extract_numeric_answer(output)
    elif type == "boolean":
        answer = extract_boolean_answer(output)
    else:
        # For the text type, take the last line
        lines = [l.strip() for l in output.split('\n') if l.strip()]
        answer = lines[-1] if lines else output
    
    return {
        "reasoning": output,
        "extracted_answer": answer,
        "raw_answer": output[-200:]  # Last 200 chars for debugging
    }

Improvement Benchmarks

These are the documented results of Zero-Shot CoT vs. standard prompting:

GSM8K Dataset (Grade School Math)

MultiArith Dataset

SVAMP Dataset (variations on arithmetic word problems)

An important observation: The more capable models (GPT-4) already have high accuracy without CoT on simple tasks. The gain from CoT is larger on mid-sized models and on more complex problems.

When Zero-Shot CoT Works

✅ Works well on:

# Multi-step arithmetic
arithmetic_problem = """
A store has a 20% discount on every product. 
A TV costs $450 before the discount. On top of that there's a 16% sales tax.
How much do you pay in total?
"""

# Sequential logical reasoning
logic_problem = """
If it's cloudy, it may rain. If it rains, John takes an umbrella.
Today John is NOT taking an umbrella. Is it cloudy? Is it raining?
"""

# Sequence/pattern problems
pattern_problem = """
The sequence is: 2, 6, 18, 54, ...
What is the next number? And the tenth number in the sequence?
"""

# Speed, time and distance problems
physics_problem = """
Two cyclists set off at the same time from cities 120 km apart.
The first rides at 25 km/h. The second at 35 km/h. When do they meet?
"""

❌ Doesn't work well on:

# Creativity
no_cot_1 = "Write a poem about the sea"
# CoT generates: "Step 1: Pick a theme. Step 2: Look for rhymes..." → a mechanical poem

# Opinion/preferences
no_cot_2 = "Python or JavaScript for my project?"
# There's no 'correct' reasoning, it depends on the context

# Direct factual questions
no_cot_3 = "What year was Google founded?"
# The answer is 1998. CoT only adds tokens: "Google is a company... founded in... 1998"

# Direct translation
no_cot_4 = "Translate 'Hello World' into Spanish"
# "Hola Mundo". There are no steps to show.

Self-Consistency: An Advanced Upgrade to Zero-Shot CoT

A powerful extension: generate multiple CoT answers with temperature > 0 and pick the most common one (majority vote).

from collections import Counter
from openai import OpenAI

client = OpenAI()

def zero_shot_cot_self_consistency(
    question: str, 
    n_samples: int = 5,
    temperature: float = 0.7
) -> dict:
    """
    Self-Consistency: generates N reasoning chains and votes for the most frequent answer.
    
    Reference: Wang et al., 2022 - "Self-Consistency Improves CoT Reasoning"
    
    Args:
        question: The problem to solve
        n_samples: Number of reasoning paths to generate
        temperature: Higher temperature = more diverse paths
    
    Returns:
        dict with every answer and the majority answer
    """
    answers = []
    reasonings = []
    
    prompt = f"{question}\n\nLet's think step by step. At the end write 'Answer: [X]'"
    
    for i in range(n_samples):
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": prompt}],
            temperature=temperature,
            max_tokens=600
        )
        output = response.choices[0].message.content
        reasonings.append(output)
        
        # Extract the final answer
        answer = extract_numeric_answer(output)
        answers.append(answer)
    
    # Count votes (ignore None)
    votes = Counter([r for r in answers if r is not None])
    winning_answer = votes.most_common(1)[0][0] if votes else None
    
    return {
        "final_answer": winning_answer,
        "vote_distribution": dict(votes),
        "confidence": votes[winning_answer] / n_samples if winning_answer else 0,
        "all_reasonings": reasonings,
        "all_answers": answers
    }


# Usage example
if __name__ == "__main__":
    problem = """
    Carl buys stock: 100 shares at $15 each.
    He sells them when they go up 30%. What is his total profit?
    """
    
    result = zero_shot_cot_self_consistency(problem, n_samples=5)
    print(f"Final answer (majority): {result['final_answer']}")
    print(f"Vote distribution: {result['vote_distribution']}")
    print(f"Confidence: {result['confidence']:.0%}")

When to use Self-Consistency:

• Critical problems where an error is expensive
• When accuracy matters more than latency/cost
• With temperature 0.5-0.8 for path diversity

Full Comparison: No CoT vs. Zero-Shot CoT vs. Self-Consistency

def full_benchmark(
    problems: list[tuple[str, str]]  # (problem, correct_answer)
) -> dict:
    """
    Compares three approaches on a set of problems.
    """
    results = {
        "without_cot": {"correct": 0, "avg_tokens": 0},
        "zero_shot_cot": {"correct": 0, "avg_tokens": 0},
        "self_consistency": {"correct": 0, "avg_tokens": 0}
    }
    
    total_tokens = {k: [] for k in results}
    
    for problem, correct_answer in problems:
        # Approach 1: No CoT
        r1 = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": problem}],
            temperature=0, max_tokens=100
        )
        answer1 = r1.choices[0].message.content
        if correct_answer in answer1:
            results["without_cot"]["correct"] += 1
        total_tokens["without_cot"].append(r1.usage.total_tokens)
        
        # Approach 2: Zero-Shot CoT
        r2 = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": f"{problem}\n\nLet's think step by step."}],
            temperature=0, max_tokens=500
        )
        answer2 = extract_numeric_answer(r2.choices[0].message.content)
        if answer2 and correct_answer in str(answer2):
            results["zero_shot_cot"]["correct"] += 1
        total_tokens["zero_shot_cot"].append(r2.usage.total_tokens)
        
        # Approach 3: Self-Consistency (3 samples for this benchmark)
        r3 = zero_shot_cot_self_consistency(problem, n_samples=3)
        if r3["final_answer"] and correct_answer in str(r3["final_answer"]):
            results["self_consistency"]["correct"] += 1
        total_tokens["self_consistency"].append(
            sum(600 for _ in range(3))  # Approximation
        )
    
    n = len(problems)
    for method in results:
        results[method]["accuracy"] = results[method]["correct"] / n
        results[method]["avg_tokens"] = sum(total_tokens[method]) / n
    
    return results

Troubleshooting

Problem 1: The model ignores the CoT instruction

Symptoms: The model answers directly without showing steps.

Causes and fixes:

# ❌ CoT at the start can get ignored in long prompts
bad_prompt = "Let's think step by step. Solve: 17 × 23"

# ✅ CoT at the end of the prompt has more effect
good_prompt = "Solve: 17 × 23\n\nLet's think step by step."

# ✅ Alternatively, use it as a system message
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {
            "role": "system", 
            "content": "Always reason step by step before giving your final answer."
        },
        {"role": "user", "content": "Solve: 17 × 23"}
    ],
    temperature=0
)

Problem 2: The reasoning is way too long

Symptoms: The model generates 10+ paragraphs for a simple problem.

Fixes:

# ✅ Cap the steps explicitly
prompt = f"""
{question}

Let's think step by step. Maximum 5 concise steps.
"""

# ✅ Ask for a structured format
prompt = f"""
{question}

Solve in this format:
Step 1: [action]
Step 2: [action]
...
Final answer: [X]
"""

Problem 3: Wrong answer but correct reasoning

Symptoms: The steps are right but the final answer gets extracted wrong.

# ✅ Add an explicit instruction for the final answer format
prompt = f"""
{question}

Let's think step by step.
At the end, write EXACTLY this line:
ANSWER: [number with no extra text]
"""

# ✅ Then extract with a simple regex
import re

def extract_labeled_answer(output: str) -> str | None:
    match = re.search(r'ANSWER:\s*([\d,.$€%\.+-]+)', output, re.IGNORECASE)
    return match.group(1).strip() if match else None

Problem 4: CoT produces different answers on every call

Symptoms: With temperature=0 there's still variation.

# ✅ Use a seed for reproducibility (when available)
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[...],
    temperature=0,
    seed=42  # For reproducibility
)

# ✅ For critical problems, use Self-Consistency
result = zero_shot_cot_self_consistency(problem, n_samples=7)

Exercises

Exercise 1: Compare with/without CoT on 5 arithmetic problems

Run the following code, record the results, and compute the accuracy of each approach.

from openai import OpenAI
import re

client = OpenAI()

# Test set
problems_with_answers = [
    ("What is 15 + 27?", "42"),
    ("What is 17 × 23?", "391"),
    ("If I have $150 and spend 40%, how much is left?", "90"),
    ("An 8-slice pizza is shared among 3 people. How many whole slices does each get, and how many are left over?", "2"),
    ("A car travels at 80 km/h for 2.5 hours. How many kilometers does it cover?", "200"),
]

# Your task: complete this code
def solve(question: str, cot: bool) -> str:
    # TODO: Implement
    pass

# Count the hits for each approach

from openai import OpenAI
import re

client = OpenAI()

problems_with_answers = [
    ("What is 15 + 27?", "42"),
    ("What is 17 × 23?", "391"),
    ("If I have $150 and spend 40%, how much is left?", "90"),
    ("An 8-slice pizza is shared among 3 people. How many whole slices does each get, and how many are left over?", "2"),
    ("A car travels at 80 km/h for 2.5 hours. How many kilometers does it cover?", "200"),
]

def solve(question: str, cot: bool) -> str:
    content = question
    if cot:
        content += "\n\nLet's think step by step. At the end write 'Answer: [number]'"
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": content}],
        temperature=0,
        max_tokens=500 if cot else 50
    )
    return response.choices[0].message.content

correct_without = 0
correct_with = 0

for question, correct in problems_with_answers:
    r_without = solve(question, cot=False)
    r_with = solve(question, cot=True)
    
    if correct in r_without:
        correct_without += 1
    if correct in r_with:
        correct_with += 1
    
    print(f"\nProblem: {question[:50]}...")
    print(f"  Without CoT: '{r_without[:60]}'")
    print(f"  With CoT: '{r_with[:60]}'")

n = len(problems_with_answers)
print(f"\n=== RESULTS ===")
print(f"Without CoT: {correct_without}/{n} ({correct_without/n:.0%})")
print(f"With CoT: {correct_with}/{n} ({correct_with/n:.0%})")

Exercise 2: Extract the final answer from a CoT output

Implement the extract_answer function so that it works correctly on the following cases:

test_cases = [
    ("Therefore, the total is 391.", "391"),
    ("The answer is 42.5 dollars", "42.5"),
    ("Final answer: $89.00", "89.00"),
    ("In conclusion, there are 15 teams.\nTotal: 15", "15"),
    ("ANSWER: 200", "200"),
]

import re

def extract_answer(output: str) -> str | None:
    """
    Extracts the final numeric answer from a CoT output.
    Handles several common formats.
    """
    # Normalize: strip extra whitespace, lowercase it for searching
    output_clean = output.strip()
    output_lower = output_clean.lower()
    
    patterns = [
        # "Answer: X" or "ANSWER: X"
        r'(?:final\s+)?answer[:\s]+\$?([\d,\.]+)',
        # "Therefore, ... X"
        r'(?:therefore|thus|in conclusion)[,.]?\s+(?:the|there|are|is)?[^\d]*([\d,\.]+)',
        # "The answer is X"
        r'(?:the answer is)[:\s]*\$?([\d,\.]+)',
        # "Total: X"
        r'total[:\s]+\$?([\d,\.]+)',
        # "= X" at end of line
        r'=\s*([\d,\.]+)\s*$',
    ]
    
    for pattern in patterns:
        match = re.search(pattern, output_lower)
        if match:
            return match.group(1).replace(',', '')
    
    # Fallback: the last number in the text
    numbers = re.findall(r'\b\d+(?:[.,]\d+)?\b', output_clean)
    if numbers:
        return numbers[-1]
    
    return None

# Check the test cases
test_cases = [
    ("Therefore, the total is 391.", "391"),
    ("The answer is 42.5 dollars", "42.5"),
    ("Final answer: $89.00", "89.00"),
    ("In conclusion, there are 15 teams.\nTotal: 15", "15"),
    ("ANSWER: 200", "200"),
]

for output, expected in test_cases:
    result = extract_answer(output)
    status = "✓" if result == expected else f"✗ (got '{result}')"
    print(f"{status} | Input: '{output[:40]}' | Expected: '{expected}'")

Exercise 3: Implement Self-Consistency with 5 samples

Implement Self-Consistency for this problem: "A salesperson earns an 8% commission on sales. In January they sold $12,500. In February they sold $9,800. How much did they earn in commissions in total?"

from openai import OpenAI
from collections import Counter
import re

client = OpenAI()

def extract_final_number(text: str) -> str | None:
    patterns = [
        r'(?:answer|total|earned)[:\s]+\$?([\d,\.]+)',
        r'\$\s*([\d,\.]+)',
        r'=\s*\$?([\d,\.]+)\s*$',
    ]
    for pattern in patterns:
        m = re.search(pattern, text.lower())
        if m:
            return m.group(1).replace(',', '')
    numbers = re.findall(r'\b\d+(?:[.,]\d+)?\b', text)
    return numbers[-1] if numbers else None

problem = """
A salesperson earns an 8% commission on sales. 
In January they sold $12,500. In February they sold $9,800. 
How much did they earn in commissions in total?
"""

prompt = f"{problem}\n\nLet's think step by step. At the end write 'Answer: $[number]'"

# Expected answer: (12500 + 9800) * 0.08 = 22300 * 0.08 = $1784

answers = []
for i in range(5):
    r = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.7,  # Some diversity for Self-Consistency
        max_tokens=400
    )
    output = r.choices[0].message.content
    answer = extract_final_number(output)
    answers.append(answer)
    print(f"Sample {i+1}: {answer}")

votes = Counter([r for r in answers if r])
winner, winner_votes = votes.most_common(1)[0]
print(f"\nFinal answer (majority): ${winner}")
print(f"Confidence: {winner_votes}/5 = {winner_votes/5:.0%}")
print(f"Correct answer: $1784")

Exercise 4: Test CoT on logical reasoning

Test this sequence of statements with and without CoT. What is the correct conclusion?

All roses are flowers.
Some flowers wilt quickly.
Do some roses necessarily wilt quickly?

Premise 1: All roses are flowers → roses ⊆ flowers
Premise 2: Some flowers wilt quickly → there is an x: flower(x) ∧ wilts_quickly(x)
Conclusion? Is there an x: rose(x) ∧ wilts_quickly(x)?

For the conclusion to hold, those "some flowers" that wilt quickly would have to belong
to the "roses" subset. But premise 2 only says that *some* flowers wilt, it doesn't say
which ones.

Counterexample: the flowers that wilt quickly are only tulips. Roses do not wilt
quickly. Premises 1 and 2 both hold, but the conclusion is false.

Therefore: INVALID / NOT necessarily.

question = """
All roses are flowers.
Some flowers wilt quickly.
Do some roses necessarily wilt quickly? Why?
"""

r_without = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": question}],
    temperature=0, max_tokens=100
)

r_with = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": question + "\n\nReason step by step."}],
    temperature=0, max_tokens=400
)

print("Without CoT:", r_without.choices[0].message.content)
print("\nWith CoT:", r_with.choices[0].message.content)

Exercise 5: Measuring the cost of Zero-Shot CoT

Compute how much it costs to use Zero-Shot CoT vs. a direct answer in production, with 10,000 daily calls to the apples problem.

from openai import OpenAI

client = OpenAI()

problem = "A kilo of apples costs $3.50. If I buy 2.5 kilos and pay with $20, how much change do I get?"

# Measure the tokens of each approach
r_direct = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": problem}],
    temperature=0, max_tokens=50
)

r_cot = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": problem + "\n\nLet's think step by step."}],
    temperature=0, max_tokens=400
)

tokens_direct = r_direct.usage.total_tokens
tokens_cot = r_cot.usage.total_tokens

# gpt-4o-mini pricing (approximate as of March 2026)
# Input: $0.15/M tokens, Output: $0.60/M tokens
price_per_token = 0.0000006  # Average of input+output

calls_per_day = 10_000
direct_cost_per_day = tokens_direct * price_per_token * calls_per_day
cot_cost_per_day = tokens_cot * price_per_token * calls_per_day

print(f"Tokens per call (direct): {tokens_direct}")
print(f"Tokens per call (CoT): {tokens_cot}")
print(f"CoT multiplier: {tokens_cot/tokens_direct:.1f}x")
print(f"\nDaily direct cost (10k calls): ${direct_cost_per_day:.2f}")
print(f"Daily CoT cost (10k calls): ${cot_cost_per_day:.2f}")
print(f"Extra cost of CoT: ${cot_cost_per_day - direct_cost_per_day:.2f}/day")
print(f"Extra cost per year: ${(cot_cost_per_day - direct_cost_per_day) * 365:.0f}")

Summary

• Zero-Shot CoT: Add "Let's think step by step" or "Piensa paso a paso" at the end of the prompt
• Typical improvement: +15-60% on math/logic depending on the complexity and the base model
• Variants: Many phrasings work; pick one based on the domain
• Two-step protocol: Reasoning → Final answer extraction
• Self-Consistency: Generate N reasoning chains and vote for the most common answer for higher accuracy
• Extraction: Use regex or an explicit instruction to get a clean answer
• Cost: 3-5x more tokens than a direct answer; use it when an error is expensive

Additional resources

1. Large Language Models are Zero-Shot Reasoners (Kojima et al., 2022)
2. Self-Consistency Improves Chain of Thought Reasoning in LLMs (Wang et al., 2022)
3. OpenAI Prompt Engineering Guide
4. Learn Prompting: Zero-Shot CoT
5. GSM8K Benchmark