Advanced OpenAI API Parameters

Capsule overview

You've seen the basic use of the OpenAI API. Now you'll learn to control the responses with advanced parameters:

• temperature (creativity)
• max_tokens (length)
• top_p (sampling)
• frequency_penalty and presence_penalty

These parameters give you fine control over how GPT generates text.

Time: 25 minutes
Difficulty: Medium

🎯 Objectives

• ✅ Master temperature (determinism vs creativity)
• ✅ Control length with max_tokens
• ✅ Understand top_p, penalties
• ✅ Choose parameters by use case

🌡️ Parameter 1: Temperature

What is it?

Controls the randomness of the responses:

• temperature=0: Deterministic (always the same)
• temperature=1: Balanced (default)
• temperature=2: Very creative (maximum)

Range: 0.0 - 2.0

Visual example:

Prompt: "Give me a name for an AI startup"

Temperature 0 (deterministic):

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    temperature=0,
    messages=[{"role": "user", "content": "Give me a name for an AI startup"}]
)

Output (always the same):

IntelliCore

Run it 10 times → Always "IntelliCore"

Temperature 1 (balanced):

temperature=1

Output (varies):

Run 1: NeuralPath
Run 2: CogniSphere
Run 3: ThinkBot AI

Temperature 2 (very creative):

temperature=2

Output (very varied, sometimes strange):

Run 1: QuantumMindGlow
Run 2: BrainyPixelForge
Run 3: ZephyrCogniVerse

When to use each value:

Test code:

from openai import OpenAI
import os
from dotenv import load_dotenv

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

def test_temperature(temp: float):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        temperature=temp,
        messages=[{"role": "user", "content": "Give me a name for an AI startup"}]
    )
    return response.choices[0].message.content

# Test 3 temperatures, 3 times each
for temp in [0, 1, 2]:
    print(f"\n=== TEMPERATURE {temp} ===")
    for i in range(3):
        result = test_temperature(temp)
        print(f"  Run {i+1}: {result}")

Run it and observe the differences.

🔢 Parameter 2: Max Tokens

What is it?

The maximum token limit in the generated response.

Important:

• 1 token ≈ 0.75 words (English)
• 1 token ≈ 0.5 words (Spanish, longer)

Example:

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    max_tokens=50,  # Maximum 50 tokens
    messages=[{"role": "user", "content": "Explain what Python is"}]
)

Output (truncated to ~50 tokens):

Python is an interpreted, high-level programming 
language, known for its clear syntax and 
readability. It is widely used in...

Note: The response is cut off if it exceeds 50 tokens.

Without max_tokens:

max_tokens=None  # Default (no explicit limit)

GPT decides when to stop (usually finishes the complete response).

When to use:

⚠️ Cost vs Max Tokens:

Example:

• Prompt: 20 tokens
• max_tokens=100 → GPT generates 80 tokens
• Cost: (20 input + 80 output) × pricing

If max_tokens=1000 but GPT only generates 80:

• Cost: (20 + 80) × pricing (same cost)
• max_tokens does NOT affect cost if GPT stops earlier

Benefit: Prevents infinite responses from a bug.

🎲 Parameter 3: Top-p (Nucleus Sampling)

What is it?

An alternative to temperature. Controls how many words GPT considers when generating text.

• top_p=0.1: Only considers the 10% most probable words (deterministic)
• top_p=1.0: Considers all words (creative)

Range: 0.0 - 1.0

Temperature vs Top-p:

Rule: Use ONE, not both.

# Option 1: Only temperature
temperature=0.7, top_p=1.0  # Default top_p

# Option 2: Only top_p
temperature=1.0, top_p=0.9  # Default temperature

Recommendation: Use temperature (more intuitive).

🚫 Parameters 4 and 5: Penalties

Frequency Penalty (penalizes repetition):

Reduces the probability of repeating words/phrases already used.

• frequency_penalty=0: No penalty (default)
• frequency_penalty=1: Maximum penalty
• frequency_penalty=2: Very aggressive (avoids almost all repetition)

Range: -2.0 to 2.0

Example:

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    frequency_penalty=1.5,
    messages=[{"role": "user", "content": "Write 5 reasons to learn Python"}]
)

Without penalty (repetitive):

1. Python is easy
2. Python is versatile
3. Python is popular
4. Python is powerful
5. Python is useful

With penalty (more varied):

1. Clear and readable syntax
2. A huge ecosystem of libraries
3. An active community and support
4. Applicable across multiple domains
5. A gentle learning curve

Presence Penalty (penalizes repeated topics):

Encourages GPT to talk about new topics.

• presence_penalty=0: No penalty (default)
• presence_penalty=1: Maximum penalty

Range: -2.0 to 2.0

Example:

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    presence_penalty=1.0,
    messages=[{"role": "user", "content": "Tell me about Python"}]
)

Without penalty: Talks only about Python
With penalty: Mentions Python but also compares it with other languages

When to use penalties:

🧪 Combined Experiments

Experiment 1: Deterministic chatbot (FAQs):

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    temperature=0,
    max_tokens=100,
    messages=[
        {"role": "system", "content": "You are a support bot. Answer concisely and consistently."},
        {"role": "user", "content": "How do I reset my password?"}
    ]
)

Result: Always the same answer (predictable for users).

Experiment 2: Creative writing:

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    temperature=1.5,
    max_tokens=500,
    frequency_penalty=1.2,
    presence_penalty=0.8,
    messages=[
        {"role": "system", "content": "You are a creative writer."},
        {"role": "user", "content": "Write the opening of a sci-fi story"}
    ]
)

Result: Very creative, varied, no repetition.

Experiment 3: Classification (yes/no):

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    temperature=0,
    max_tokens=1,  # Only "Yes" or "No"
    messages=[
        {"role": "system", "content": "Answer only 'Yes' or 'No'."},
        {"role": "user", "content": "Is Python an interpreted language?"}
    ]
)

Result: "Yes" (deterministic, minimum cost).

📊 Parameters Summary Table

🎯 Decision Guide: Which Parameters to Use

Use Case 1: FAQ Bot / Technical Support

temperature=0.0          # Deterministic
max_tokens=150           # Short answers
frequency_penalty=0      # Default
presence_penalty=0       # Default

Why: Consistency is critical (same answer every time).

Use Case 2: General Chatbot

temperature=0.7          # Slightly creative
max_tokens=None          # GPT decides
frequency_penalty=0      # Default
presence_penalty=0       # Default

Why: A balance of naturalness/consistency.

Use Case 3: Creative Writing

temperature=1.5          # Very creative
max_tokens=1000          # Long answers
frequency_penalty=1.0    # Avoids repetition
presence_penalty=0.8     # Explores diverse topics

Why: Maximizes creativity and variety.

Use Case 4: Classification (ML labeling)

temperature=0.0          # Deterministic
max_tokens=5             # Only the label
frequency_penalty=0      # Default
presence_penalty=0       # Default

Why: You need an exact and consistent answer.

🏭 Educational Code vs Production

Educational version (for learning):

from openai import OpenAI
import os
from dotenv import load_dotenv

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Simple code to understand the concept
def generate_response(prompt: str, temp: float = 0.7) -> str:
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        temperature=temp,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

Explanation: This version focuses on understanding how the basic parameters work without distractions.

Production version (for real projects):

from openai import OpenAI
import os
import time
import logging
from typing import Optional, Dict, Any
from dotenv import load_dotenv

# Setup logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

def generate_response_production(
    prompt: str,
    temperature: float = 0.7,
    max_tokens: Optional[int] = None,
    max_retries: int = 3,
    timeout: int = 30
) -> Dict[str, Any]:
    """
    Generate a response with the OpenAI API with robust error handling.
    
    Args:
        prompt: The prompt text
        temperature: Controls randomness (0-2)
        max_tokens: Token limit (None = no limit)
        max_retries: Maximum attempts if it fails
        timeout: Timeout in seconds
    
    Returns:
        Dict with response, metadata (tokens, cost, latency)
    """
    
    # Input validation
    if not prompt or not isinstance(prompt, str):
        raise ValueError("Prompt must be a non-empty string")
    
    if not 0 <= temperature <= 2:
        raise ValueError("Temperature must be between 0 and 2")
    
    if max_tokens is not None and max_tokens < 1:
        raise ValueError("max_tokens must be >= 1")
    
    # Retry logic with exponential backoff
    for attempt in range(max_retries):
        try:
            start_time = time.time()
            
            response = client.chat.completions.create(
                model="gpt-3.5-turbo",
                temperature=temperature,
                max_tokens=max_tokens,
                messages=[{"role": "user", "content": prompt}],
                timeout=timeout
            )
            
            latency = time.time() - start_time
            
            # Response validation
            if not response.choices:
                raise ValueError("API returned an empty response")
            
            # Calculate cost (February 2026 prices)
            input_tokens = response.usage.prompt_tokens
            output_tokens = response.usage.completion_tokens
            cost = (input_tokens * 0.0015 / 1000) + (output_tokens * 0.002 / 1000)
            
            # Full metadata
            result = {
                "content": response.choices[0].message.content,
                "metadata": {
                    "model": response.model,
                    "tokens": {
                        "input": input_tokens,
                        "output": output_tokens,
                        "total": response.usage.total_tokens
                    },
                    "cost_usd": round(cost, 6),
                    "latency_seconds": round(latency, 2),
                    "finish_reason": response.choices[0].finish_reason,
                    "temperature": temperature
                }
            }
            
            logger.info(
                f"OpenAI request successful | "
                f"tokens={response.usage.total_tokens} | "
                f"latency={latency:.2f}s | "
                f"cost=${cost:.6f}"
            )
            
            return result
            
        except Exception as e:
            logger.error(
                f"Attempt {attempt + 1}/{max_retries} failed: {type(e).__name__}: {e}"
            )
            
            # If it's the last attempt, propagate the error
            if attempt == max_retries - 1:
                raise RuntimeError(
                    f"Failed after {max_retries} attempts: {e}"
                ) from e
            
            # Exponential backoff: 2^attempt seconds
            wait_time = 2 ** attempt
            logger.info(f"Retrying in {wait_time}s...")
            time.sleep(wait_time)
    
    # Should never reach here
    raise RuntimeError("Unexpected error in retry logic")

# Example of production usage
if __name__ == "__main__":
    try:
        result = generate_response_production(
            prompt="Explain what FastAPI is",
            temperature=0.7,
            max_tokens=300
        )
        
        print(f"Response: {result['content']}\n")
        print(f"Tokens used: {result['metadata']['tokens']['total']}")
        print(f"Cost: ${result['metadata']['cost_usd']}")
        print(f"Latency: {result['metadata']['latency_seconds']}s")
        
    except Exception as e:
        logger.error(f"Fatal error: {e}")
        # Here you could send an alert to Sentry, Datadog, etc.

Key differences:

When to use each:

• Educational: To learn the concept, quick prototypes, one-off scripts
• Production: For APIs, applications with real users, critical systems

Important note: In Modules 7-8 you'll use patterns from the production version for the final project.

🏋️ Practical Exercises

Exercise 1: Basic temperature (Easy)

Modify the code to test 3 temperatures (0, 0.7, 1.5) with the prompt "Give me 3 names for tech products".

from openai import OpenAI
import os
from dotenv import load_dotenv

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

for temp in [0, 0.7, 1.5]:
    print(f"\n=== Temperature {temp} ===")
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        temperature=temp,
        messages=[{"role": "user", "content": "Give me 3 names for tech products"}]
    )
    print(response.choices[0].message.content)

Exercise 2: Max tokens (Easy)

Create a function that returns responses of exactly 50 tokens.

def short_response(prompt: str) -> str:
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        max_tokens=50,  # Strict limit
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

# Test
result = short_response("Explain what Python is")
print(result)
print(f"Tokens used: {len(result.split())}")  # Approximate

Exercise 3: Penalties (Medium)

Create a function that generates 10 unique ideas (without repeating concepts) using penalties.

def generate_unique_ideas(topic: str, count: int = 10) -> list:
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        temperature=1.2,  # Creative
        frequency_penalty=1.5,  # Avoids repeating words
        presence_penalty=1.0,  # Explores diverse topics
        messages=[{
            "role": "user",
            "content": f"List {count} unique ideas about: {topic}"
        }]
    )
    return response.choices[0].message.content

# Test
ideas = generate_unique_ideas("AI apps for productivity", 10)
print(ideas)

Exercise 4: Specific use case (Medium)

Configure parameters for a technical support chatbot (consistent, short answers, no creativity).

def support_bot(question: str) -> str:
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        temperature=0.0,  # Deterministic
        max_tokens=150,  # Short answers
        frequency_penalty=0,  # Default
        presence_penalty=0,  # Default
        messages=[
            {"role": "system", "content": "You are a technical support bot. Answer concisely and consistently."},
            {"role": "user", "content": question}
        ]
    )
    return response.choices[0].message.content

# Test
print(support_bot("How do I change my password?"))
print(support_bot("How do I change my password?"))  # Same answer

Exercise 5: Optimized classification (Advanced)

Create a sentiment classifier (Positive/Negative/Neutral) optimized for cost and speed.

def classify_sentiment(text: str) -> str:
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        temperature=0.0,  # Deterministic
        max_tokens=1,  # Only 1 word
        messages=[
            {
                "role": "system",
                "content": "Classify the sentiment. Answer only: Positive, Negative or Neutral"
            },
            {
                "role": "user",
                "content": text
            }
        ]
    )
    return response.choices[0].message.content

# Test
texts = [
    "I love this product!",
    "Terrible service, I don't recommend it",
    "It's an average product"
]

for text in texts:
    sentiment = classify_sentiment(text)
    print(f"{text[:30]:30} → {sentiment}")

Exercise 6: A/B comparison (Advanced)

Compare cost and quality between 2 configurations: temp=0 vs temp=1.5.

import time

def compare_configs(prompt: str):
    configs = [
        {"name": "Deterministic", "temperature": 0.0},
        {"name": "Creative", "temperature": 1.5}
    ]
    
    results = []
    
    for config in configs:
        start = time.time()
        response = client.chat.completions.create(
            model="gpt-3.5-turbo",
            temperature=config["temperature"],
            messages=[{"role": "user", "content": prompt}]
        )
        latency = time.time() - start
        
        tokens = response.usage.total_tokens
        cost = (tokens * 0.002 / 1000)  # Approximate
        
        results.append({
            "config": config["name"],
            "temperature": config["temperature"],
            "response": response.choices[0].message.content[:100] + "...",
            "tokens": tokens,
            "cost": f"${cost:.6f}",
            "latency": f"{latency:.2f}s"
        })
    
    # Print results
    for r in results:
        print(f"\n{'='*60}")
        print(f"Config: {r['config']} (temp={r['temperature']})")
        print(f"Response: {r['response']}")
        print(f"Tokens: {r['tokens']} | Cost: {r['cost']} | Latency: {r['latency']}")

# Test
compare_configs("Write a slogan for an AI startup")

🔗 Additional resources

1. OpenAI API Parameters - Complete official reference
2. Prompt Engineering Guide - OpenAI best practices
3. Temperature Playground - Interactive parameter testing
4. OpenAI Cookbook - Parameters - Advanced examples
5. Rate Limits & Optimization - Official guide
6. Cost Calculator - Cost calculator
7. Error Handling Best Practices - Error guide
8. Community Examples - Community discussions

➡️ Next step

Next capsule: 06-pricing-rate-limits.md

Now that you've mastered the parameters, you'll learn:

• How to calculate exact costs
• Rate limits and quotas
• Cost optimization
• Usage monitoring

Time: 25 minutes

Estimated time: 25 minutes
Next: 06-pricing-rate-limits.md