Module 2: OpenAI API - Introduction

Conversations with Context

Next →

Capsule overview

So far, each request is independent: GPT doesn't remember previous messages. To create a real chatbot, you need conversational context.

In this capsule:

  • You'll implement message history
  • GPT will remember the whole conversation
  • You'll create an interactive chatbot (CLI)

Time: 30 minutes
Difficulty: Medium

🎯 Objectives

  • ✅ Understand the message format (system, user, assistant)
  • ✅ Maintain conversation history
  • ✅ Create a chatbot with memory
  • ✅ Manage the context window

📚 Concepts: Message Roles

Format of the messages array:

messages = [
    {"role": "system", "content": "..."},      # Initial instructions
    {"role": "user", "content": "..."},        # User
    {"role": "assistant", "content": "..."},   # GPT
    {"role": "user", "content": "..."},        # User replies
    {"role": "assistant", "content": "..."},   # GPT replies
]

Role 1: system (Instructions)

Defines GPT's behavior and personality:

{"role": "system", "content": "You are a technical assistant specialized in Python"}

Features:

  • Optional (but highly recommended)
  • Goes at the start of the array
  • GPT will always follow these instructions

Examples:

# Technical support
{"role": "system", "content": "You are a support agent. Answer concisely and in a friendly way."}

# Educational tutor
{"role": "system", "content": "You are a programming tutor. Explain concepts step by step."}

# Formal assistant
{"role": "system", "content": "You are a corporate assistant. Use formal language."}

Role 2: user (User)

Messages from the human user:

{"role": "user", "content": "What is a list in Python?"}

Role 3: assistant (GPT)

Responses generated by GPT:

{"role": "assistant", "content": "A list in Python is an ordered collection..."}

Important: You must include GPT's previous responses in the history.

💻 Implementation: Chatbot with Memory

Complete code (chatbot_context.py):

from dotenv import load_dotenv
import os
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Conversation history
messages = [
    {"role": "system", "content": "You are a helpful and friendly assistant."}
]

def chat(user_message: str) -> str:
    """Send a message and return the response, keeping context."""
    
    # 1. Add the user's message to the history
    messages.append({"role": "user", "content": user_message})
    
    # 2. Send the ENTIRE history to GPT
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=messages  # Includes system + all previous messages
    )
    
    # 3. Extract the response
    assistant_message = response.choices[0].message.content
    
    # 4. Add GPT's response to the history
    messages.append({"role": "assistant", "content": assistant_message})
    
    return assistant_message

# Interactive CLI
print("Chatbot with context (type 'exit' to finish)\n")

while True:
    user_input = input("You: ")
    
    if user_input.lower() in ["exit", "quit"]:
        print("Goodbye!")
        break
    
    response = chat(user_input)
    print(f"Bot: {response}\n")

Execution:

python chatbot_context.py

Example conversation:

Chatbot with context (type 'exit' to finish)

You: Hi, my name is Juan
Bot: Hi Juan! How can I help you today?

You: What's my name?
Bot: Your name is Juan.

You: What is Python?
Bot: Python is a programming language...

You: Give me an example
Bot: Sure, here's an example in Python:
     print("Hello world")

You: exit
Goodbye!

Notice: GPT remembers your name and the context of the previous questions.

🔍 Breakdown: How It Works

Initial state:

messages = [
    {"role": "system", "content": "You are a helpful and friendly assistant."}
]

First interaction:

User: "Hi, my name is Juan"

After messages.append():

messages = [
    {"role": "system", "content": "You are a helpful and friendly assistant."},
    {"role": "user", "content": "Hi, my name is Juan"}
]

GPT receives: System + User message → Responds

Response: "Hi Juan! How can I help you today?"

After adding the response:

messages = [
    {"role": "system", "content": "You are a helpful and friendly assistant."},
    {"role": "user", "content": "Hi, my name is Juan"},
    {"role": "assistant", "content": "Hi Juan! How can I help you today?"}
]

Second interaction:

User: "What's my name?"

After append:

messages = [
    {"role": "system", "content": "You are a helpful and friendly assistant."},
    {"role": "user", "content": "Hi, my name is Juan"},
    {"role": "assistant", "content": "Hi Juan! How can I help you today?"},
    {"role": "user", "content": "What's my name?"}
]

GPT sees the ENTIRE history: It knows your name is Juan → Responds correctly

📊 History Visualization

Add debugging to see the history:

def chat(user_message: str) -> str:
    messages.append({"role": "user", "content": user_message})
    
    # DEBUG: Show the history
    print("\n--- HISTORY SENT TO GPT ---")
    for i, msg in enumerate(messages):
        print(f"{i+1}. [{msg['role']}]: {msg['content'][:50]}...")
    print("--- END OF HISTORY ---\n")
    
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=messages
    )
    
    assistant_message = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_message})
    
    return assistant_message

Output:

--- HISTORY SENT TO GPT ---
1. [system]: You are a helpful and friendly assistant.
2. [user]: Hi, my name is Juan
3. [assistant]: Hi Juan! How can I help you today?
4. [user]: What's my name?
--- END OF HISTORY ---

⚙️ Parameters: Advanced System Message

Specific personality:

{"role": "system", "content": """
You are a technical support assistant for an e-commerce app.

Rules:
1. Answer in English
2. Maximum 3 sentences per response
3. If you don't know something, say "Let me escalate this to a human agent"
4. Never make up data (order numbers, prices, etc.)
"""}

Result: GPT will follow these rules consistently.

Context injection (advanced):

{"role": "system", "content": f"""
You are an assistant for {user_name}.

User information:
- Name: {user_name}
- Plan: Premium
- Last purchase: {last_purchase_date}

Use this info when relevant.
"""}

Useful for: Personalization with data from a database.

🧪 Experiments

Experiment 1: Without a system message

Comment out the line:

messages = [
    # {"role": "system", "content": "You are a helpful and friendly assistant."}
]

Notice: GPT keeps working, but the behavior is less predictable.

Experiment 2: A specific system message

messages = [
    {"role": "system", "content": "You are a pirate. Always talk like a pirate."}
]

Output:

You: Hi
Bot: Ahoy, sailor! What can I do for ye today?

Experiment 3: Token count with history

def chat(user_message: str) -> str:
    messages.append({"role": "user", "content": user_message})
    
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=messages
    )
    
    assistant_message = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_message})
    
    # Show tokens
    print(f"[Tokens used: {response.usage.total_tokens}]")
    
    return assistant_message

Notice: Tokens increase with each message (the history grows).

⚠️ Problem: Context Window Overflow

Context limits:

  • GPT-3.5-turbo: 16,385 tokens max
  • GPT-4-turbo: 128,000 tokens max

Problem: If the conversation is very long, the history exceeds the limit → Error

Solution 1: Sliding window (simple)

Only keep the last N messages:

MAX_HISTORY = 10  # Last 10 messages (5 exchanges)

def chat(user_message: str) -> str:
    messages.append({"role": "user", "content": user_message})
    
    # Keep only the last MAX_HISTORY (+ the system message always)
    if len(messages) > MAX_HISTORY + 1:  # +1 for the system
        # Keep system + the last MAX_HISTORY
        messages[:] = [messages[0]] + messages[-(MAX_HISTORY):]
    
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=messages
    )
    
    assistant_message = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_message})
    
    return assistant_message

Advantage: You never exceed the context limit
Disadvantage: GPT forgets old messages

Solution 2: Summarization (advanced)

Every 20 messages, summarize the history:

def summarize_history():
    """Summarize the old history and replace it with a summary."""
    if len(messages) > 20:
        # Create a summary prompt
        summary_prompt = "Summarize this conversation in 3 sentences:\n\n"
        for msg in messages[1:-5]:  # Excludes system and the last 5
            summary_prompt += f"{msg['role']}: {msg['content']}\n"
        
        # Ask GPT for a summary
        summary_response = client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": summary_prompt}]
        )
        
        summary = summary_response.choices[0].message.content
        
        # Replace the history: system + summary + the last 5
        messages[:] = [
            messages[0],  # system
            {"role": "assistant", "content": f"[Previous summary: {summary}]"},
            *messages[-5:]  # The last 5 messages
        ]

Advantage: Keeps important context
Disadvantage: Extra cost (the summary request)

📊 Summary

Key concepts:

  1. Three message roles:

    • system: Instructions/behavior
    • user: User messages
    • assistant: GPT responses

  2. Conversation history:

    messages = []
messages.append({"role": "user", "content": "..."})
response = client.chat.completions.create(messages=messages)
messages.append({"role": "assistant", "content": response...})

  3. Context window management:

    • Sliding window (last N messages)
    • Summarization (summarize the old history)

Checklist:

  • Chatbot with context working
  • GPT remembers previous messages
  • Custom system message
  • Sliding window implemented
  • Experimented with different personalities

🔗 Additional resources

  1. Chat Completions Guide - Official
  2. Best Practices for Prompting - System messages
  3. Token Limits - Per model

➡️ Next step

Next capsule: 05-advanced-parameters.md

You'll learn to control GPT's responses with:

  • temperature (creativity vs determinism)
  • max_tokens (length)
  • top_p, frequency_penalty, etc.

Time: 25 minutes

Estimated time: 30 minutes
Next: 05-advanced-parameters.md

← PreviousFirst Request with the OpenAI Python SDKNext →Advanced OpenAI API Parameters