3. Roles: system, user, assistant

Overview

LLM APIs use roles to structure conversations: system, user, and assistant. Each role has a distinct purpose and shapes how the model generates its answer. In this capsule you'll learn to use the system prompt as a behavior contract, the user prompt as input, and the assistant as the shape of the answer. You'll also see how to design stateful multi-turn conversations.

Why it matters: The system prompt is the most powerful lever you have for controlling the model's behavior. Most users only send user messages; professionals configure system first. The difference in consistency and quality is dramatic.

The three roles

system

Purpose: Defines the assistant's global behavior for the whole conversation. It's the model's "configuration".

Characteristics:

• It's processed first and establishes the model's "persona" and rules
• Typically a single message at the start of the conversation
• Not visible to the end user in most UIs
• High priority: the model tries to follow these instructions throughout the conversation
• Permanent for the session: it doesn't change between turns

Example:

system_prompt = """
You are a technical assistant specialized in Python.
- Respond in English
- Give code examples when relevant
- If you don't know something, say so explicitly
- Don't invent information
"""

What to put in system:

• The assistant's role or persona
• Behavior constraints
• The expected output format
• Guardrails and edge cases
• Permanent domain context

user

Purpose: Represents the user's input (or the system's, when it stands in for the user). It's the "prompt" that triggers each answer.

Characteristics:

• There can be multiple user messages in a conversation (multi-turn)
• It contains the question, the instruction, or the data to process
• In programmatic applications, the "user" is sometimes generated by your system (e.g. "Classify this ticket: ...")
• It's the variable input that changes on every call

When to use user for extra instructions:

# If you need instructions that vary per call, put them in user
messages = [
    {
        "role": "system",
        "content": "You are a sentiment classifier. Respond with JSON."
    },
    {
        "role": "user",
        "content": """
        Classify the sentiment with a confidence score.
        
        Text: "I loved the product, I'd definitely recommend it"
        
        Format: {"sentiment": "POSITIVE|NEGATIVE|NEUTRAL", "confidence": 0.0-1.0}
        """
    }
]

assistant

Purpose: Represents the model's previous answers. It's used in multi-turn conversations to provide the context of the history.

Characteristics:

• On the first call there usually are no assistant messages
• On subsequent calls, you include the full history
• The model "sees" its own previous answers and stays coherent
• You can "prefill" the assistant to steer the answer (an advanced technique)

Prefilling with assistant (Anthropic):

# Technique: force the answer to start in a certain way
messages = [
    {"role": "user", "content": "Classify: 'Great product'"},
    {"role": "assistant", "content": "{"}  # Forces it to continue with JSON
]

The system prompt as a behavior contract

The system prompt works like a contract: you define what the model will and won't do across the whole conversation.

Elements of an effective system prompt

from openai import OpenAI
import json

client = OpenAI()

# A system prompt well structured as a contract
SYSTEM = """
# Role
You are a support ticket classifier for a B2B software company.

# Capabilities
- Classify tickets into exactly one category
- Identify urgency (HIGH/MEDIUM/LOW)
- Detect the ticket's language

# Constraints
- You only classify tickets; you don't solve technical problems
- You don't give opinions about the quality of the support
- If the text doesn't look like a ticket, respond with category="OTHER"

# Response format
Always valid JSON with this exact structure:
{
  "category": "TECHNICAL|BILLING|ACCOUNT|FEATURE_REQUEST|OTHER",
  "urgency": "HIGH|MEDIUM|LOW",
  "language": "ES|EN|PT|FR|OTHER",
  "summary": "10 words maximum"
}

# Urgency definitions
- HIGH: system down, data loss, unable to work
- MEDIUM: limited functionality, workaround available
- LOW: questions, improvements, inquiries
"""

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "I can't access the application since this morning and I have a presentation in 2 hours"}
    ],
    temperature=0
)

result = json.loads(response.choices[0].message.content)
print(result)
# {
#   "category": "TECHNICAL",
#   "urgency": "HIGH",
#   "language": "EN",
#   "summary": "No access to app with urgent presentation"
# }

Multi-turn conversations

In conversations with several exchanges, the history is built by accumulating messages:

from openai import OpenAI

client = OpenAI()

# Simulating a multi-turn conversation
def chat(messages: list, user_input: str) -> tuple[str, list]:
    """
    Adds the user's input, calls the API,
    and returns the answer + the updated history.
    """
    messages.append({"role": "user", "content": user_input})
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
        temperature=0.7
    )
    
    assistant_response = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_response})
    
    return assistant_response, messages

# Initialize with the system prompt
history = [
    {
        "role": "system",
        "content": "You are a travel assistant. Respond in English. Be concise and practical."
    }
]

# Turn 1
answer, history = chat(history, "Which cities do you recommend in Japan?")
print(f"Assistant: {answer}\n")

# Turn 2 — the model remembers the context (Japan)
answer, history = chat(history, "How many days do I need for each one?")
print(f"Assistant: {answer}\n")

# Turn 3 — it stays coherent with the conversation
answer, history = chat(history, "Which is best for a first trip?")
print(f"Assistant: {answer}\n")

print(f"Total messages in history: {len(history)}")
# Total: 7 (1 system + 3 user + 3 assistant)

Rule: Always include the full history (or a summary if it's very long) so the model stays coherent.

Managing history in production

The history grows with every turn. You have to manage it actively:

from openai import OpenAI

client = OpenAI()

MAX_HISTORY_TOKENS = 4000  # Limit for the history
SYSTEM_PROMPT = "You are a Python technical assistant."

def truncate_history(messages: list, max_chars: int = 8000) -> list:
    """
    Keeps the system prompt and the last N messages
    so as not to exceed the context limit.
    """
    system = [m for m in messages if m["role"] == "system"]
    conversation = [m for m in messages if m["role"] != "system"]
    
    # Compute the total size
    total_chars = sum(len(m["content"]) for m in conversation)
    
    # If it exceeds the limit, drop the oldest ones (not the system)
    while total_chars > max_chars and len(conversation) > 2:
        # Drop the oldest pair (user + assistant)
        removed = conversation.pop(0)
        total_chars -= len(removed["content"])
    
    return system + conversation

# Example of use
history = [{"role": "system", "content": SYSTEM_PROMPT}]

# Simulate several turns
for i in range(10):
    history.append({"role": "user", "content": f"Question {i}: How does X work?"})
    history.append({"role": "assistant", "content": f"Answer {i}: X works like this..."})
    
    # Truncate before each call
    history = truncate_history(history)

print(f"Messages in history (truncated): {len(history)}")

How each role affects the output

Experiment: same user, different systems

from openai import OpenAI

client = OpenAI()

user_msg = "Explain what an API is"

systems = {
    "technical": "You are a senior engineer. Explain technically and concisely. Use terms like REST, HTTP, endpoints.",
    "beginner": "You are a teacher explaining to someone with no technical background. Use real-world analogies.",
    "business": "You are a business consultant. Explain the business value, without technical terms.",
}

for name, system in systems.items():
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": system},
            {"role": "user", "content": user_msg}
        ],
        temperature=0.3,
        max_tokens=100
    )
    print(f"\n--- System: {name} ---")
    print(response.choices[0].message.content[:200])  # First 200 chars

Expected output:

--- System: technical ---
An API (Application Programming Interface) is a communication contract between systems. It defines HTTP endpoints that accept requests with specific parameters and return structured responses (JSON/XML)...

--- System: beginner ---
Imagine you're at a restaurant. The menu is the list of available dishes (what you can order). The waiter is the API: they take your order, go to the kitchen, and bring you the result...

--- System: business ---
An API is a digital "front door" that lets different software systems talk to each other. For your company, it means your software can connect with vendors, customers and partners automatically...

Differences by provider

# OpenAI
client_oai = OpenAI()
response_oai = client_oai.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Respond in Spanish."},
        {"role": "user", "content": "Hello, how are you?"}
    ]
)

# Anthropic
import anthropic
client_ant = anthropic.Anthropic()
response_ant = client_ant.messages.create(
    model="claude-3-5-sonnet-20241022",
    system="Respond in Spanish.",  # System separate, not in messages
    messages=[
        {"role": "user", "content": "Hello, how are you?"}
    ],
    max_tokens=200
)

Comparison: with vs without a system prompt

How this connects to the project

In the Prompt Analyzer (capsule 08) you'll identify whether a prompt uses roles correctly:

• Does it have a system prompt when it should?
• Does the system define clear constraints and format?
• Is the user input well delimited?
• Does the multi-turn conversation stay coherent?

Troubleshooting

Problem 1: the model ignores the system prompt

Cause: Some models (or weaker versions) give less weight to the system. Or the user prompt is very long and "dilutes" the system.

Fix:

# Option A: Repeat the key instructions in the user message
messages = [
    {"role": "system", "content": "Respond ONLY in JSON. Nothing else."},
    {
        "role": "user",
        "content": """
        Classify this ticket.
        IMPORTANT: Respond ONLY with JSON, no extra text.
        
        Ticket: "I can't get into my account"
        Format: {"category": "...", "urgency": "..."}
        """
    }
]

Problem 2: in multi-turn, the model "forgets" constraints

Cause: The history grows and the system ends up "far away" in the context. Recent messages carry more weight.

Fix:

# Re-inject the critical instructions into user messages periodically
def build_user_message(text: str, is_first_turn: bool) -> str:
    if is_first_turn:
        return text
    # On turns 5, 10, 15... re-inject the rules
    return f"""
    [REMINDER: Always respond in JSON. Never include extra text.]
    
    {text}
    """

Problem 3: Anthropic has no "system" in the messages array

Cause: Anthropic uses a separate system parameter, not one inside the messages array.

Fix:

import anthropic

client = anthropic.Anthropic()

# ✅ Correct for Anthropic
response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    system="You are a classifier. Respond only with: POSITIVE, NEGATIVE or NEUTRAL.",
    messages=[{"role": "user", "content": "Sentiment of 'Excellent service'?"}],
    max_tokens=10
)

# ❌ Wrong (system inside messages doesn't work on Anthropic)
# messages=[{"role": "system", "content": "..."}]

Problem 4: incoherent answers in multi-turn

Cause: The history doesn't include every turn, or they're in the wrong order.

Fix: Check that the history is in chronological order and alternating (user, assistant, user, assistant...):

# ✅ Correct order
messages = [
    {"role": "system", "content": "..."},
    {"role": "user", "content": "Question 1"},       # Turn 1 user
    {"role": "assistant", "content": "Answer 1"},     # Turn 1 assistant
    {"role": "user", "content": "Question 2"},        # Turn 2 user
    # The model will generate: turn 2 assistant
]

# ❌ Wrong: two user messages in a row with no assistant
messages = [
    {"role": "user", "content": "Question 1"},
    {"role": "user", "content": "Question 2"},  # Error: no assistant between them
]

Exercises

Exercise 1: Design a system prompt for an extractor

Create a system prompt for a system that extracts dates from text. It must: (1) only extract explicit dates, (2) return JSON, (3) use ISO format whenever possible.

from openai import OpenAI
import json

client = OpenAI()

SYSTEM = """
# Role
You are a date extractor working on natural language text.

# Task
Identify and extract every explicit date in the text.

# Rules
- Explicit dates only (not "yesterday", "last week", or inferred dates)
- Convert to ISO format YYYY-MM-DD whenever possible
- If there's ambiguity (e.g. "03/04/2024" → April 3 or March 4), use the most common interpretation in the context of the text
- If there are no dates, return an empty list

# Response format
Always valid JSON, nothing else:
{"dates": ["YYYY-MM-DD", ...]}
"""

texts = [
    "The meeting is on March 15, 2025 and the follow-up on 03/22/2025",
    "We don't have confirmed dates yet",
    "It's due on Tuesday but I don't know when"
]

for text in texts:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": text}
        ],
        temperature=0
    )
    result = json.loads(response.choices[0].message.content)
    print(f"Input: {text}")
    print(f"Output: {result}\n")

# Expected output:
# Input: The meeting is on March 15, 2025 and the follow-up on 03/22/2025
# Output: {'dates': ['2025-03-15', '2025-03-22']}
#
# Input: We don't have confirmed dates yet
# Output: {'dates': []}
#
# Input: It's due on Tuesday but I don't know when
# Output: {'dates': []}

Exercise 2: Multi-turn with context

Implement a 3-turn conversation where the user asks about a city, then about the weather, and finally about recommendations. The system must establish that you're a travel assistant who answers concisely.

from openai import OpenAI

client = OpenAI()

SYSTEM = """
You are an expert travel assistant. Respond in English.
- Concise answers (3-4 sentences maximum)
- Practical, useful information only
- If something varies by season, mention the best time
"""

def chat(messages, user_input):
    messages.append({"role": "user", "content": user_input})
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
        temperature=0.5
    )
    assistant_msg = response.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_msg})
    return assistant_msg

history = [{"role": "system", "content": SYSTEM}]

# Turn 1: the city
r = chat(history, "What do you recommend visiting in Tokyo?")
print(f"[Turn 1] {r}\n")

# Turn 2: the weather (the model remembers we're talking about Tokyo)
r = chat(history, "What's the best month to go?")
print(f"[Turn 2] {r}\n")

# Turn 3: recommendations (it keeps the context of Tokyo + the season)
r = chat(history, "What should I pack?")
print(f"[Turn 3] {r}\n")

print(f"Total history: {len(history)} messages")

Exercise 3: An adapter pattern for OpenAI and Anthropic

Write a function call_llm(system, user, provider) that works with both providers through the same interface.

from openai import OpenAI
import anthropic
from typing import Literal

oai_client = OpenAI()
ant_client = anthropic.Anthropic()

def call_llm(
    system: str,
    user: str,
    provider: Literal["openai", "anthropic"] = "openai",
    temperature: float = 0,
    max_tokens: int = 200
) -> str:
    """
    A unified interface for OpenAI and Anthropic.
    Always returns the answer's text.
    """
    if provider == "openai":
        response = oai_client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[
                {"role": "system", "content": system},
                {"role": "user", "content": user}
            ],
            temperature=temperature,
            max_tokens=max_tokens
        )
        return response.choices[0].message.content
    
    elif provider == "anthropic":
        response = ant_client.messages.create(
            model="claude-3-5-haiku-20241022",
            system=system,  # Anthropic uses a separate parameter
            messages=[{"role": "user", "content": user}],
            temperature=temperature,
            max_tokens=max_tokens
        )
        return response.content[0].text
    
    else:
        raise ValueError(f"Unsupported provider: {provider}")

# Same prompt, two providers
SYSTEM = "Classify the sentiment. Respond only with: POSITIVE, NEGATIVE or NEUTRAL."
USER = "This product exceeded my expectations"

for provider in ["openai", "anthropic"]:
    result = call_llm(SYSTEM, USER, provider=provider)
    print(f"{provider}: {result}")

Exercise 4 (Advanced): A system prompt with guardrails

Design a system prompt for a help chatbot that: (a) answers questions about the product, (b) rejects off-topic questions, (c) detects user frustration and responds empathetically.

SYSTEM_WITH_GUARDRAILS = """
# Role
You are the support assistant for TechApp, a project management application.

# Capabilities
- Answer questions about TechApp's features
- Guide users through the application
- Escalate unresolved technical problems

# Constraints (Guardrails)
- If the question isn't related to TechApp, respond:
  "I can only help you with questions about TechApp. Is there anything specific about the app I can help you with?"
- If you detect frustration (words like "doesn't work", "terrible", "awful", "fed up"), 
  start with: "I understand your frustration. I'm going to help you fix this."
- Never mention competitors directly
- If you don't know something, say: "I don't have that information. Let me connect you with an agent."

# Format
- Concise answers (3 paragraphs maximum)
- Use a friendly, professional tone
- If there are steps to follow, use a numbered list
"""

# Guardrail tests
test_cases = [
    "How do I create a new project?",     # Normal
    "What's the capital of France?",      # Off-topic
    "Nothing works, this is terrible",    # Frustration
]

from openai import OpenAI
client = OpenAI()

for test in test_cases:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": SYSTEM_WITH_GUARDRAILS},
            {"role": "user", "content": test}
        ],
        temperature=0.3
    )
    print(f"User: {test}")
    print(f"Assistant: {response.choices[0].message.content[:150]}...\n")

Summary

• system: The behavior contract for the whole conversation. It configures persona, constraints, format, guardrails.
• user: The input that triggers each answer. It can include extra instructions that vary per call.
• assistant: The history of answers. Essential for coherent multi-turn.
• The system prompt is the most powerful lever: same user message + different systems = radically different outputs.
• Multi-turn: Build a cumulative history (system + alternating user/assistant). Truncate it if it grows too much.
• By provider: OpenAI includes system in the messages array; Anthropic has it as a separate parameter.
• Guardrails: Explicit constraints in the system for edge-case behavior and safety.

Further resources

1. OpenAI Chat Completions Guide — Full documentation of roles and message structure
2. Anthropic Messages API — Claude's message format, with system separate
3. Anthropic System Prompts — How to use system prompts in Claude effectively
4. OpenAI Best Practices - System Messages — Strategies for effective system messages
5. tiktoken — For counting the history's tokens and avoiding limit overruns
6. Multi-turn Conversations (Anthropic) — Managing multi-turn conversations