Parameters and Configuration

Capsule overview

Initializing a model is only the first step. The model's actual behavior — how creative it is, how many tokens it generates, how long it waits before timing out — is controlled with configuration parameters.

In this capsule you'll learn the most important parameters (temperature, max_tokens, timeout, max_retries), how the runtime-configurable model system works, and how to use base_url for providers compatible with the OpenAI API. Once you master these parameters, you'll be able to fine-tune any model for your specific use case.

In the module's project, these parameters will let you configure each provider independently, according to its strengths.

temperature: controlling creativity

temperature controls how "random" or "deterministic" the model's responses are.

low temperature (0.0-0.3) → Consistent, predictable responses
medium temperature (0.4-0.7) → A balance of creativity and consistency
high temperature (0.8-1.0+) → Varied, more creative responses

A practical example

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

# Low temperature: consistent responses
model_precise = init_chat_model("openai:gpt-4.1-mini", temperature=0.0)

# Run it 3 times — you'll get very similar responses
for i in range(3):
    response = model_precise.invoke("What is the capital of Japan?")
    print(f"Run {i+1}: {response.content}")
# Expected output:
# Run 1: The capital of Japan is Tokyo.
# Run 2: The capital of Japan is Tokyo.
# Run 3: The capital of Japan is Tokyo.

print("---")

# High temperature: varied responses
model_creative = init_chat_model("openai:gpt-4.1-mini", temperature=1.0)

for i in range(3):
    response = model_creative.invoke("Write a creative slogan for a coffee shop")
    print(f"Run {i+1}: {response.content}")
# Expected output: 3 different slogans every time

When do you use each value?

Rule of thumb: If you need consistent, factual responses, use a low temperature. If you need variety and creativity, turn it up.

max_tokens: limiting response length

max_tokens defines the maximum number of tokens the model can generate in its response. A token is roughly 3/4 of an English word (it varies from language to language).

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

# Short response (~50 words max)
model_short = init_chat_model("openai:gpt-4.1-mini", max_tokens=50)
response = model_short.invoke("Explain what machine learning is")
print(f"Tokens: ~{len(response.content.split())}")
print(response.content)
# Output: a response truncated at ~50 tokens

# Long response (up to ~500 words)
model_long = init_chat_model("openai:gpt-4.1-mini", max_tokens=500)
response = model_long.invoke("Explain what machine learning is")
print(f"Tokens: ~{len(response.content.split())}")
print(response.content)
# Output: a more detailed response

Why limit tokens?

• Cost: You pay for generated tokens. Fewer tokens = lower cost
• Latency: Fewer tokens = a faster response
• Control: In a chat API, you don't want 2,000-word answers
• Structured output: JSON responses usually need few tokens

Typical values:

timeout and max_retries: robustness in production

LLM APIs fail. They have variable latency, rate limits, and transient errors. timeout and max_retries protect you.

timeout

Defines how many seconds to wait before considering the request failed.

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

# 30-second timeout
model = init_chat_model(
    "openai:gpt-4.1-mini",
    timeout=30
)

response = model.invoke("Write an essay about AI")
print(response.content)
# If the model doesn't respond within 30 seconds, it raises TimeoutError

max_retries

Defines how many times to automatically retry if the request fails.

# 3 automatic retries on transient errors
model = init_chat_model(
    "openai:gpt-4.1-mini",
    max_retries=3,
    timeout=30
)

response = model.invoke("Hello")
# If it fails the first time (429, 500, etc.), it retries up to 3 times

Recommended production configuration

model = init_chat_model(
    "openai:gpt-4.1-mini",
    temperature=0.3,
    max_tokens=500,
    timeout=30,
    max_retries=3
)

Rule: In development you can skip timeout/retries. In production, always set them.

Which errors does max_retries retry?

Not every error gets retried. The behavior depends on the provider, but generally:

Retries use exponential backoff automatically: the first retry waits ~1s, the second ~2s, the third ~4s. This keeps you from hammering a service that's already overloaded.

Other useful parameters

top_p (nucleus sampling)

An alternative to temperature for controlling randomness. top_p=0.9 means the model only considers tokens whose cumulative probability reaches 90%.

model = init_chat_model("openai:gpt-4.1-mini", top_p=0.9)

Recommendation: Use temperature OR top_p, not both at once. temperature is more intuitive and the more widely used of the two.

stop (stop sequences)

The model stops generating text when it hits one of the stop sequences.

# The model will stop if it generates "###" or "END"
model = init_chat_model("openai:gpt-4.1-mini", stop=["###", "END"])

Handy for controlling output format in cases where you aren't using structured output.

Passing multiple parameters

Every parameter is passed as a kwarg to init_chat_model:

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

# Full production configuration
model = init_chat_model(
    "openai:gpt-4.1-mini",
    temperature=0.3,       # Consistent responses
    max_tokens=500,         # 500 response tokens max
    timeout=30,             # 30-second timeout
    max_retries=3,          # 3 automatic retries
)

response = model.invoke("What is RAG?")
print(response.content)

Runtime-configurable models

One of init_chat_model's most powerful features is the ability to create models that change configuration at runtime without reinitializing.

What problem does it solve?

Imagine a system where the user picks which model to use, or where you want to switch models depending on how complex the question is. Without configurable models, you'd need multiple instances:

# Without configurable — multiple instances
model_gpt = init_chat_model("openai:gpt-4.1")
model_claude = init_chat_model("anthropic:claude-sonnet-4-20250514")

# Pick which one to use
if complexity == "high":
    response = model_gpt.invoke(question)
else:
    response = model_claude.invoke(question)

With configurable models, one instance handles everything:

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

# Create a configurable model (without pinning a fixed model)
configurable_model = init_chat_model(
    configurable_fields="any",  # Allows any field to be configured
    temperature=0.5
)

# Use it with OpenAI
response = configurable_model.invoke(
    "Hello",
    config={"configurable": {"model": "openai:gpt-4.1-mini"}}
)
print(f"OpenAI: {response.content}")

# Use it with Anthropic — same instance, different model
response = configurable_model.invoke(
    "Hello",
    config={"configurable": {"model": "anthropic:claude-haiku-4-20250514"}}
)
print(f"Anthropic: {response.content}")

Configuring specific fields

You can restrict which fields are configurable:

# Only allows the model to change, not other parameters
configurable_model = init_chat_model(
    configurable_fields=["model"],
    temperature=0.3  # Fixed, not configurable
)

# Change the model at runtime
response = configurable_model.invoke(
    "Hello",
    config={"configurable": {"model": "openai:gpt-4.1-mini"}}
)

When do you use configurable models?

• ✅ When the user picks the model (a UI with a selector)
• ✅ When you have automatic routing (model A for simple, model B for complex)
• ✅ For A/B testing models
• ✅ For dynamic fallback

base_url: OpenAI-compatible providers

Many LLM providers are compatible with the OpenAI API — they use the same HTTP request format. You can connect to them using base_url:

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

# An OpenAI API-compatible provider (example: Together AI)
model = init_chat_model(
    "openai:meta-llama/Llama-3.1-70B-Instruct",
    base_url="https://api.together.xyz/v1",
    api_key="your-together-api-key"
)

response = model.invoke("Hello")
print(response.content)

This works with providers like Together AI, Fireworks, Groq, Perplexity, and any server that implements the OpenAI API spec.

Popular compatible providers:

Comparison: common parameters vs provider-specific ones

Note: temperature and max_tokens are universal. Parameters like seed only work with specific providers. If you pass an unsupported parameter, it's usually ignored silently.

Connection to the project

In the Multi-Provider Chat with Fallback:

• You'll configure each provider with a low temperature (0.3) for consistent responses
• You'll use timeout to detect quickly whether a provider is down
• max_retries will prevent failures from transient errors
• Configurable models will let you switch providers without reinitializing

Troubleshooting

Problem 1: temperature doesn't seem to do anything

Cause: Some reasoning models (like o3-mini) ignore temperature. Fix: Check that the model supports the parameter. Reasoning models have their own control system (reasoning effort levels, which you'll see in Capsule 06).

Problem 2: The response is cut off mid-sentence

Cause: max_tokens is too low. Fix: Increase max_tokens, or leave it unspecified to use the model's default.

Problem 3: configurable_fields raises an error

Cause: Your langchain version doesn't support configurable models. Fix:

pip install --upgrade langchain langchain-core

Exercises

Exercise 1: Temperature experiment (Easy)

Create two models, with temperature 0.0 and 1.0 respectively. Ask both the same creative question 3 times and observe the difference in variability.

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

model_precise = init_chat_model("openai:gpt-4.1-mini", temperature=0.0)
model_creative = init_chat_model("openai:gpt-4.1-mini", temperature=1.0)

question = "Invent a name for an AI startup"

print("=== Temperature 0.0 ===")
for i in range(3):
    r = model_precise.invoke(question)
    print(f"  {i+1}: {r.content}")

print("\n=== Temperature 1.0 ===")
for i in range(3):
    r = model_creative.invoke(question)
    print(f"  {i+1}: {r.content}")

Exercise 2: Limit the response (Easy)

Make the model answer in exactly one sentence using max_tokens. Experiment with different values until you find the right range.

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

for max_t in [20, 50, 100]:
    model = init_chat_model("openai:gpt-4.1-mini", max_tokens=max_t)
    response = model.invoke("Explain what Docker is")
    print(f"max_tokens={max_t}: {response.content}")
    print(f"  (length: {len(response.content)} characters)")
    print()

Exercise 3: Production config (Medium)

Write a function create_production_model(provider) that returns a model configured with production-appropriate parameters: low temperature, a 30s timeout, 3 retries, and max_tokens of 1000.

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

PROVIDER_MODELS = {
    "openai": "openai:gpt-4.1-mini",
    "anthropic": "anthropic:claude-haiku-4-20250514",
    "google": "google_genai:gemini-2.0-flash",
}

def create_production_model(provider: str):
    """Creates a model with a production-ready configuration."""
    if provider not in PROVIDER_MODELS:
        raise ValueError(f"Unsupported provider: {provider}")
    
    return init_chat_model(
        PROVIDER_MODELS[provider],
        temperature=0.3,
        max_tokens=1000,
        timeout=30,
        max_retries=3
    )

# Usage
model = create_production_model("openai")
response = model.invoke("What is a microservice?")
print(response.content)

Exercise 4: Robust timeout (Medium)

Write a function that calls a model with a 5-second timeout and handles the timeout error gracefully, returning a fallback message.

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

def safe_invoke(question: str, timeout_seconds: int = 5) -> str:
    """Invokes the model with a timeout and returns a fallback if it fails."""
    model = init_chat_model(
        "openai:gpt-4.1-mini",
        timeout=timeout_seconds,
        max_retries=1
    )
    
    try:
        response = model.invoke(question)
        return response.content
    except Exception as e:
        return f"[Model unavailable: {type(e).__name__}]"

# Usage
result = safe_invoke("What is Docker?")
print(result)

Exercise 5: Configurable model (Medium)

Create a configurable model that lets you change the provider at runtime. Ask the same question with 2 different providers using the same instance.

from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import init_chat_model

configurable = init_chat_model(
    configurable_fields="any",
    temperature=0.3
)

question = "What is an AI agent?"

providers = [
    ("OpenAI", "openai:gpt-4.1-mini"),
    ("Anthropic", "anthropic:claude-haiku-4-20250514"),
]

for name, model_id in providers:
    response = configurable.invoke(
        question,
        config={"configurable": {"model": model_id}}
    )
    print(f"{name}: {response.content[:150]}...")
    print()

Summary

In this capsule you learned:

• temperature controls creativity: 0.0 for consistency, 1.0 for variation
• max_tokens limits response length and controls costs
• timeout and max_retries make your application robust against API failures
• Parameters are passed as kwargs to init_chat_model
• Configurable models let you change model/provider at runtime without reinitializing
• base_url connects to providers compatible with the OpenAI API
• In production: always configure temperature, timeout, and retries

Next capsule: Invoke, Stream and Batch — the 3 execution modes that determine how you receive the model's responses.

Additional resources

1. Chat Model Parameters - Standard parameters in the official docs
2. Configurable Runnables - Guide to configurable models
3. OpenAI API Parameters - OpenAI parameter reference
4. Anthropic API Parameters - Anthropic parameter reference
5. Temperature and Sampling - How temperature works internally
6. LangChain Error Handling - Retry and error handling

Module 1 — LangChain & LangGraph: From Chains to Agents