Module 6: Prompt Composition and Chaining
7. Routing by Complexity
Overview
Routing by complexity is a cost optimization strategy that classifies incoming requests by their complexity and directs them to the most appropriate model or strategy. The core idea: not every task needs the most expensive model or the most sophisticated technique.
With a good routing system you can cut your operating costs by 60-80% without sacrificing quality the user would notice, because 70-80% of the requests in most systems are simple tasks any model can solve correctly.
The Problem Without Routing
System without routing (everything to gpt-4o):
- "What are your opening hours?" → gpt-4o → $0.005
- "Analyze this 20-page legal contract" → gpt-4o → $0.08
- "Can I return a product?" → gpt-4o → $0.003
- "How does the points system work?" → gpt-4o → $0.002
- "Design a pricing strategy for our company..." → gpt-4o → $0.12
Estimated: $0.21 for 5 requests
System with routing:
- "What are your opening hours?" → gpt-4o-mini → $0.0001
- "Analyze this 20-page legal contract" → gpt-4o → $0.08
- "Can I return a product?" → gpt-4o-mini → $0.0001
- "How does the points system work?" → gpt-4o-mini → $0.0001
- "Design a pricing strategy..." → gpt-4o → $0.12
Estimated: $0.2004 for 5 requests
Savings in this example: ~$0.01 (small)
At 10,000 requests/day: $200/day in savings if 80% are simple
Routing System Architecture
Input
│
▼
┌─────────────────────────────────┐
│ CLASSIFIER │
│ (always gpt-4o-mini, fast) │
│ │
│ Evaluation criteria: │
│ - Length and complexity │
│ - Task type │
│ - Complexity keywords │
│ - Does it need reasoning? │
└──────────────┬──────────────────┘
│
┌───────┴───────┐
│ │
▼ ▼
SIMPLE COMPLEX
│ │
▼ ▼
gpt-4o-mini gpt-4o or
(0.15x/0.60x claude-opus
per 1K tokens) (2.5x/10x
per 1K tokens)
Complete Implementation
from openai import OpenAI
from dataclasses import dataclass
from enum import Enum
from typing import Optional
import time
import json
client = OpenAI()
class ComplexityLevel(Enum):
SIMPLE = "simple"
MEDIUM = "medium"
COMPLEX = "complex"
VERY_COMPLEX = "very_complex"
@dataclass
class RoutingConfig:
"""Configuration of the routing system."""
simple_model: str = "gpt-4o-mini"
complex_model: str = "gpt-4o"
very_complex_model: str = "gpt-4o" # With more tokens or a special temperature
simple_threshold: float = 0.7 # Minimum confidence to classify as simple
max_tokens_simple: int = 300
max_tokens_complex: int = 1000
use_heuristic: bool = True # Use the heuristic before the LLM
verbose: bool = False
@dataclass
class RoutingResult:
"""Result of a request processed by the router."""
query: str
complexity: ComplexityLevel
model_used: str
answer: str
time_seconds: float
classification_method: str # "heuristic" or "llm"
classification_reason: str
confidence: float
# ============================================================
# HEURISTIC CLASSIFIER (no LLM calls)
# ============================================================
class HeuristicClassifier:
"""
Rule-based classifier with no LLM calls.
Very fast and free.
"""
SIMPLE_WORDS = {
"hours", "price", "cost", "address", "phone",
"how much", "when", "where", "what is", "what are",
"yes or no", "true or false", "confirm", "true", "false",
"number of", "date of", "list of", "hi", "thanks"
}
COMPLEX_WORDS = {
"analyze", "analyse", "analyze in depth",
"compare", "comparison", "difference between",
"design", "designing", "strategy",
"reason", "reasoning", "explain why",
"evaluate", "assess", "pros and cons",
"predict", "forecast", "project",
"optimize", "optimise", "improve",
"code", "implement", "implementation",
"full report", "detailed analysis"
}
LONG_THRESHOLD = 200 # Characters: above this it's considered more complex
VERY_LONG_THRESHOLD = 800 # Very long text = complex
def classify(self, text: str) -> Optional[tuple[ComplexityLevel, str, float]]:
"""
Classifies the text using heuristics.
Returns:
Tuple (level, reason, confidence), or None if it can't decide
"""
text_lower = text.lower()
length = len(text)
# Rule 1: Very long text = complex
if length > self.VERY_LONG_THRESHOLD:
return (
ComplexityLevel.COMPLEX,
f"Long text ({length} chars)",
0.85
)
# Rule 2: High-complexity words
complex_words_found = [
w for w in self.COMPLEX_WORDS
if w in text_lower
]
if complex_words_found:
return (
ComplexityLevel.COMPLEX,
f"High-complexity words: {complex_words_found[:2]}",
0.80
)
# Rule 3: Simplicity words
simple_words_found = [
w for w in self.SIMPLE_WORDS
if w in text_lower
]
if simple_words_found and length < self.LONG_THRESHOLD:
return (
ComplexityLevel.SIMPLE,
f"Simple words: {simple_words_found[:2]}",
0.82
)
# Rule 4: Short text with no complex words = probably simple
if length < 100:
return (
ComplexityLevel.SIMPLE,
f"Short text ({length} chars) with no detected complexity",
0.70
)
# The heuristic can't decide
return None
# ============================================================
# LLM CLASSIFIER
# ============================================================
class LLMClassifier:
"""
LLM-based classifier.
More accurate but it has a cost and adds latency.
"""
CLASSIFICATION_PROMPT = """Classify the complexity of this request:
REQUEST: {text}
Criteria:
- SIMPLE: Direct question, a 1-2 sentence answer, no complex reasoning
Examples: yes/no questions, basic information lookups, greetings
- MEDIUM: Requires some context or reasoning, a 3-5 sentence answer
Examples: explaining how something works, basic steps, simple comparisons
- COMPLEX: Deep analysis, multiple perspectives, planning, long code
Examples: system design, document analysis, business strategy
- VERY_COMPLEX: Maximum reasoning capability, very long documents, critical decisions
Examples: full financial analysis, legal review, software architecture
Respond in JSON:
{{"complexity": "simple|medium|complex|very_complex", "confidence": 0.0-1.0, "reason": "short explanation"}}"""
def classify(self, text: str) -> tuple[ComplexityLevel, str, float]:
"""Classifies using gpt-4o-mini (always the cheap model for classifying)."""
response = client.chat.completions.create(
model="gpt-4o-mini", # Always the cheap one for classifying
messages=[{
"role": "user",
"content": self.CLASSIFICATION_PROMPT.format(text=text[:500])
}],
temperature=0,
max_tokens=100,
response_format={"type": "json_object"}
)
try:
data = json.loads(response.choices[0].message.content)
complexity_str = data.get("complexity", "medium")
confidence = float(data.get("confidence", 0.7))
reason = data.get("reason", "")
level_map = {
"simple": ComplexityLevel.SIMPLE,
"medium": ComplexityLevel.MEDIUM,
"complex": ComplexityLevel.COMPLEX,
"very_complex": ComplexityLevel.VERY_COMPLEX
}
level = level_map.get(complexity_str, ComplexityLevel.MEDIUM)
return level, reason, confidence
except Exception:
return ComplexityLevel.MEDIUM, "Classification error", 0.5
# ============================================================
# MAIN ROUTER
# ============================================================
class ComplexityRouter:
"""
Routing system that directs requests to the appropriate model
based on their detected complexity.
"""
def __init__(self, config: RoutingConfig = None):
self.config = config or RoutingConfig()
self.heuristic = HeuristicClassifier()
self.llm_classifier = LLMClassifier()
# Metrics tracking
self.stats = {
"total": 0,
"simple": 0,
"medium": 0,
"complex": 0,
"very_complex": 0,
"via_heuristic": 0,
"via_llm": 0,
"estimated_cost_usd": 0.0,
"savings_vs_gpt4o_usd": 0.0
}
# Estimated costs per 1K tokens (input/output)
self.COSTS = {
"gpt-4o-mini": {"input": 0.00015, "output": 0.0006},
"gpt-4o": {"input": 0.0025, "output": 0.01}
}
def classify(self, text: str) -> tuple[ComplexityLevel, str, str, float]:
"""
Classifies the complexity of the request.
First it tries the heuristic (free), then the LLM if needed.
Returns:
Tuple (level, reason, method, confidence)
"""
# Try the heuristic first
if self.config.use_heuristic:
heuristic_result = self.heuristic.classify(text)
if heuristic_result:
level, reason, confidence = heuristic_result
# Only use the heuristic if the confidence is high enough
if confidence >= self.config.simple_threshold:
return level, reason, "heuristic", confidence
# If the heuristic isn't enough, use the LLM
level, reason, confidence = self.llm_classifier.classify(text)
return level, reason, "llm", confidence
def select_model(self, level: ComplexityLevel) -> tuple[str, int]:
"""
Selects the model and max_tokens based on the complexity.
Returns:
Tuple (model_name, max_tokens)
"""
if level in [ComplexityLevel.SIMPLE, ComplexityLevel.MEDIUM]:
return self.config.simple_model, self.config.max_tokens_simple
elif level == ComplexityLevel.COMPLEX:
return self.config.complex_model, self.config.max_tokens_complex
else: # VERY_COMPLEX
return self.config.very_complex_model, 2000
def process(
self,
text: str,
system_prompt: str = "You are a helpful assistant.",
temperature: float = 0,
force_model: str = None
) -> RoutingResult:
"""
Processes a request with automatic routing.
Args:
text: The user's request
system_prompt: The assistant's system prompt
temperature: Generation temperature
force_model: If given, bypasses the routing and uses this model
Returns:
RoutingResult with the answer and metadata
"""
t0 = time.time()
self.stats["total"] += 1
# Classify the complexity
level, reason, method, confidence = self.classify(text)
if self.config.verbose:
print(f"[Router] Complexity: {level.value} ({confidence:.0%} confidence, via {method})")
print(f" Reason: {reason}")
# Update the classification statistics
self.stats[level.value] += 1
self.stats[f"via_{method}"] += 1
# Select the model
if force_model:
model = force_model
max_tokens = self.config.max_tokens_complex
else:
model, max_tokens = self.select_model(level)
if self.config.verbose:
print(f" Using: {model} (max_tokens={max_tokens})")
# Generate the answer
response = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": text}
],
temperature=temperature,
max_tokens=max_tokens
)
answer = response.choices[0].message.content
elapsed = time.time() - t0
# Compute the costs
tokens_in = response.usage.prompt_tokens
tokens_out = response.usage.completion_tokens
model_costs = self.COSTS.get(model, self.COSTS["gpt-4o-mini"])
cost = (tokens_in / 1000 * model_costs["input"]) + (tokens_out / 1000 * model_costs["output"])
# Compute the savings vs. always using gpt-4o
gpt4o_costs = self.COSTS["gpt-4o"]
cost_without_routing = (tokens_in / 1000 * gpt4o_costs["input"]) + (tokens_out / 1000 * gpt4o_costs["output"])
savings = cost_without_routing - cost
self.stats["estimated_cost_usd"] += cost
self.stats["savings_vs_gpt4o_usd"] += savings
return RoutingResult(
query=text[:100],
complexity=level,
model_used=model,
answer=answer,
time_seconds=elapsed,
classification_method=method,
classification_reason=reason,
confidence=confidence
)
def stats_report(self) -> str:
"""Builds a report of the usage statistics."""
stats = self.stats
total = stats["total"]
if total == 0:
return "No requests processed"
pct_simple = (stats["simple"] + stats["medium"]) / total * 100
pct_complex = (stats["complex"] + stats["very_complex"]) / total * 100
savings_pct = stats["savings_vs_gpt4o_usd"] / (stats["estimated_cost_usd"] + stats["savings_vs_gpt4o_usd"]) * 100 if stats["estimated_cost_usd"] > 0 else 0
return f"""
=== ROUTING REPORT ===
Total requests: {total}
Simple/Medium (cheap model): {stats['simple'] + stats['medium']} ({pct_simple:.1f}%)
Complex/Very complex (powerful model): {stats['complex'] + stats['very_complex']} ({pct_complex:.1f}%)
Classification:
Via heuristic (free): {stats['via_heuristic']} ({stats['via_heuristic']/total*100:.1f}%)
Via LLM: {stats['via_llm']} ({stats['via_llm']/total*100:.1f}%)
Costs:
Actual cost with routing: ${stats['estimated_cost_usd']:.4f}
Cost without routing (all gpt-4o): ${stats['estimated_cost_usd'] + stats['savings_vs_gpt4o_usd']:.4f}
Savings: ${stats['savings_vs_gpt4o_usd']:.4f} ({savings_pct:.1f}%)
"""
# ============================================================
# USAGE EXAMPLE
# ============================================================
if __name__ == "__main__":
router = ComplexityRouter(RoutingConfig(verbose=True))
requests = [
("What are your customer service hours?", "You are a customer service assistant"),
("Analyze the impact of LLMs on the programming job market", "You are a technology trends analyst"),
("How much does the basic product cost?", "You are a sales assistant"),
("Design a microservices architecture for an e-commerce site with 1M users", "You are a software architect"),
("What is Python?", "You are a programming tutor"),
("Write a comparative analysis of React vs Vue vs Angular considering performance, ecosystem and learning curve", "You are a frontend expert")
]
print("=== ROUTING DEMO ===\n")
results = []
for request, system in requests:
print(f"\nRequest: {request[:60]}...")
result = router.process(request, system)
results.append(result)
print(f"Answer: {result.answer[:100]}...")
print(f"Time: {result.time_seconds:.2f}s")
print(router.stats_report())Routing Across Multiple Dimensions
@dataclass
class MultidimensionalAnalysis:
"""Analysis of multiple factors for advanced routing."""
reasoning_complexity: str # low/medium/high
expected_length: str # short/medium/long
needs_fresh_data: bool # does it need current data?
error_risk: str # low/medium/high (cost of getting it wrong)
sensitivity: str # normal/sensitive (PII, important decisions)
def classify_multidimensional(request: str) -> MultidimensionalAnalysis:
"""Multidimensional analysis for more accurate routing."""
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": f"""Analyze this request:
Request: {request}
JSON:
{{
"reasoning_complexity": "low|medium|high",
"expected_length": "short|medium|long",
"needs_fresh_data": true|false,
"error_risk": "low|medium|high",
"sensitivity": "normal|sensitive"
}}
Guidelines:
- high error_risk: if getting it wrong has financial/legal/medical consequences
- sensitive sensitivity: if it involves personal data or important decisions"""
}],
temperature=0,
response_format={"type": "json_object"}
).choices[0].message.content
data = json.loads(response)
return MultidimensionalAnalysis(**data)
def advanced_routing(request: str, analysis: MultidimensionalAnalysis) -> tuple[str, dict]:
"""
Advanced routing that takes multiple factors into account.
Returns (model, extra_params)
"""
model = "gpt-4o-mini"
params = {"temperature": 0, "max_tokens": 300}
# Update based on the multidimensional analysis
if analysis.reasoning_complexity == "high" or analysis.error_risk == "high":
model = "gpt-4o"
params["max_tokens"] = 1500
if analysis.expected_length == "long":
params["max_tokens"] = max(params["max_tokens"], 1000)
if analysis.sensitivity == "sensitive":
params["temperature"] = 0 # Maximum consistency
# Add a verification instruction for high risk
extra_instruction = ""
if analysis.error_risk == "high":
extra_instruction = "\n\nCRITICAL: This answer has a high impact. Double-check before responding."
return model, {**params, "extra_instruction": extra_instruction}
# Example:
risky_request = "What medication should I take for my type 2 diabetes?"
analysis = classify_multidimensional(risky_request)
model, params = advanced_routing(risky_request, analysis)
print(f"Model: {model}")
print(f"Parameters: {params}")
print(f"Analysis: risk={analysis.error_risk}, sensitivity={analysis.sensitivity}")Advanced Heuristics Without an LLM
import re
from dataclasses import dataclass
@dataclass
class HeuristicRule:
name: str
pattern: str # Regex pattern or keyword
level: ComplexityLevel
confidence: float
is_regex: bool = False
HEURISTIC_RULES = [
# High-complexity patterns
HeuristicRule("deep_analysis", r"(analy[zs]|compare|evaluate|design|strategy)", ComplexityLevel.COMPLEX, 0.85, is_regex=True),
HeuristicRule("complex_code", r"(implement|architecture|microservice|refactor)", ComplexityLevel.COMPLEX, 0.88, is_regex=True),
HeuristicRule("long_document", r"(?:analysis|review|evaluation).+(?:full|detailed|exhaustive)", ComplexityLevel.COMPLEX, 0.90, is_regex=True),
# Low-complexity patterns
HeuristicRule("yes_no_question", r"^(can i|do you|is it|does it|are there|is there)", ComplexityLevel.SIMPLE, 0.82, is_regex=True),
HeuristicRule("hours_price", r"(hours|price|cost|how much does it cost|address)", ComplexityLevel.SIMPLE, 0.90, is_regex=False),
HeuristicRule("greeting", r"^(hi|hey|good morning|good afternoon|hello)", ComplexityLevel.SIMPLE, 0.95, is_regex=True),
]
def apply_heuristic_rules(text: str) -> Optional[tuple[ComplexityLevel, str, float]]:
"""Applies the heuristic rules in order of confidence."""
text_lower = text.lower().strip()
results = []
for rule in HEURISTIC_RULES:
if rule.is_regex:
if re.search(rule.pattern, text_lower):
results.append((rule.confidence, rule.level, rule.name))
else:
if rule.pattern in text_lower:
results.append((rule.confidence, rule.level, rule.name))
if not results:
return None
# Take the rule with the highest confidence
confidence, level, name = max(results, key=lambda x: x[0])
if confidence >= 0.80:
return level, f"Rule: {name}", confidence
return NoneA/B Testing the Router
import random
class RouterABTest:
"""
A/B testing system to compare routing strategies.
Lets you evaluate whether routing improves quality vs. just using gpt-4o.
"""
def __init__(self, control_percentage: float = 0.2):
"""
Args:
control_percentage: % of requests that go to the control group (always gpt-4o)
"""
self.control_percentage = control_percentage
self.experimental_router = ComplexityRouter(RoutingConfig(verbose=False))
self.ab_results = {
"control": {"calls": 0, "cost": 0, "satisfaction": []},
"experimental": {"calls": 0, "cost": 0, "satisfaction": []}
}
def process(self, request: str, feedback_fn=None) -> tuple[str, str]:
"""
Processes with A/B testing.
Args:
request: The user's request
feedback_fn: Optional function that scores the quality of the answer (0-1)
Returns:
Tuple (answer, group)
"""
group = "control" if random.random() < self.control_percentage else "experimental"
self.ab_results[group]["calls"] += 1
if group == "control":
# Always gpt-4o (control)
answer = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": request}],
temperature=0, max_tokens=500
).choices[0].message.content
else:
# Smart router (experimental)
result = self.experimental_router.process(request)
answer = result.answer
if feedback_fn:
score = feedback_fn(request, answer)
self.ab_results[group]["satisfaction"].append(score)
return answer, group
def ab_report(self) -> dict:
"""Builds a comparative report of the A/B test."""
for group in self.ab_results:
sats = self.ab_results[group]["satisfaction"]
self.ab_results[group]["average_satisfaction"] = sum(sats) / len(sats) if sats else None
return self.ab_resultsTroubleshooting
Problem 1: The classifier sends complex requests to the simple model
Symptom: The quality of the answers drops for some queries because the classifier incorrectly marked them as "simple".
Diagnosis:
def calibrate_classifier(
labeled_requests: list[tuple[str, ComplexityLevel]],
router: ComplexityRouter
) -> dict:
"""
Evaluates the accuracy of the classifier on a test set.
"""
correct = 0
errors = []
for request, true_level in labeled_requests:
pred_level, reason, method, conf = router.classify(request)
if pred_level == true_level:
correct += 1
else:
errors.append({
"request": request[:60],
"true": true_level.value,
"predicted": pred_level.value,
"confidence": conf,
"method": method
})
accuracy = correct / len(labeled_requests)
return {"accuracy": accuracy, "errors": errors[:5]}
# Tuning: If there are many false "simples", raise the threshold
# config.simple_threshold = 0.85 # Stricter about classifying as simpleProblem 2: The LLM classifier's latency increases the total time
Symptom: Routing adds 500ms-1s of latency because of the classification call.
Solution: Optimize the classification:
def fast_classification(text: str) -> ComplexityLevel:
"""
Ultra-fast classification using only heuristics + length analysis.
No LLM calls.
"""
length = len(text)
text_lower = text.lower()
# Heuristics ordered from most to least reliable
if length > 500:
return ComplexityLevel.COMPLEX
complex_words = ["analy", "design", "strateg", "implement", "architect"]
if any(w in text_lower for w in complex_words):
return ComplexityLevel.COMPLEX
if length < 80:
return ComplexityLevel.SIMPLE
return ComplexityLevel.MEDIUM # DefaultProblem 3: Over-routing (too many requests go to the expensive model)
Symptom: The expected savings don't materialize because 60%+ goes to gpt-4o.
Diagnosis and tuning:
def analyze_distribution(router: ComplexityRouter, requests: list[str]) -> dict:
"""Analyzes the complexity distribution without generating the answers."""
counts = {"simple": 0, "medium": 0, "complex": 0, "very_complex": 0}
for s in requests:
level, _, _, _ = router.classify(s)
counts[level.value] += 1
total = len(requests)
print("Complexity distribution:")
for level, n in counts.items():
print(f" {level}: {n} ({n/total*100:.1f}%)")
return counts
# If there are too many "complex" ones, review the heuristic rules
# or lower the threshold: config.simple_threshold = 0.65Exercises
Exercise 1: Routing for an e-commerce system
Design a routing system for an e-commerce chatbot. Define heuristic rules for:
- Simple catalog questions (price, availability)
- Order status queries (needs external data)
- Complaints or disputes (requires careful handling)
- Technical questions about products
See solution
ECOMMERCE_RULES = [
# Simple: catalog queries
HeuristicRule("price", r"(price|cost|how much is it|how much does it cost)", ComplexityLevel.SIMPLE, 0.9, True),
HeuristicRule("availability", r"(available|in stock|do you have|got any)", ComplexityLevel.SIMPLE, 0.88, True),
# Complex: external data (orders)
HeuristicRule("order", r"(order|shipment|delivery|tracking|where is)", ComplexityLevel.COMPLEX, 0.92, True),
HeuristicRule("return", r"(return|refund|warranty|claim|complaint)", ComplexityLevel.COMPLEX, 0.90, True),
# Medium: technical queries
HeuristicRule("technical", r"(compatib|specification|measurement|size|works with)", ComplexityLevel.MEDIUM, 0.82, True),
]
ecommerce_config = RoutingConfig(
simple_model="gpt-4o-mini",
complex_model="gpt-4o",
simple_threshold=0.85,
verbose=True
)
ecommerce_router = ComplexityRouter(ecommerce_config)
# Replace the heuristic rules:
ecommerce_router.heuristic = HeuristicClassifier()
# Ideally: extend the class to use ECOMMERCE_RULESExercise 2: Measure the real savings
Implement a system that runs 20 test requests with and without routing, and computes the real savings in API costs.
See solution
def benchmark_routing_vs_no_routing(requests: list[str]) -> dict:
"""Compares costs with and without routing."""
router = ComplexityRouter(RoutingConfig(verbose=False))
costs = {"with_routing": 0, "without_routing": 0}
COST_MINI = {"input": 0.00015, "output": 0.0006}
COST_4O = {"input": 0.0025, "output": 0.01}
for request in requests:
# With routing
result = router.process(request)
model = result.model_used
tokens_in = len(request.split()) * 1.3
tokens_out = len(result.answer.split()) * 1.3
model_costs = COST_MINI if "mini" in model else COST_4O
costs["with_routing"] += (tokens_in/1000 * model_costs["input"] + tokens_out/1000 * model_costs["output"])
costs["without_routing"] += (tokens_in/1000 * COST_4O["input"] + tokens_out/1000 * COST_4O["output"])
savings_pct = (costs["without_routing"] - costs["with_routing"]) / costs["without_routing"] * 100
print(f"With routing: ${costs['with_routing']:.4f}")
print(f"Without routing (all gpt-4o): ${costs['without_routing']:.4f}")
print(f"Savings: {savings_pct:.1f}%")
return costsSummary
- Routing by complexity: Classify requests → direct them to the appropriate model. It can save 60-80% of the costs.
- Two-step classification: Heuristic first (free, instant) → LLM only if the heuristic isn't enough.
- Effective heuristics: Text length, complexity keywords, question patterns.
- Multidimensional routing: Consider error risk, sensitivity, and freshness of data on top of complexity.
- A/B testing: To validate that routing doesn't degrade the perceived quality.
- Calibration: Tune thresholds and rules with real labeled requests.