Module 8: Your First AI System Design

3. Common Patterns: Chatbot, RAG, Agents and Classifier

Next →

Description

You don't need to invent an architecture from scratch. There are common patterns (proven architectures) that solve typical problems.

4 main patterns:

  1. A simple chatbot: The LLM answers questions (without external context).
  2. RAG (Retrieval-Augmented Generation): The LLM + external documents.
  3. Agents: The LLM decides which tool to use (multi-tool).
  4. A classifier: The LLM categorizes text (sentiment, intent, category).

In this lesson you'll understand each pattern, when to use it, and how it's designed.

Pattern 1: A Simple Chatbot

What it is

A chatbot that answers questions using only the LLM's knowledge (without external documents).

When to use it

  • A general FAQ (e.g. "What is Python?").
  • A conversational assistant (e.g. "Help me write an email").
  • It doesn't require information specific to your product/company.

Architecture

User → Frontend → Backend → LLM API → Response → Frontend

Components:

  • Frontend: A chat interface (React).
  • Backend: FastAPI (handles the requests).
  • LLM: GPT-3.5-turbo (a quality/cost balance).
  • DB: PostgreSQL (conversations).

Flow

  1. User: "What is machine learning?"
  2. The backend builds the prompt: 
    System: You are a technical assistant with expertise in AI.
User: What is machine learning?
  3. The backend calls GPT-3.5.
  4. The LLM answers: "Machine learning is a subset of AI..."
  5. The backend saves the conversation in the DB.
  6. The backend returns the response to the frontend.

A real example

ChatGPT (in simple mode): It doesn't use external documents, only the model's knowledge.

Limitations

  • Outdated knowledge: The LLM was trained up to a certain date (e.g. GPT-4: April 2023).
  • No specific information: It doesn't know about your product/company.
  • Hallucinations: It can make up information.

Pattern 2: RAG (Retrieval-Augmented Generation)

What it is

A chatbot that uses external documents to answer (combining retrieval + generation).

When to use it

  • Q&A over documentation (a product, code, manuals).
  • Technical support (searching a knowledge base).
  • Specific information the LLM doesn't know.

Architecture

User query → Backend → Embed the query → Vector DB (search the docs) → Retrieve the top K docs → LLM (generate a response with the docs) → Backend → Frontend

Components:

  • Frontend: A chat interface.
  • Backend: FastAPI + LangChain (RAG orchestration).
  • Embeddings: OpenAI embeddings (text-embedding-3-small).
  • Vector DB: Pinecone (it stores the docs' embeddings).
  • LLM: GPT-4 (reasoning over the docs).
  • DB: PostgreSQL (conversations).

Flow

  1. Ingestion (once):

    • Docs (PDFs, Markdown) → chunks (512 tokens) → embeddings → Pinecone.

  2. Query (each request):

    • User: "How do I reset my password?"
    • The backend embeds the query (a vector).
    • The backend searches Pinecone (the top 5 similar docs by cosine similarity).
    • The backend builds the prompt: 
      System: Use these documents to answer.
Documents:
1. [a doc about resetting a password]
2. [a doc about security]
...
User: How do I reset my password?
    • The backend calls GPT-4.
    • The LLM answers based on the docs.
    • The backend returns the response.

A real example

ChatGPT with "Browse" mode: It searches the web and uses the results to answer.

Advantages over a Simple Chatbot

  • ✅ Up-to-date information (the docs can be updated).
  • ✅ Specific information (your product, your company).
  • ✅ Fewer hallucinations (the LLM cites the docs).

Disadvantages

  • ❌ More complex (a vector DB, embeddings).
  • ❌ More expensive (embeddings + an LLM call).
  • ❌ Slower (retrieval + generation).

Pattern 3: Agents (Multi-Tool)

What it is

An LLM that decides which tool to use to solve a task (search, calculator, database, an API call).

When to use it

  • Complex tasks that require multiple steps.
  • Access to external tools (APIs, databases, search).

Architecture

User query → Backend → Agent (the LLM decides the tool) → Tool 1 (search) → Tool 2 (calculator) → ... → LLM (synthesize) → Backend → Frontend

Components:

  • Frontend: A chat interface.
  • Backend: FastAPI + LangChain (a ReAct agent).
  • LLM: GPT-4 (reasoning + tool calling).
  • Tools: A search API (Google), a calculator, a database query, a weather API.
  • DB: PostgreSQL (conversations).

Flow (Example)

User: "What's AAPL's price today and how much is 100 shares?"

  1. The backend sends the query to the agent (the LLM).
  2. The LLM reasons: 
    Thought: I need AAPL's price.
Action: search("AAPL stock price today")
  3. The backend runs the search tool → result: "$150".
  4. The LLM reasons: 
    Thought: Now I compute 100 × 150.
Action: calculator("100 * 150")
  5. The backend runs the calculator → result: "15000".
  6. The LLM synthesizes: 
    Thought: I have both results.
Final Answer: AAPL costs $150 today. 100 shares cost $15,000.
  7. The backend returns the response.

A real example

ChatGPT with Plugins: It can call external APIs (Kayak for flights, Wolfram Alpha for math).

Advantages

  • ✅ Complex tasks (multiple steps).
  • ✅ Access to real-time information (APIs).

Disadvantages

  • ❌ Very complex (orchestration, loops).
  • ❌ Expensive (multiple LLM calls).
  • ❌ It can fail (an infinite agent loop, incorrect tool calling).

Pattern 4: A Classifier

What it is

An LLM that categorizes text (sentiment, intent, category).

When to use it

  • Classifying support tickets (billing, technical, account).
  • Sentiment analysis (positive, negative, neutral).
  • Intent detection (purchase, inquiry, complaint).

Architecture

User text → Backend → LLM API (a classification prompt) → Category → Backend → Store in the DB

Components:

  • Backend: FastAPI.
  • LLM: GPT-3.5-turbo (simple classification).
  • DB: PostgreSQL (storing the text + the category).

Flow

User input: "My account isn't working, I need help urgently."

  1. The backend builds the prompt: 
    System: Classify this ticket as: billing, technical, account.
User: My account isn't working, I need help urgently.
Output: [category]
  2. The backend calls GPT-3.5.
  3. The LLM answers: "account".
  4. The backend stores it in the DB (ticket_id, text, category: "account").
  5. The backend routes the ticket to the appropriate team (account support).

A real example

Zendesk AI: It classifies tickets automatically.

Advantages

  • ✅ Simple (one LLM call).
  • ✅ Cheap (GPT-3.5 is enough).
  • ✅ Fast (latency <1s).

Disadvantages

  • ❌ Limited to classification (it doesn't generate long responses).

Comparison of Patterns

AspectSimple ChatbotRAGAgentsClassifier
ComplexityLowMediumHighLow
CostLowMediumHighLow
Latency<1s1-3s3-10s<1s
UseA general FAQQ&A over docsComplex tasksCategorization
ExampleSimple ChatGPTNotion AIChatGPT PluginsZendesk AI

When to Use Each Pattern

Use a Simple Chatbot when:

  • A general FAQ (it doesn't require specific docs).
  • A conversational assistant.
  • A limited budget.

Use RAG when:

  • Q&A over documentation (a product, code).
  • Specific information the LLM doesn't know.
  • Reducing hallucinations (the LLM cites the docs).

Use Agents when:

  • Complex tasks (multiple steps).
  • Access to external tools (APIs, search).
  • A high budget (GPT-4 + multiple calls).

Use a Classifier when:

  • Categorizing text (tickets, sentiment, intent).
  • You don't need to generate long responses.
  • A limited budget, critical latency.

Hybrid Patterns

Many systems combine patterns:

Example: A Chatbot with RAG + Classification

  1. The classifier: It detects the intent (FAQ vs technical support).
  2. If FAQ: A simple chatbot (GPT-3.5, fast, cheap).
  3. If technical support: RAG (search the docs, GPT-4, accurate).

Advantage: It optimizes cost (80% FAQ uses GPT-3.5, 20% technical uses RAG + GPT-4).

Why this matters for an AI Engineer

1. Not reinventing the wheel

Without knowing the patterns:

  • "How do I implement a chatbot over docs?" → you invent a solution from scratch.

Knowing the patterns:

  • "This is RAG" → you use an existing pattern (LangChain has templates).

2. Communicating with the team

The Product Manager: "We want a chatbot that answers questions about our product."

An AI Engineer (without patterns): "Okay, I'll see what I can do."

An AI Engineer (with patterns): "That's RAG. I need: docs → embeddings → a vector DB (Pinecone) → GPT-4. Timeline: 3 weeks. Estimated cost: $500/month."

Summary

4 common patterns:

  1. A simple chatbot: The LLM answers (without external docs). Use: a general FAQ.
  2. RAG: The LLM + external docs. Use: Q&A over documentation.
  3. Agents: The LLM + tools (search, calculator, APIs). Use: complex tasks.
  4. A classifier: The LLM categorizes text. Use: tickets, sentiment, intent.

Comparison:

  • A simple chatbot: Low cost, low complexity, <1s latency.
  • RAG: Medium cost, medium complexity, 1-3s latency.
  • Agents: High cost, high complexity, 3-10s latency.
  • A classifier: Low cost, low complexity, <1s latency.

When to use:

  • A general FAQ → a simple chatbot.
  • Q&A over docs → RAG.
  • Complex tasks → agents.
  • Categorization → a classifier.

Hybrid patterns: Combining patterns (e.g. classification + RAG) to optimize.

Next step: Lesson 04: Design Trade-Offs — Cost vs latency vs quality.

← Previous2. The Components of an AI System: Frontend, Backend, LLM, Vectors and DBsNext →4. Design Trade-Offs: Cost vs Latency vs Quality