Module 5: LLMs (Large Language Models) - GPT, Claude and More

2. What is an LLM? Definition, Scale and Capabilities

Next →

Description

In this lesson you'll understand what an LLM (Large Language Model) is in detail: how they're trained, what makes them "large", and what capabilities they have.

What you'll learn:

  • A precise definition of an LLM.
  • Scale (parameters, training data).
  • The training process (pre-training, fine-tuning, RLHF).
  • Emergent capabilities (few-shot learning, reasoning, code generation).
  • Limitations (hallucinations, outdated knowledge, biases).

Definition: What is an LLM?

LLM (Large Language Model): A language model based on Transformers with hundreds of billions of parameters, trained on a massive text corpus to predict the next token.

Key components:

  1. Architecture: A decoder-only Transformer (seen in Module 4).
  2. Scale: 10-1,000+ billion parameters.
  3. Training: Pre-training on a massive corpus (500B-2T tokens) + fine-tuning.
  4. Task: Predicting the next token (autoregressive language modeling).

Scale: Why "Large"

Parameters

Parameters = Weights + Biases (seen in Module 3).

Size comparison:

ModelParametersYearType
BERT-base110M2018Encoder
GPT-21.5B2019Decoder
GPT-3175B2020Decoder (LLM)
GPT-4~1T (rumored)2023Decoder (LLM)
Llama 38B, 70B, 400B2024Decoder (LLM)

Moral: LLMs have 100-1,000× more parameters than traditional models → far more expressive, but also more expensive to train/run.

Training Data

A massive corpus: LLMs are trained on hundreds of billions of tokens of text.

Sources:

  • CommonCrawl: Internet data (web pages).
  • Wikipedia: Structured knowledge.
  • Books: Project Gutenberg, book corpora.
  • Code: GitHub, Stack Overflow (for code LLMs like GPT Codex).
  • Conversations: Reddit, forums.

Example (GPT-3):

  • ~500 billion tokens of text.
  • ~570 GB of compressed text.
  • Mix: 60% CommonCrawl, 16% books, 10% Wikipedia, 3% code.

Compute

Training LLMs is extremely expensive:

ModelEstimated costTimeHardware
GPT-3$4-12M~1 month10,000+ GPUs
GPT-4$100M+ (rumored)~3-6 months25,000+ GPUs
Llama 3 (400B)$10-20M~2 months16,000 GPUs

Moral: Only large companies (OpenAI, Google, Meta, Anthropic) can train LLMs from scratch. As an AI Engineer, you use pre-trained LLMs (via APIs or open-source downloads).

The Training Process

Phase 1: Pre-Training (base training)

Goal: Learning general patterns of language (grammar, facts, basic reasoning).

Task: Next Token Prediction → given the context, predict the next token.

Example:

Input: "The cat sat on the"
Target: "mat"

The model learns to predict "mat" given the preceding context.

Result: A pre-trained model that can generate coherent text, but not necessarily useful text (it can generate toxic content, it doesn't follow instructions well).

Phase 2: Fine-Tuning

Goal: Adjusting the model for specific tasks (e.g. following instructions, answering questions).

Traditional method: Supervised fine-tuning with labeled data.

Example:

Input: "Translate to Spanish: Hello world"
Target: "Hola mundo"

You train the model on thousands of instruction examples → it learns to follow instructions.

Phase 3: RLHF (Reinforcement Learning from Human Feedback)

Goal: Making the model helpful, honest, and harmless (aligned with human values).

Process (simplified):

  1. Generate outputs: The model generates multiple responses to a question.
  2. Humans rank them: Humans rank the responses (best → worst).
  3. Train a reward model: The model learns which responses are "good" according to humans.
  4. Optimize with RL: The model is trained to maximize the reward (generating responses humans prefer).

Result: An aligned model (e.g. ChatGPT) that:

  • Follows instructions better.
  • Refuses inappropriate requests.
  • Admits when it doesn't know something (vs generating a hallucination).

Example: GPT-3 (base) vs ChatGPT (GPT-3.5 + RLHF).

Emergent Capabilities

Emergent capabilities: Abilities that appear only at large scale (they aren't present in small models).

1. Few-Shot Learning

Definition: The model can do tasks it never saw in training with just 2-5 examples.

Example:

Prompt (few-shot):
Translate to Spanish:
- Hello → Hola
- Goodbye → Adiós
- Thank you → Gracias
- How are you? →

Output: "¿Cómo estás?"

Key point: The model was never explicitly trained on translation, but it learns the pattern from the examples and applies it.

2. Reasoning (Chain-of-Thought)

Definition: The model can reason step by step to solve complex problems.

Example:

Prompt: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 balls. How many tennis balls does he have now? Think step by step.

Output:
1. Roger starts with 5 balls.
2. He buys 2 cans, each with 3 balls → 2 × 3 = 6 balls.
3. Total: 5 + 6 = 11 balls.

Answer: 11 balls.

Key point: "Think step by step" (Chain-of-Thought prompting) activates explicit reasoning.

3. Code Generation

Definition: The model can generate functional code in multiple languages.

Example:

Prompt: Write a Python function to calculate factorial.

Output:
def factorial(n):
    if n == 0 or n == 1:
        return 1
    return n * factorial(n - 1)

Specialized models: GPT Codex (GitHub Copilot), Claude 3, GPT-4.

4. Multimodal (GPT-4, Gemini)

Definition: The model can process text + images (or video, audio).

Example:

Input: [An image of a bar chart] + "What does this chart show?"
Output: "This chart shows sales growth over 5 years, with peak in 2022."

Use: Document analysis (PDFs with images), accessibility (describing images for blind people).

LLMs' Limitations

1. Hallucinations

Definition: The model generates incorrect but plausible information (it sounds right statistically, but it's false).

Example:

Prompt: Who won the 2026 soccer World Cup?
Output: "Argentina won the 2026 World Cup, defeating Brazil 3-2 in the final."

Problem: The 2026 World Cup hasn't happened yet (it's in the future), but the model generates a plausible answer.

Why it happens: The LLM predicts the next token statistically; it has no access to a database of facts.

Solution: RAG (connecting the LLM to an up-to-date database), fact-checking, prompts that ask it to "admit if you don't know".

2. Outdated Knowledge

Definition: An LLM only knows what was in its training data (e.g. GPT-4: knowledge through December 2023).

Example:

Prompt: What's the price of Bitcoin today? (asked in 2024)
Output: (an answer based on 2023 data, incorrect)

Solution: Connect the LLM to real-time APIs (e.g. a crypto price API), RAG with up-to-date data.

3. Biases

Definition: An LLM reflects the biases of its training data (the internet has gender, race, etc. biases).

Example: The model can associate certain roles (e.g. "doctor") with the male gender more frequently.

Solution (partial): RLHF helps mitigate biases, but it doesn't eliminate them completely. It requires care when designing prompts and validating outputs.

4. They Don't Have Real Understanding

Definition: LLMs are statistical models; they have no consciousness, no real understanding of the world, and no symbolic reasoning.

Consequence: They can fail at tasks that require deep common sense or complex causal reasoning.

Why this matters for an AI Engineer

1. Understanding capabilities → designing applications

Example: You know LLMs have few-shot learning → you can design applications where users give examples instead of fine-tuning.

2. Understanding limitations → mitigating risks

Example: You know LLMs generate hallucinations → you implement fact-checking, RAG, or interfaces that ask for user confirmation.

3. Model selection

GPT-4: Better reasoning, multimodal → use it for complex tasks.
GPT-3.5: Cheaper → use it for simple tasks.
Claude 3: A large context window → use it for long documents.

Summary

What is an LLM?

  • A decoder-only Transformer with 10-1,000B parameters.
  • Trained on 500B-2T tokens of text.
  • It predicts the next token (autoregressive).

Training:

  1. Pre-training (next token prediction on a massive corpus).
  2. Fine-tuning (adjusting for specific tasks).
  3. RLHF (alignment with human values).

Emergent capabilities:

  • Few-shot learning, reasoning (Chain-of-Thought), code generation, multimodal.

Limitations:

  • Hallucinations, outdated knowledge, biases, no real understanding.

Next step: Lesson 03: Tokenization — How text is converted into tokens, why tokens ≠ words.

← Previous1. Module Introduction: The Era of LLMsNext →3. Tokenization: How Text Becomes Numbers