Module 5: LLMs (Large Language Models) - GPT, Claude and More
1. Module Introduction: The Era of LLMs
Description
Welcome to Module 5: LLMs (Large Language Models). This is the module where everything you learned (Transformers, attention, neural networks) connects with the technology you use every day: ChatGPT, Claude, Gemini, and every modern AI assistant.
LLMs are decoder-only Transformers (the architecture you saw in Module 4) trained with hundreds of billions of parameters on massive text corpora. They're the reason AI exploded in popularity from 2022 onward.
In this module you'll understand:
- What LLMs are (scale, training, capabilities).
- How they work internally (tokenization, embeddings, context window).
- How to use them effectively (model parameters, provider comparison).
Approach: Conceptual but practical. You'll see how to use LLMs (via APIs), NOT how to train them from scratch (that's extremely expensive and complex).
The LLM Revolution (2020-2024)
2020: GPT-3 - The Turning Point
OpenAI releases GPT-3:
- 175 billion parameters (100× more than GPT-2).
- Trained on ~500 billion tokens of text from the internet.
- Emergent capabilities: Few-shot learning (it can do tasks it never saw in training with just 2-3 examples).
Impact: It shows that scale works (more data + more parameters = qualitatively different capabilities).
2022: ChatGPT - The Massive Boom
OpenAI releases ChatGPT (based on GPT-3.5):
- GPT-3.5 fine-tuned with RLHF (Reinforcement Learning from Human Feedback) to follow instructions and be useful.
- A chat interface accessible to anyone (not just via API).
Result:
- 100 million users in 2 months (the fastest-growing application in history).
- A massive boom in AI startups, investment, and enterprise adoption.
Why it worked: The combination of a very capable model (GPT-3.5) + a simple interface (chat) + RLHF (the model follows instructions well) = a magical experience.
2023: Massive Competition
GPT-4 (OpenAI, March 2023):
- ~1 trillion parameters (rumored, OpenAI doesn't confirm).
- Multimodal (text + images).
- A significant improvement in reasoning, math, code.
Claude 2 (Anthropic, July 2023):
- A direct competitor to GPT-4.
- An emphasis on safety and accuracy.
- A context window of 100K tokens (vs GPT-4: 32K at the time).
Llama 2 (Meta, July 2023):
- Open-source (you can download the weights and run it locally).
- 7B, 13B, 70B parameters (versions of different sizes).
- Free for commercial use.
Gemini (Google, December 2023):
- Multimodal (text, image, audio, video).
- Integrated into Google products (Bard, Google Search, etc.).
2024: Continuous Improvements
Claude 3 (Anthropic, March 2024):
- A context window of 200K tokens (~150K words → ~500 pages).
- 3 variants: Haiku (fast, cheap), Sonnet (balanced), Opus (most capable).
Llama 3 (Meta, April 2024):
- 8B, 70B, 400B parameters (the largest version).
- Open-source, free.
- Performance comparable to GPT-4 on many tasks.
GPT-4 Turbo (OpenAI, 2024):
- A context window of 128K tokens (~96K words).
- Cheaper than the original GPT-4.
- Knowledge updated through December 2023 (vs April 2023 in the original GPT-4).
Moral: The field evolves rapidly. New models every 3-6 months with improvements in capability, context window, and price.
What LLMs are
Definition
LLM (Large Language Model): A language model (it predicts the next word) with hundreds of billions of parameters, trained on a massive text corpus (internet, books, code).
Key characteristics:
- Scale: 10-1,000 billion parameters (vs traditional models: <1B).
- Training: Pre-trained on a massive corpus (500B-2T tokens), then fine-tuned.
- Emergent capabilities: Few-shot learning, reasoning, code generation (they appear only at large scale).
LLMs are Decoder-Only Transformers
Reminder from Module 4:
- Encoder-only (BERT): Comprehension/classification.
- Decoder-only (GPT): Generation/autocomplete.
- Encoder-Decoder (T5): Transformation.
Modern LLMs (GPT-4, Claude, Llama): They're all decoder-only.
Why decoder-only:
- They generate text sequentially (word by word).
- Autoregressive: They predict the next token given the previous context.
- Scalable to trillions of parameters.
Why this matters for an AI Engineer
1. LLMs are your main tool
As an AI Engineer (2024), almost everything you build involves LLMs:
- Chatbots: Claude, GPT-4 via API.
- RAG (Retrieval-Augmented Generation): Combining an LLM with a vector database.
- Agents: LLMs that can use tools (calling APIs, running code).
- Code generation: GitHub Copilot (GPT Codex), Cursor (GPT-4).
- Content generation: Summarization, translation, copywriting.
- Data extraction: Extracting structured data from unstructured text.
- Classification: Sentiment analysis, intent detection, categorization.
Statistics:
- 80% of AI Engineering projects (2024) use LLMs as their main component.
- 60% of AI startups (2023-2024) build on LLM APIs (OpenAI, Anthropic).
Moral: If you don't understand LLMs, you can't work effectively as an AI Engineer.
Analogy: LLMs are to AI Engineers what SQL is to Backend Engineers → a fundamental tool you use daily.
2. You need to understand the internals in order to optimize
Example: Your chatbot is slow and expensive.
Diagnosis (it requires understanding the internals):
- Tokenization: Your prompts use a lot of tokens (a non-English language, verbose formatting) → optimize the prompts.
- Context window: You pass complete long documents (100K tokens) when you only need the relevant sections → use RAG.
- Parameters: You use temperature 1 (very creative) when you only need deterministic responses → switch to temperature 0.
Without understanding the internals: You don't know why it's slow/expensive or how to optimize it.
3. Model selection
Task: A chatbot for customer support.
Options:
| Model | Pros | Cons | Cost (1M tokens) |
|---|---|---|---|
| GPT-4 | Smarter, better reasoning | More expensive, slower | $30 |
| GPT-3.5 | Cheaper, faster | Less capable | $2 |
| Claude 3 Haiku | Very fast, very cheap | Less capable than GPT-4 | $1 |
| Llama 3 (local) | Free | Requires infrastructure, less capable | $0 (hosting) |
Decision: It depends on budget, traffic volume, required quality.
Real example:
- A startup (MVP): 10K requests/month → GPT-3.5 ($20/month) is enough.
- A company (production): 1M requests/month → a multi-model strategy:
- 80% simple queries → GPT-3.5 ($1,600/month).
- 20% complex queries → GPT-4 ($6,000/month).
- Total: $7,600/month (vs $30,000/month with pure GPT-4 → a 75% saving).
Without understanding LLMs: You can't evaluate trade-offs (intelligence vs cost vs latency) or optimize your budget.
What you'll learn in this module
Key concepts (8 lessons)
| # | Lesson | What you'll learn |
|---|---|---|
| 01 | Module introduction | The 2020-2024 chronology, why LLMs matter (this lesson) |
| 02 | What is an LLM? | Definition, scale, training, emergent capabilities |
| 03 | Tokenization | Text → tokens, why tokens ≠ words, cost per token |
| 04 | Embeddings | Vectors of meaning, semantic search, similarity |
| 05 | Context window | The memory limit, why it exists, solutions (RAG) |
| 06 | Model parameters | Temperature, top_p, max_tokens → controlling generation |
| 07 | The major LLMs | GPT-4, Claude 3, Llama 3, Gemini → a comparison |
| 08 | Integrative exercise | Experimenting with parameters, calculating costs |
Connection with previous modules
This module integrates concepts from earlier modules:
- Module 2 (ML): LLMs use supervised learning (pre-training) + reinforcement learning (RLHF).
- Module 3 (Neural Networks): LLMs are deep neural networks (billions of parameters).
- Module 4 (Transformers): LLMs are decoder-only Transformers (the GPT architecture).
Without the previous modules: LLMs would be "black boxes" (you wouldn't understand how they work internally).
With the previous modules: You understand that LLMs are Transformers trained with supervised learning + RLHF, with billions of parameters organized in neural layers.
Connection with the following modules
This module prepares you for:
- Module 6 (The API ecosystem): How to access LLMs via APIs (OpenAI, Anthropic), pricing, local vs cloud.
- Module 7 (AI Engineering): How LLMs are the AI Engineer's main tool.
- Module 8 (System Design): How to design systems that use LLMs (chatbots, RAG, agents).
Common mistakes when learning about LLMs
1. Thinking that LLMs "understand" like humans
Mistake: Assuming LLMs have real comprehension of the world (like humans).
Reality: LLMs are statistical models that predict the next token given the context. They have no consciousness, no real understanding, and no symbolic reasoning.
Example:
- User: "How many Rs are there in 'strawberry'?"
- GPT-4: "There are 2 Rs in 'strawberry'" (correct answer: there are 3).
Why it fails: The LLM processes tokens, not individual characters. "strawberry" can be 1-2 tokens, not 10 separate characters.
Consequence: LLMs can generate hallucinations (incorrect but plausible information) because they "sound right" statistically, not because they're true.
How to mitigate:
- Cite sources (RAG).
- Clear system prompts ("If you don't know, say 'I don't have that information'").
- Use GPT-4 (fewer hallucinations than GPT-3.5).
2. Not understanding tokens (expensive)
Mistake: Thinking that "1 token = 1 word" and not optimizing prompts.
Reality: In English, 1 token ≈ 0.75 words. In non-English languages (Spanish, Chinese), 1 word can be 2-4 tokens.
Example:
- English: "Hello world" = 2 tokens.
- Spanish: "Hola mundo" = 3 tokens (50% more expensive).
- Chinese: "你好世界" = 4-8 tokens (2-4× more expensive).
Consequence: Your costs are 2-4× higher than expected if you work in non-English languages.
Solution:
- Use the OpenAI Tokenizer to count exact tokens.
- Optimize prompts (concise > verbose).
- Consider a model specific to your language if one exists.
3. Ignoring the context window
Mistake: Passing 200K-word documents to GPT-4 (context window 128K tokens) and expecting it to work.
Reality: If you exceed the context window, the model truncates the input or gives an error → information is lost.
Real example:
- Document: A 500-page PDF (~150K words → ~200K tokens).
- GPT-4: A context window of 128K tokens → it can process only ~96K words (~320 pages).
- Result: The model truncates the last 180 pages → incomplete answers.
Solution: Use RAG (search for the relevant sections), split it into chunks, or use Claude 3 (200K tokens) / Gemini 1.5 (1M tokens).
4. Assuming that "more parameters = always better"
Mistake: Thinking that GPT-4 (1T params, rumored) is better for EVERY task vs GPT-3.5 (175B).
Reality: For simple tasks (FAQ, basic classification), GPT-3.5 is enough and 30× cheaper.
Example:
- Task: Classifying a support ticket (billing, technical, account).
- GPT-4: 95% accuracy, cost $30/1M tokens.
- GPT-3.5: 92% accuracy, cost $1/1M tokens.
- Trade-off: Is 3% more accuracy worth 30× the cost?
Lesson: Choose the appropriate model for the task (not always the biggest one).
Summary
The era of LLMs:
- 2020: GPT-3 (175B parameters) shows that scale works.
- 2022: ChatGPT → a massive boom (100M users in 2 months).
- 2023-2024: GPT-4, Claude 3, Llama 3, Gemini → competition, continuous improvements.
What LLMs are:
- Decoder-only Transformers with hundreds of billions of parameters.
- Trained on a massive corpus (500B-2T tokens).
- Emergent capabilities (few-shot, reasoning, code).
Why it matters:
- LLMs are the AI Engineer's main tool.
- You need to understand the internals (tokenization, embeddings, context window) in order to optimize.
- Model selection requires evaluating trade-offs (intelligence vs cost vs latency).
Next step: Lesson 02: What is an LLM? — A detailed definition, how they're trained, emergent capabilities.
Additional resources
- OpenAI GPT-4 Technical Report — GPT-4's technical paper. In English.
- Anthropic Claude Documentation — Claude's official docs. In English.
- Llama 3 Model Card — Llama 3's documentation. In English.