Module 4: Transformers - The Modern AI Revolution
5. The Transformer Architecture: Encoder, Decoder and Positional Encoding
Description
You've already seen self-attention (Lesson 04) and attention in general (Lesson 03). Now you're going to understand how all those components come together in the complete architecture of a Transformer.
What you'll learn:
- The Transformer's components: Encoder, Decoder, Positional Encoding, Multi-Head Attention, Feed-Forward.
- The differences: Encoder-only (BERT), Decoder-only (GPT), Encoder-Decoder (T5).
- How data flows through the Transformer.
The complete architecture: Encoder-Decoder
Simplified diagram
Input (English)
↓
[Positional Encoding]
↓
[ENCODER]
- Multi-Head Self-Attention
- Feed-Forward
(×N layers)
↓
Representation of the input
↓
[DECODER]
- Masked Multi-Head Self-Attention
- Cross-Attention (attends to the encoder)
- Feed-Forward
(×N layers)
↓
Output (Spanish)
Component 1: Positional Encoding
Problem: Self-attention processes all the words simultaneously (in parallel) → it doesn't know the order of the words.
Example: "The cat sat" vs "sat cat The" → without order, they mean the same thing to self-attention (but they're clearly different!).
Solution: Positional Encoding → adding position information to each word.
How it works:
- Each word has an embedding (a vector that represents its meaning).
- A positional vector (a vector that represents its position in the sentence) is added to it.
- Result:
final_embedding = word_embedding + positional_encoding
Example:
"The" (position 1) → embedding("The") + pos_encoding(1)
"cat" (position 2) → embedding("cat") + pos_encoding(2)
"sat" (position 3) → embedding("sat") + pos_encoding(3)
Moral: Positional encoding lets the model know that "cat" comes after "The" and before "sat".
Component 2: Encoder
What it does: It processes the input (e.g. a sentence in English) and generates a rich representation of each word (capturing context, relationships, meaning).
Structure:
Input + Positional Encoding
↓
Multi-Head Self-Attention (the words "see" each other)
↓
Add & Norm (residual sum + normalization)
↓
Feed-Forward (additional per-word processing)
↓
Add & Norm
↓
[Repeat N times, e.g. 12 layers in BERT]
The encoder's output: A contextual representation of each word of the input.
Component 3: Decoder
What it does: It generates the output (e.g. a sentence in Spanish) word by word, attending to the input (via cross-attention) and to the words already generated (via masked self-attention).
Structure:
Partial output (e.g. "El gato") + Positional Encoding
↓
Masked Multi-Head Self-Attention (the output attends to the output generated so far)
↓
Add & Norm
↓
Cross-Attention (the output attends to the encoder → the input in English)
↓
Add & Norm
↓
Feed-Forward
↓
Add & Norm
↓
[Repeat N times, e.g. 12 layers]
↓
Linear + Softmax → predicts the next word
Key point: The decoder generates sequentially (word by word), but each step uses attention (parallel internally).
Differences: Encoder-only, Decoder-only, Encoder-Decoder
1. Encoder-only (BERT)
Architecture: Only an encoder (no decoder).
How it works:
- It processes the complete input (e.g. a sentence in English).
- It generates a contextual representation of each word.
- It uses that representation for classification, NER, Q&A, etc.
It does NOT generate text: It only understands/classifies.
Use: Text classification, NER, sentiment analysis, Q&A.
Examples: BERT, RoBERTa, ALBERT.
2. Decoder-only (GPT)
Architecture: Only a decoder (no encoder).
How it works:
- It processes the input (e.g. "The cat sat on").
- It predicts the next word (e.g. "the").
- It repeats (now the input is "The cat sat on the", it predicts "mat").
It generates text sequentially: Autocomplete.
Use: Text generation, chat, autocomplete, translation (less common than encoder-decoder).
Examples: GPT-2, GPT-3, GPT-4, Claude, Llama.
3. Encoder-Decoder (T5, BART)
Architecture: Encoder + Decoder.
How it works:
- The encoder processes the input (e.g. a sentence in English).
- The decoder generates the output (e.g. a sentence in Spanish), attending to the encoder via cross-attention.
It transforms text: Input → Output (translation, summarization, etc.).
Use: Translation, summarization, format transformation.
Examples: T5, BART, mT5.
Comparison
| Type | Architecture | Use | Example |
|---|---|---|---|
| Encoder-only | Only an encoder | Comprehension, classification | BERT |
| Decoder-only | Only a decoder | Generation, chat | GPT-4, Claude |
| Encoder-Decoder | Encoder + Decoder | Translation, summarization | T5, BART |
Component 4: Feed-Forward Network
What it is: After each attention layer, there's a feed-forward network (2 fully connected layers) that processes each word independently.
Why it matters: It adds additional non-linear transformation capacity (beyond attention).
Structure:
Input (the attention output)
↓
Linear (e.g. 768 → 3072 dimensions)
↓
ReLU (activation)
↓
Linear (e.g. 3072 → 768 dimensions)
↓
Output
Component 5: Add & Norm (Residual Connections + Layer Normalization)
Add (Residual Connection): It sums the layer's input with its output → it helps train deep networks (avoiding vanishing gradients).
Norm (Layer Normalization): It normalizes the values → it stabilizes training.
Why it matters: It makes it possible to train Transformers with many layers (e.g. GPT-3: 96 layers).
Why this matters for an AI Engineer
1. Selecting models by architecture
- Classifying text: Encoder-only (BERT).
- Generating text: Decoder-only (GPT).
- Translating text: Encoder-Decoder (T5).
2. Interpreting documentation
"BERT-base: 12 encoder layers, 768 hidden size, 12 attention heads"
- 12 encoder layers (each one with multi-head attention + feed-forward).
- 768 dimensions per vector (the size of the embeddings).
- 12 attention heads per layer.
3. Fine-tuning
If you do fine-tuning:
- BERT: You add a classification layer on top of the encoder.
- GPT: You adjust the decoder for your task (e.g. generating responses specific to your domain).
- T5: You adjust the encoder-decoder for your transformation task.
Summary
The complete Transformer architecture:
- Positional Encoding: Adds position information.
- Encoder: Processes the input (multi-head self-attention + feed-forward × N layers).
- Decoder: Generates the output (masked self-attention + cross-attention + feed-forward × N layers).
- Add & Norm: Residual connections + normalization (they make it possible to train deep networks).
Three variants:
- Encoder-only (BERT): Comprehension/classification.
- Decoder-only (GPT): Generation/chat.
- Encoder-Decoder (T5): Translation/summarization.
Next step: Lesson 06: Why Transformers Won — A detailed comparison with RNNs (parallelization, scalability, long context).
Additional resources
-
The Illustrated Transformer (Jay Alammar) — A complete visualization. In English.
-
Attention Is All You Need (paper) — The original paper. Very technical.