Module 3: Neural Networks & Deep Learning

1. Module Introduction: Neural Networks & Deep Learning

Next →

Description

Welcome to Module 3: Neural Networks & Deep Learning. This is the module where everything starts to connect: in Module 1 you saw what AI is, in Module 2 you understood Machine Learning (supervised, unsupervised, training vs inference), and now you're going to understand what neural networks are and how they work.

Neural networks are the technology behind practically all the modern AI you use: ChatGPT, DALL-E, facial recognition, machine translation, Netflix recommendations, self-driving cars, etc. If you don't understand how they work, you won't be able to work effectively as an AI Engineer.

The most important thing: This module is conceptual and visual, NOT mathematical. You won't see backpropagation formulas or derivatives; you'll see diagrams, analogies and architectural reasoning. The goal is NOT for you to learn to train neural networks from scratch (other roles do that); the goal is for you to understand how they work so you can integrate them, optimize them and communicate with technical teams.

Why do you need to understand Neural Networks?

1. Context: All modern AI uses Neural Networks

When you work with AI today (2024-2026), almost everything you use is deep neural networks (Deep Learning):

  • LLMs (ChatGPT, Claude, Gemini): Transformer neural networks with billions of parameters.
  • Image generation (DALL-E, Stable Diffusion, Midjourney): Generative neural networks (GANs, Diffusion Models).
  • Speech recognition (Siri, Alexa): Neural networks for processing audio.
  • Computer vision (facial recognition, object detection): CNNs (Convolutional Neural Networks).
  • Machine translation (Google Translate): Transformers (you'll see them in Module 4).

If you don't understand what a neural network is, how it's organized, and how it learns, working with these models is like driving a car without knowing what the engine does: it works, but you can't diagnose problems, optimize, or make informed technical decisions.

2. As an AI Engineer, you need to understand architectures

Your job is NOT to design new neural networks from scratch (Research Scientists do that). Your job is to:

  1. Integrate pre-trained models into applications (e.g. using a Hugging Face model to classify text).
  2. Optimize models for production (reducing latency, cost, size).
  3. Fine-tune models when necessary (adjusting a pre-trained model to your specific domain).
  4. Diagnose problems (e.g. the model is slow → how many layers does it have? how many parameters? can I use a smaller version?).
  5. Communicate technical decisions (e.g. explaining to product why you need a GPU, why a model takes X seconds, why fine-tuning requires labeled data).

To do all of this, you need to understand neural network architectures:

  • What is a layer?
  • What are parameters (weights)?
  • What does "deep" network mean?
  • What architectures exist (CNN, RNN, Transformer) and when to use each one?
  • How does a neural network learn (forward pass, backpropagation)?
  • Why are some networks slower than others?

Without this knowledge, you're a "copy-paste engineer": You follow tutorials without understanding what they do, and when something fails you don't know why or how to fix it.

3. Concrete example: Debugging

Imagine you integrate an image classification model (e.g. identifying whether a photo is of a dog or a cat) and the model is slow (it takes 2 seconds per image). Product asks you: "Can we make it faster?"

If you understand Neural Networks:

  • You review the architecture: the model has 50 convolutional layers (ResNet-50), 25 million parameters.
  • You know that each layer adds latency (forward pass: passing the image through 50 layers takes time).
  • You propose solutions:
    • Use a smaller model (ResNet-18: 18 layers, 11M parameters → faster but less accurate).
    • Reduce the input resolution (from 512x512 to 256x256 → fewer pixels processed by each layer).
    • Use optimization techniques (model quantization: reducing parameter precision from float32 to int8 → faster, lower precision).
  • You explain the trade-offs to product: "We can reduce latency to 0.5 seconds using ResNet-18, but we'll lose 2% accuracy. Is that acceptable?"

If you do NOT understand Neural Networks:

  • You don't know why the model is slow.
  • You don't know what "ResNet-50" or "25M parameters" means.
  • You can't propose technical solutions.
  • You depend completely on others (an ML Engineer, Stack Overflow, tutorials).

Moral: Understanding architectures gives you technical autonomy and the ability to make informed decisions.

4. Difference from traditional ML/Deep Learning courses

Academic courses (e.g. Stanford CS231n, the global course platform Deep Learning):

  • They teach math (linear algebra, calculus, derivatives, gradient descent).
  • They teach you to implement backpropagation from scratch.
  • Goal: Training Research Scientists or ML Engineers who design new architectures.

This module:

  • It teaches concepts (what a neuron is, how it's organized into layers, how information flows).
  • It teaches you to reason about architectures (when to use CNN, RNN, feedforward).
  • Goal: Training AI Engineers who integrate existing models into applications.

Analogy: Traditional courses teach you to design car engines. This module teaches you how an engine works so you can drive, diagnose problems, and choose the right car for each situation.

Do you need to learn the math? Only if you want to do research or design new architectures. For AI Engineering (integration, optimization, fine-tuning), you do NOT need to implement backpropagation from scratch; frameworks (PyTorch, TensorFlow) do it for you. But you do need to understand the concepts (what backpropagation is, why it matters, how it affects training).

What will you learn in this module?

Key concepts (8 lessons)

#LessonWhat you'll learn
01Module introductionWhy you need to understand NNs, the module's roadmap (this lesson)
02What is a Neural Network?The artificial neuron, biological inspiration, the perceptron, how it decides
03Layers and architectureInput layer, hidden layers, output layer, fully connected, parameters
04Activation functionsWhy they matter, ReLU, sigmoid, tanh, non-linearity (without formulas)
05Forward passHow data flows from input to output, computing a prediction
06BackpropagationHow the network learns (adjusting weights based on error), conceptually, without derivatives
07Specialized architecturesCNN (images), RNN (sequences), feedforward (tables), when to use each one
08Integrative exerciseDesigning architectures for given problems, identifying mistakes

What you will NOT see (and why you don't need it)

  • ❌ Mathematical formulas for backpropagation (derivatives, gradients): Frameworks (PyTorch, TensorFlow) implement them automatically. You use .backward() or .fit() and the framework computes the gradients.

  • ❌ Advanced linear algebra (matrix multiplication, eigenvalues): Useful for understanding framework internals, but NOT necessary for using pre-trained models or doing fine-tuning.

  • ❌ Implementing neural networks from scratch (without frameworks): Useful as an academic exercise, but in practice you use PyTorch, TensorFlow, Keras, Hugging Face Transformers.

Why don't you need it? Because your role (AI Engineer) is to integrate existing models, NOT to design architectures from scratch. It's like learning to drive: you don't need to design the engine, but you do need to understand what the accelerator, the brake, and the steering wheel do.

If you want to learn it (optional): At the end of each lesson there are additional resources (e.g. 3Blue1Brown: Neural Networks, Fast.ai, Stanford CS231n).

Module roadmap

Phase 1: Fundamentals (Lessons 02-03)

Goal: Understand what an artificial neuron is and how it's organized into layers.

  • Lesson 02: What is a neuron? Biological inspiration (a real neuron), simplified model (the perceptron), how it decides (inputs → weights → sum → activation → output).
  • Lesson 03: How are neurons organized? Layers (input, hidden, output), fully connected (each neuron connected to all the neurons in the next layer), parameters (weights + biases).

Central analogy: A neuron = a worker who decides yes/no based on inputs. A neural network = a factory with departments (layers) that process information sequentially.

Phase 2: How they learn (Lessons 04-06)

Goal: Understand how a neural network processes data (forward pass) and how it learns (backpropagation).

  • Lesson 04: Why activation functions? Without activation, the network is linear (it can only learn linear relationships). With activation (ReLU, sigmoid), the network can learn complex (non-linear) relationships.
  • Lesson 05: How does information flow? Forward pass: data enters through the input layer, passes through hidden layers (each layer transforms the data), exits through the output layer (a prediction).
  • Lesson 06: How does it learn? Backpropagation (conceptually): compute the error (prediction vs reality), propagate the error backward (from output to input), adjust the weights to reduce the error.

Central analogy: Forward pass = the factory processes a product (data → layers → output). Backpropagation = the boss gives feedback (the error), the feedback propagates to the departments (layers) so they improve.

Phase 3: Specialized architectures (Lesson 07)

Goal: Understand what architectures exist and when to use each one.

  • CNNs (Convolutional Neural Networks): For images (they capture spatial patterns: edges, textures, shapes). Examples: ResNet, VGG, MobileNet.
  • RNNs (Recurrent Neural Networks): For sequences (they capture temporal dependencies: time series prediction, text). Examples: LSTM, GRU.
  • Feedforward (Fully Connected): For tabular data (tables with rows and columns). Examples: house price prediction, binary classification.

When to use each one:

  • Classifying images → CNN
  • Processing sequential text → RNN (or Transformer, which you'll see in Module 4)
  • Predicting a house price (tabular data) → Feedforward

Phase 4: Practice (Lesson 08)

Goal: Apply the concepts by designing architectures for real problems.

Exercises:

  1. Given a problem (e.g. "classify emails as spam/not-spam"), choose an architecture (feedforward, CNN, RNN), justify it.
  2. Given a model (e.g. "ResNet-50"), interpret what it means (CNN, 50 layers, residual architecture).
  3. Identify common mistakes (e.g. using feedforward for images, not using activation, overfitting from a very deep network).

Why this matters for an AI Engineer

Situations where you'll use this knowledge

1. Model selection:

You go to Hugging Face, you see text classification models:

  • bert-base-uncased: Transformer, 12 layers, 110M parameters.
  • distilbert-base-uncased: Transformer, 6 layers, 66M parameters (a smaller version of BERT).

Which do you choose? It depends on trade-offs:

  • BERT: More accurate, slower, more expensive (110M parameters → more compute).
  • DistilBERT: Less accurate (it loses ~2-3% accuracy), faster, cheaper (66M parameters → less compute).

Without understanding architectures: You don't know what "12 layers" or "110M parameters" means → you choose at random or copy what a tutorial says.

With understanding: You know that more layers → more expressiveness but more latency. More parameters → more compute. You can reason: "If latency isn't critical, I use BERT. If I need responses in <100ms, I use DistilBERT."

2. Debugging:

Your image classifier is failing (65% accuracy on the test set). Why?

Possible causes:

  • Inadequate architecture: You used feedforward (a simple network) instead of a CNN (specialized in images) → the network doesn't capture spatial patterns (edges, textures).
  • Underfitting: A very simple network (e.g. 1 hidden layer with 10 neurons) → it doesn't have the capacity to learn complex patterns.
  • Overfitting: A very complex network (e.g. 100 layers) without regularization → it learns the training set's noise, it doesn't generalize to the test set.
  • Insufficient data: 100 training images → the network doesn't have enough examples to learn.

Without understanding architectures: You don't know how to diagnose the problem → you try random changes (changing the learning rate, adding layers) without understanding why.

With understanding: You identify the problem (e.g. underfitting), propose a solution (use a CNN, add layers), explain why to your team.

3. Optimization:

Your image classification app is slow (2 seconds per image). Product asks you to reduce it to <500ms.

Analysis:

  • Model: ResNet-50 (50 layers, 25M parameters).
  • Bottleneck: The forward pass (passing the image through 50 layers takes time).

Solutions:

  1. Use a smaller model: ResNet-18 (18 layers, 11M parameters) → faster, less accurate.
  2. Reduce the resolution: From 512x512 to 256x256 → fewer pixels processed by each layer.
  3. Model quantization: Reduce parameter precision (float32 → int8) → faster, lower precision.

Trade-offs: Fewer layers/parameters → less accuracy. You choose based on product priorities (speed vs accuracy).

Without understanding architectures: You don't know why the model is slow or how to optimize it.

With understanding: You identify the bottleneck (50 layers), propose solutions with clear trade-offs.

4. Fine-tuning:

You need to adjust a pre-trained model (e.g. BERT) to your domain (e.g. classifying reviews of tech products in Spanish).

Technical decisions:

  • Which layers to adjust? The last layers (they learn characteristics specific to your problem). The early layers (which learn general characteristics of language) are left frozen.
  • How much data do you need? Fine-tuning requires less data than training from scratch (because the model already learned general characteristics). E.g. 1,000-10,000 examples vs millions for training from scratch.
  • What learning rate to use? Lower than in training from scratch (because you're already near a good point; you're only adjusting).

Without understanding architectures: You don't know which layers to adjust, how much data you need, or why the learning rate should be small.

With understanding: You know that the last layers learn specific characteristics → you adjust them. You know that fine-tuning is more efficient → you propose a solution with less data.

Common mistakes when learning Neural Networks

1. Thinking you need the math from the start

Mistake: Starting by reading academic papers about backpropagation with derivatives, linear algebra, gradient computation.

Reality: For AI Engineering, you first need visual concepts (what each component does, how it connects, when to use each architecture). The math is useful if you want to do research, but NOT for integrating pre-trained models.

Advice: Start with concepts (this module). If you later want to go deeper, study the math (resources at the end of the module).

2. Confusing "understanding" with "implementing from scratch"

Mistake: Thinking you don't understand Neural Networks if you can't implement backpropagation from scratch in NumPy.

Reality: Implementing from scratch is useful as an academic exercise, but in practice you use frameworks (PyTorch, TensorFlow) that do it for you. What matters is understanding what each component does (what a layer is, what weights are, how they're adjusted).

Analogy: You don't need to know how to build a car engine in order to drive effectively.

3. Assuming more layers = always better

Mistake: Thinking that deeper networks (more layers) are always better.

Reality: More layers → more expressiveness (they can learn more complex patterns) BUT also → more compute (slower), more risk of overfitting (if you don't have enough data or regularization), harder to train (vanishing/exploding gradient problems).

Rule of thumb: Start with simple architectures; add complexity only if necessary.

4. Not understanding the training vs inference distinction (again)

Mistake: Confusing training (adjusting the network's weights using backpropagation) with inference (using the already-trained network to make predictions).

Reality (review from Module 2):

  • Training: Expensive (days, costly GPUs), requires labeled data, adjusts weights. OTHERS DO IT (OpenAI, Google, Meta).
  • Inference: Cheap (milliseconds, a CPU is enough for small models), uses the already-trained network, generates predictions. YOU DO IT (as an AI Engineer).

Consequence: As an AI Engineer, you do NOT train from scratch (you use pre-trained models). But you do run inference (integrating models into apps) and occasionally fine-tuning (a light adjustment with your data).

Frequently asked questions

Do I need to know how to program for this module?

No. This module is conceptual (no code). You only need to understand basic programming concepts (what a variable, a function, an array is) in order to follow the analogies.

In future bootcamp modules (not in this guide) you will see code (Python, PyTorch, Hugging Face), but this guide is conceptual preparation.

Will I see code in this module?

No. This module uses diagrams, analogies and visualizations. The goal is for you to understand the concepts without getting distracted by code syntax.

Optional resources with code: If you want to experiment, at the end of each lesson there are resources (e.g. TensorFlow Playground: a visual interface with no code; Fast.ai: a practical course with code).

How long does it take to learn this?

This module: 3 hours (2.5h reading + 0.5h exercises).

To master Deep Learning (implementation, advanced architectures): 3-6 months (courses like Fast.ai, Stanford CS231n, hands-on experience).

For AI Engineering (integrating pre-trained models): This module + Modules 4-6 of this guide (Transformers, LLMs, APIs) = enough fundamentals. Then practice on real projects (the bootcamp).

Can I skip this module and go straight to Transformers (Module 4)?

Not recommended. Transformers (the architecture of GPT-4, BERT, Claude) are a type of neural network. If you don't understand what a neuron, a layer, a forward pass, backpropagation are, you'll struggle to understand attention (Transformers' key mechanism).

Logical progression:

Module 2: Machine Learning (training, supervised, etc.)
        ↓
Module 3: Neural Networks (neuron, layers, forward/backward)  ← YOU ARE HERE
        ↓
Module 4: Transformers (attention, encoder-decoder)
        ↓
Module 5: LLMs (GPT, Claude, tokenization)

Each module builds on the previous one.

Evidence of success

You know you've made good use of this module when:

  • You can explain to a non-technical person what a neural network is and how it learns (using analogies, without formulas).
  • You can diagram a simple neural network (e.g. 3 layers: input, hidden, output) with weights and activations.
  • You can identify which architecture to use for a given problem (CNN → images, RNN → sequences, feedforward → tables).
  • You can interpret model documentation (e.g. "ResNet-50 with 50 convolutional layers, 25M parameters").
  • You can diagnose basic problems (e.g. "the model is slow because it has 100 layers", "it uses feedforward instead of a CNN for images").

If by the end of the module (Lesson 08) you can solve the integrative exercise without consulting earlier lessons, you're ready for Module 4 (Transformers).

Summary of this lesson

Why you need to understand Neural Networks:

  • All modern AI (LLMs, vision, recommendations) uses neural networks.
  • As an AI Engineer, you integrate pre-trained models → you need to understand architectures in order to select, optimize, diagnose.
  • This module is conceptual (without complex math), focused on architectural reasoning.

What you'll learn:

  • The artificial neuron, layers, architecture (Lessons 02-03)
  • Forward pass, conceptual backpropagation (Lessons 04-06)
  • CNNs, RNNs, feedforward: when to use each one (Lesson 07)
  • Practice: designing architectures (Lesson 08)

Next step: Move on to Lesson 02: What is a Neural Network? where you'll see what an artificial neuron is and how it decides.

Additional resources (optional)

  1. 3Blue1Brown: Neural Networks — A series of 4 visual videos about NNs. In English with subtitles. Excellent for visualization.

  2. TensorFlow Playground — A visual interface for experimenting with NNs (no code). You can see how a network learns in real time. In English, but very intuitive.

  3. Fast.ai: Practical Deep Learning for Coders — A practical Deep Learning course. In English. More technical (with Python/PyTorch code).

  4. Stanford CS231n: Convolutional Neural Networks — An academic course on CNNs. In English. Very technical (with math).

  5. Deep Learning Book (Goodfellow et al.) — The reference book on Deep Learning. In English. Very technical (linear algebra, calculus).

Note: You do NOT need these resources for AI Engineering; this module covers the essentials. They're optional if you want to go deeper.

← Previous7. Exercise: Classifying Machine Learning ProblemsNext →2. What is a Neural Network?