Deep Learning,
Week 1

Every week of this course builds on one question: how can a program learn a function instead of being told it? This week answers it for the simplest case, a stack of small units called neurons. Week 2 turns the ideas into PyTorch; the weeks after change the shape of the network (convolutions, recurrence, attention, diffusion) but never the core loop you meet here: predict, measure the mistake, push the mistake back through the network, adjust the weights.

1.1Machine learning: learn the mapping, not the rules

In ordinary programming you write the rules and the computer applies them. In machine learning (ML) you give the computer examples (inputs with the answers you want) and it finds the rules itself, the way a child works out what a dog looks like from many dogs. What it finds is stored as numbers called parameters (weights).

Classical ML (linear models, trees, random forests, boosting) has one hidden cost: feature engineering. To predict house prices you decide that "area" and "location" matter and feed them in. A forest of decision trees then votes on the answer. The quality of the model is capped by how good your hand-made features are.

For an image, an audio clip or a sentence nobody can write down the right features by hand: what number captures "this is a cat"? Deep learning (DL) removes that step. It takes the raw data (pixels, waveform, characters) and learns the features and the final mapping together, in one model with many layers. Three things made that practical: big data, GPUs and better algorithms (better activations, initialisation, optimisers).

1.2The neuron, in three generations

1943: McCulloch-Pitts

The first mathematical neuron takes binary inputs, adds them and outputs 1 if the sum reaches a fixed threshold, else 0. Choosing the threshold gives logic gates: threshold 2 on two inputs is AND, threshold 1 is OR.

def mp_neuron(inputs, threshold):
    """McCulloch-Pitts: binary inputs, fixed threshold, no weights, no learning."""
    return 1 if sum(inputs) >= threshold else 0

print("AND:", [mp_neuron(x, 2) for x in [(0, 0), (0, 1), (1, 0), (1, 1)]])
print("OR :", [mp_neuron(x, 1) for x in [(0, 0), (0, 1), (1, 0), (1, 1)]])

AND: [0, 0, 0, 1]
OR : [0, 1, 1, 1]

• No weights: every input counts equally, so it cannot say "this feature matters more".
• Fixed threshold: a human sets it; nothing is learned.
• Binary only: no continuous values.

1958: the perceptron

Rosenblatt's perceptron adds two ideas. Weighted inputs: each feature gets an importance. A learning rule: when the prediction is wrong, move the weights toward the right answer, $w \leftarrow w + \eta\,(y - \hat y)\,x$. The model now learns from data. Watch it learn AND and OR, and fail on XOR:

import numpy as np

def train_perceptron(X, y, epochs=20, lr=1.0):
    w = np.zeros(X.shape[1]); b = 0.0
    for epoch in range(epochs):
        mistakes = 0
        for xi, yi in zip(X, y):
            pred = 1 if xi @ w + b >= 0 else 0
            if pred != yi:                       # the perceptron learning rule
                w += lr * (yi - pred) * xi
                b += lr * (yi - pred)
                mistakes += 1
        if mistakes == 0:
            return w, b, epoch + 1
    return w, b, None

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
for name, y in [("AND", [0, 0, 0, 1]), ("OR", [0, 1, 1, 1]), ("XOR", [0, 1, 1, 0])]:
    w, b, done = train_perceptron(X, np.array(y))
    print(f"{name}: weights={w}, bias={b}, " + (f"converged after {done} epochs" if done else "did NOT converge in 20 epochs"))

AND: weights=[2. 1.], bias=-3.0, converged after 6 epochs
OR: weights=[1. 1.], bias=-1.0, converged after 4 epochs
XOR: weights=[-1.  0.], bias=0.0, did NOT converge in 20 epochs

AND and OR converge in a few epochs. XOR never does: the weights wander without reaching zero mistakes, because no single straight line separates its classes (proved in Chapter 6). This was the "fatal flaw" that stalled neural-network research for years.

The modern neuron

Two steps. Step 1, a linear combination with a bias: $z = \mathbf{w}\cdot\mathbf{x} + b$. Step 2, a non-linear activation: $a = f(z)$. The activation is chosen smooth because training needs its derivative (Chapter 5); the old hard step has a zero or undefined slope almost everywhere, so no gradient could flow through it.

import numpy as np

x = np.array([0.5, -1.0, 2.0])      # three input features
w = np.array([0.4, 0.3, -0.2])      # one weight per input
b = 0.1                              # bias

z = w @ x + b                        # step 1: linear combination
a = 1 / (1 + np.exp(-z))             # step 2: non-linear activation (sigmoid)
print("z =", round(z, 4))
print("a =", round(a, 4))

z = -0.4
a = 0.4013

The bias $b$ shifts where the neuron "switches on", the way the intercept shifts a line. Without it every neuron would have to pass through the origin.

1.3Lecture summary

• ML learns a mapping from examples; DL also learns the features, from raw data.
• McCulloch-Pitts: binary, fixed threshold, no weights. Perceptron: weights and a learning rule, but only linearly separable data. Modern neuron: $a = f(w\cdot x + b)$ with a differentiable $f$.
• The brain is an analogy, not a blueprint.

A neuron is one small piece. This chapter shows the three ideas that let us use pieces like it to solve real problems, and how pieces are stacked into a feedforward network.

2.1Three fundamental tricks

Trick 1: numerization

Networks do arithmetic, so everything must become numbers. A self-driving car turns camera pixels and LiDAR distances into vectors. For a category like cat / dog / bird there are two standard encodings:

• One-hot: cat = [1,0,0], dog = [0,1,0], bird = [0,0,1]. Simple, but every pair is equally unrelated and the vector is as long as the vocabulary.
• Embedding: a short dense vector per item that is learned, so similar things land close together.

import numpy as np

vocab = ["cat", "dog", "bird"]
one_hot = np.eye(3)
print("one-hot cat vs dog dot product:", one_hot[0] @ one_hot[1])   # 0: no notion of similarity
print("one-hot cat vs cat dot product:", one_hot[0] @ one_hot[0])

# an embedding is a lookup of a dense vector (here hand-made; real ones are learned)
E = np.array([[0.9, 0.1],      # cat
              [0.8, 0.2],      # dog   (close to cat)
              [0.1, 0.9]])     # bird
cos = lambda a, b: a @ b / (np.linalg.norm(a) * np.linalg.norm(b))
print("embedding cat~dog :", round(cos(E[0], E[1]), 3))
print("embedding cat~bird:", round(cos(E[0], E[2]), 3))

one-hot cat vs dog dot product: 0.0
one-hot cat vs cat dot product: 1.0
embedding cat~dog : 0.991
embedding cat~bird: 0.22

The dot product of two different one-hot vectors is always 0: the encoding knows nothing about meaning. The (hand-made, for illustration) embeddings put cat and dog close (cosine 0.991) and bird far (0.22). Real embeddings are learned during training; Week 9 returns to them.

Trick 2: learn the function

Data is a set of (input, output) pairs. The model is a function that maps inputs to outputs. Instead of writing that function, we choose a family of functions and let data pick the member.

Trick 3: parameterization

Every function in the family has the same form (the architecture) and differs only in its parameters: the weights and biases. So "finding the function" becomes "finding the numbers". A loss function says how wrong the current numbers are (high loss = bad model), which turns learning into minimisation.

2.2The feedforward network (MLP)

Neurons are arranged in layers: an input layer (just the features), one or more hidden layers that transform them, and an output layer. Information moves in one direction, with no loops, hence feedforward. Layer $l$ computes

If layer $l-1$ has $n_{l-1}$ units and layer $l$ has $n_l$, then $W^{[l]}$ is $n_l \times n_{l-1}$ and $b^{[l]}$ has $n_l$ entries. So a layer costs $n_l n_{l-1} + n_l$ parameters. Count them for a 4-5-3-2 network, by PyTorch and by hand:

import torch.nn as nn

net = nn.Sequential(
    nn.Linear(4, 5), nn.ReLU(),      # input 4 -> hidden 5
    nn.Linear(5, 3), nn.ReLU(),      # hidden 5 -> hidden 3
    nn.Linear(3, 2),                 # hidden 3 -> output 2
)
total = 0
for name, p in net.named_parameters():
    print(f"{name:10s} shape={tuple(p.shape)!s:8s} count={p.numel()}")
    total += p.numel()
print("total parameters:", total)
print("by hand: (4*5+5) + (5*3+3) + (3*2+2) =", (4*5+5) + (5*3+3) + (3*2+2))

0.weight   shape=(5, 4)   count=20
0.bias     shape=(5,)     count=5
2.weight   shape=(3, 5)   count=15
2.bias     shape=(3,)     count=3
4.weight   shape=(2, 3)   count=6
4.bias     shape=(2,)     count=2
total parameters: 51
by hand: (4*5+5) + (5*3+3) + (3*2+2) = 51

2.3Why a hidden layer needs a non-linear activation

Stack two linear layers with nothing between them: $W_2(W_1 x + b_1) + b_2 = (W_2W_1)x + (W_2 b_1 + b_2)$, which is just one linear layer. A hundred linear layers are no more powerful than one. The activation between layers is what stops the stack collapsing, and it is what lets a network bend its decision boundary to fit complicated data.

import numpy as np
rng = np.random.default_rng(0)
W1, b1 = rng.normal(size=(4, 3)), rng.normal(size=4)
W2, b2 = rng.normal(size=(2, 4)), rng.normal(size=2)
x = rng.normal(size=3)

two_layers = W2 @ (W1 @ x + b1) + b2            # two linear layers, no activation
W, b = W2 @ W1, W2 @ b1 + b2                     # ... collapse into ONE linear layer
print("same output:", np.allclose(two_layers, W @ x + b))
print("collapsed W shape:", W.shape)

relu = lambda z: np.maximum(0, z)
with_relu = W2 @ relu(W1 @ x + b1) + b2         # with a ReLU in between
print("with ReLU still equals the single layer?", np.allclose(with_relu, W @ x + b))

same output: True
collapsed W shape: (2, 3)
with ReLU still equals the single layer? False

2.4Lecture summary

• Numerize everything; embeddings beat one-hot because they capture similarity.
• A model = fixed form + learnable parameters; loss measures wrongness.
• MLP layer: $a^{[l]} = f(W^{[l]}a^{[l-1]} + b^{[l]})$; parameters per layer $n_l n_{l-1} + n_l$.
• Without activations, any depth collapses into one linear layer.

The activation function is the single non-linear ingredient of a neuron, and it decides how well gradients survive a long trip backwards. Which one you pick, and where, is one of the most-asked topics in the course.

3.1The classic four

The numbers behind the plot, computed:

import numpy as np
z = np.array([-4.0, -1.0, 0.0, 1.0, 4.0])
sigmoid = 1 / (1 + np.exp(-z))
tanh = np.tanh(z)
relu = np.maximum(0, z)
leaky = np.where(z > 0, z, 0.01 * z)
print("z      ", z)
print("sigmoid", sigmoid.round(4))
print("tanh   ", tanh.round(4))
print("relu   ", relu)
print("leaky  ", leaky)
print()
print("sigmoid'", (sigmoid * (1 - sigmoid)).round(4))
print("tanh'   ", (1 - tanh ** 2).round(4))
print("relu'   ", (z > 0).astype(float))

z       [-4. -1.  0.  1.  4.]
sigmoid [0.018  0.2689 0.5    0.7311 0.982 ]
tanh    [-0.9993 -0.7616  0.      0.7616  0.9993]
relu    [0. 0. 0. 1. 4.]
leaky   [-0.04 -0.01  0.    1.    4.  ]

sigmoid' [0.0177 0.1966 0.25   0.1966 0.0177]
tanh'    [0.0013 0.42   1.     0.42   0.0013]
relu'    [0. 0. 0. 1. 1.]

Read the last block: the slope of sigmoid is at most 0.25 (at $z=0$) and falls to 0.018 by $z=\pm4$; tanh reaches 1 but is 0.0013 at $\pm4$; ReLU is exactly 0 or 1. Useful identities (write the derivative using the output $a$, which you already stored in the forward pass):

(The notes warn about confusing $\sigma(z)(1-\sigma(z))$ with $\sigma(z)\sigma(-z)$. They are the same number, because $\sigma(-z) = 1-\sigma(z)$.)

3.2The vanishing gradient problem, measured

Backpropagation multiplies one derivative per layer. If each factor is below 1, the product shrinks exponentially with depth, and the early layers get almost no learning signal. The sigmoid factor is at most 0.25, so even in the best case:

import numpy as np
s = lambda z: 1 / (1 + np.exp(-z))
zs = np.linspace(-8, 8, 160001)
d = s(zs) * (1 - s(zs))
print("max of sigmoid'(z) =", round(d.max(), 6), "at z =", round(zs[d.argmax()], 4))
print("sigmoid'(0) =", s(0) * (1 - s(0)))
print()
for L in (1, 5, 10, 20, 30):
    print(f"best case after {L:2d} sigmoid layers: 0.25^{L} = {0.25 ** L:.3e}")

max of sigmoid'(z) = 0.25 at z = 0.0
sigmoid'(0) = 0.25

best case after  1 sigmoid layers: 0.25^1 = 2.500e-01
best case after  5 sigmoid layers: 0.25^5 = 9.766e-04
best case after 10 sigmoid layers: 0.25^10 = 9.537e-07
best case after 20 sigmoid layers: 0.25^20 = 9.095e-13
best case after 30 sigmoid layers: 0.25^30 = 8.674e-19

Note that $0.25^L$ is a bound on the activation factors only; the weight matrices multiply in too, so real networks can be better or worse. To see the real thing, build a 20-layer network and compare the gradient norm at the first layer with the last, for different activations:

import torch, torch.nn as nn
torch.manual_seed(0)

def grad_norms(activation, init=None, depth=20, width=32):
    layers = []
    for _ in range(depth):
        layers += [nn.Linear(width, width), activation()]
    net = nn.Sequential(*layers, nn.Linear(width, 1))
    if init == "he":                                    # He / Kaiming initialisation, made for ReLU
        for m in net:
            if isinstance(m, nn.Linear):
                nn.init.kaiming_normal_(m.weight, nonlinearity="relu"); nn.init.zeros_(m.bias)
    net(torch.randn(64, width)).pow(2).mean().backward()
    return net[0].weight.grad.norm().item(), net[-1].weight.grad.norm().item()

for name, act, init in [("Sigmoid", nn.Sigmoid, None), ("Tanh", nn.Tanh, None),
                        ("ReLU (default init)", nn.ReLU, None), ("ReLU (He init)", nn.ReLU, "he")]:
    first, last = grad_norms(act, init)
    print(f"{name:20s} first layer = {first:.2e}   last layer = {last:.2e}   first/last = {first / last:.2e}")

Sigmoid              first layer = 2.25e-18   last layer = 1.60e+00   first/last = 1.41e-18
Tanh                 first layer = 2.28e-07   last layer = 8.98e-02   first/last = 2.54e-06
ReLU (default init)  first layer = 3.77e-10   last layer = 4.40e-02   first/last = 8.58e-09
ReLU (He init)       first layer = 1.90e-01   last layer = 7.72e-01   first/last = 2.45e-01

Sigmoid collapses the gradient by 18 orders of magnitude, tanh by 6. The surprise is ReLU with PyTorch's default initialisation: it also shrinks (by 8). That is an initialisation effect, not a ReLU flaw: with He (Kaiming) initialisation, designed for ReLU, the ratio is 0.25 and learning signal reaches layer 1. So the notes' "ReLU has no vanishing gradient" is true for the derivative (1 for $z>0$), but a deep ReLU net also needs the right initialisation.

3.3Dying ReLU

A ReLU neuron whose input is negative for every training example outputs 0 and passes gradient 0, so its weights never update again: it is dead. A large learning rate or a very negative bias can cause it. A leaky ReLU keeps a small slope (0.01) for negative inputs so the neuron can recover; a parametric ReLU makes that slope a learned parameter.

import numpy as np
rng = np.random.default_rng(1)
x = rng.normal(size=(1000, 1))
w, b = -1.0, -5.0                    # a bias pushed so negative that z < 0 for almost every input
z = w * x + b
print("fraction of inputs with z > 0:", (z > 0).mean())
print("ReLU gradient for those inputs:", (z > 0).astype(float).sum(), "non-zero entries out of", len(z))
print("-> the neuron outputs 0 and passes 0 gradient: it cannot recover by itself")
leaky_grad = np.where(z > 0, 1.0, 0.01)
print("leaky ReLU gradient is still:", leaky_grad.mean().round(4), "(small, but alive)")

fraction of inputs with z > 0: 0.0
ReLU gradient for those inputs: 0.0 non-zero entries out of 1000
-> the neuron outputs 0 and passes 0 gradient: it cannot recover by itself
leaky ReLU gradient is still: 0.01 (small, but alive)

3.4Softmax, and which activation goes where

For multi-class output we need numbers that are positive and sum to 1. Softmax $p_i = e^{z_i}/\sum_j e^{z_j}$ does that. Always subtract the maximum logit first: the answer is identical, but $e^{1000}$ no longer overflows.

import numpy as np
def softmax(z):
    z = z - z.max()              # subtract the max: same result, no overflow
    e = np.exp(z)
    return e / e.sum()

logits = np.array([2.0, 1.0, 0.1])
p = softmax(logits)
print("probabilities:", p.round(4), " sum =", p.sum())
print("big logits, naive exp would overflow:", softmax(np.array([1000.0, 1001.0])).round(4))

probabilities: [0.659  0.2424 0.0986]  sum = 1.0
big logits, naive exp would overflow: [0.2689 0.7311]

3.5Lecture summary

• Sigmoid and tanh saturate; sigmoid's slope is at most 0.25, so deep sigmoid nets vanish (measured: 18 orders).
• ReLU is the default hidden activation; it can die, and wants He initialisation.
• Output: sigmoid (binary), softmax (multi-class), linear (regression).

A network starts with random weights and knows nothing. Learning is a loop of four steps repeated thousands of times: forward pass (predict), loss (how wrong?), gradient (which way to nudge each weight?), update (nudge). This chapter covers three of the four; the gradient step is backpropagation, Chapter 5.

4.1Loss functions: one number for "how wrong"

The loss compresses a whole set of predictions into one number that optimisation can push down. Pick it by the kind of target:

import numpy as np
y, y_hat = np.array([3.0, -0.5, 2.0]), np.array([2.5, 0.0, 2.0])
print("MSE:", np.mean((y - y_hat) ** 2))

# binary cross-entropy: label 1, prediction 0.9 (good) vs 0.1 (bad)
bce = lambda y, p: -(y * np.log(p) + (1 - y) * np.log(1 - p))
print("BCE label=1, p=0.9 :", round(bce(1, 0.9), 4))
print("BCE label=1, p=0.1 :", round(bce(1, 0.1), 4))
print("BCE label=1, p=0.01:", round(bce(1, 0.01), 4), "  <- a confident wrong answer is punished hard")
print("MSE label=1, p=0.01:", round((1 - 0.01) ** 2, 4), "  <- MSE can never exceed 1 here")

MSE: 0.16666666666666666
BCE label=1, p=0.9 : 0.1054
BCE label=1, p=0.1 : 2.3026
BCE label=1, p=0.01: 4.6052   <- a confident wrong answer is punished hard
MSE label=1, p=0.01: 0.9801   <- MSE can never exceed 1 here

Why not MSE for classification? Look at the last two lines. A confident wrong prediction (true label 1, predicted 0.01) costs MSE only 0.98, but cross-entropy 4.6, and the gap grows without limit as the prediction approaches 0. Cross-entropy punishes confident mistakes hard and keeps a strong gradient where MSE (through a sigmoid) goes flat. Multi-class cross-entropy only looks at the probability given to the true class:

import numpy as np
logits = np.array([2.0, 1.0, 0.1])
p = np.exp(logits) / np.exp(logits).sum()
for true_class in range(3):
    print(f"true class {true_class}: loss = -log({p[true_class]:.4f}) = {-np.log(p[true_class]):.4f}")

true class 0: loss = -log(0.6590) = 0.4170
true class 1: loss = -log(0.2424) = 1.4170
true class 2: loss = -log(0.0986) = 2.3170

4.2Gradient descent

Imagine standing on a foggy mountain and wanting the valley. You cannot see it, but you can feel the slope under your feet. The gradient $\nabla_\theta L$ points uphill (steepest ascent), so you step the opposite way:

$\eta$ is the learning rate, the step size. The notes' example $L(w)=(w-3)^2$ has slope $2(w-3)$. From $w=1$ with $\eta=0.1$ the first step is $1-0.1\cdot(-4)=1.4$, exactly the notes' answer. Five steps:

# L(w) = (w - 3)^2   ->   dL/dw = 2 (w - 3)
w, lr = 1.0, 0.1
for step in range(1, 6):
    grad = 2 * (w - 3)
    w = w - lr * grad
    print(f"step {step}: grad = {grad:+.4f}, w = {w:.4f}, loss = {(w - 3) ** 2:.4f}")

step 1: grad = -4.0000, w = 1.4000, loss = 2.5600
step 2: grad = -3.2000, w = 1.7200, loss = 1.6384
step 3: grad = -2.5600, w = 1.9760, loss = 1.0486
step 4: grad = -2.0480, w = 2.1808, loss = 0.6711
step 5: grad = -1.6384, w = 2.3446, loss = 0.4295

The learning rate is a Goldilocks choice

def run(lr, steps=8, w=1.0):
    path = [w]
    for _ in range(steps):
        w = w - lr * 2 * (w - 3)
        path.append(round(w, 3))
    return path

for lr in (0.01, 0.1, 0.9, 1.0, 1.1):
    print(f"lr = {lr:<4}:", run(lr))

lr = 0.01: [1.0, 1.04, 1.079, 1.118, 1.155, 1.192, 1.228, 1.264, 1.298]
lr = 0.1 : [1.0, 1.4, 1.72, 1.976, 2.181, 2.345, 2.476, 2.581, 2.664]
lr = 0.9 : [1.0, 4.6, 1.72, 4.024, 2.181, 3.655, 2.476, 3.419, 2.664]
lr = 1.0 : [1.0, 5.0, 1.0, 5.0, 1.0, 5.0, 1.0, 5.0, 1.0]
lr = 1.1 : [1.0, 5.4, 0.12, 6.456, -1.147, 7.977, -2.972, 10.166, -5.6]

Read the rows. $\eta=0.01$ crawls (1.3 after 8 steps; the minimum is 3). $\eta=0.1$ heads there smoothly. At $\eta=0.9$ every step overshoots, zig-zagging around 3. At exactly $\eta=1.0$ it bounces between 1 and 5 forever. At $\eta=1.1$ each overshoot is bigger than the last and the loss explodes. For this parabola the rule is $\eta<1$ to converge (in general $\eta<2/\text{curvature}$).

4.3Batch, stochastic and mini-batch

The gradient of the full loss averages over all $n$ examples. How many do you use per update?

import numpy as np
rng = np.random.default_rng(0)
X = rng.normal(size=(1000, 1))
y = 3.0 * X[:, 0] + 2.0 + rng.normal(scale=0.3, size=1000)     # true line: y = 3x + 2

def train(batch_size, lr=0.05, epochs=5):
    w, b = 0.0, 0.0
    updates = 0
    for _ in range(epochs):
        idx = rng.permutation(len(X))
        for s in range(0, len(X), batch_size):
            j = idx[s:s + batch_size]
            err = (w * X[j, 0] + b) - y[j]
            w -= lr * 2 * (err * X[j, 0]).mean()
            b -= lr * 2 * err.mean()
            updates += 1
    return w, b, updates

for name, bs in [("batch (all 1000)", 1000), ("SGD (1 sample)", 1), ("mini-batch (32)", 32)]:
    w, b, n = train(bs)
    print(f"{name:18s} updates={n:5d}  w={w:.3f}  b={b:.3f}")

batch (all 1000)   updates=    5  w=1.160  b=0.770
SGD (1 sample)     updates= 5000  w=3.023  b=2.129
mini-batch (32)    updates=  160  w=3.040  b=2.021

Same learning rate, same 5 epochs. Batch GD made only 5 updates and has barely started ($w=1.16$, true value 3). SGD made 5000 noisy updates and lands near the truth; mini-batch gets there in 160 updates with less jitter. The lesson: more updates beat more accurate updates, up to the noise level. "SGD" in PyTorch almost always means mini-batch SGD.

4.4Lecture summary

• Loop: forward, loss, gradient, update. Loss by target: MSE (regression), cross-entropy (classification).
• $\theta \leftarrow \theta - \eta\nabla L$; too small crawls, too large diverges.
• Mini-batch is the practical standard.

A modern network has millions of weights, and gradient descent needs the slope of the loss with respect to every one. Doing that naively would mean running the network once per weight. Backpropagation gets all the gradients in about the cost of one forward pass, by applying the chain rule from the output back to the input and reusing every intermediate result.

5.1The chain rule on a computational graph

Write the computation as a graph of small operations. The forward pass fills in values and stores them. The backward pass starts with $\partial L/\partial L = 1$ and, at each node, multiplies the gradient arriving from above (the upstream gradient) by the node's own local gradient:

Walk it: forward gives $s=a+b=5$ and $L=s\cdot c=20$. Backward: $\partial L/\partial c = s = 5$; $\partial L/\partial s = c = 4$; an addition node passes its upstream gradient to both inputs unchanged, so $\partial L/\partial a=\partial L/\partial b=4$. Then confirm with PyTorch's autograd:

# L = (a + b) * c
a, b, c = 2.0, 3.0, 4.0
s = a + b                    # forward: s = 5
L = s * c                    # forward: L = 20
print("forward:", s, L)

dL_dL = 1.0                  # backward starts here
dL_ds = c * dL_dL            # dL/ds = c = 4
dL_dc = s * dL_dL            # dL/dc = s = 5
dL_da = 1.0 * dL_ds          # s = a + b  -> local gradient 1
dL_db = 1.0 * dL_ds
print("dL/da =", dL_da, " dL/db =", dL_db, " dL/dc =", dL_dc)

import torch
a, b, c = [torch.tensor(v, requires_grad=True) for v in (2.0, 3.0, 4.0)]
((a + b) * c).backward()
print("autograd agrees:", a.grad.item(), b.grad.item(), c.grad.item())

forward: 5.0 20.0
dL/da = 4.0  dL/db = 4.0  dL/dc = 5.0
autograd agrees: 4.0 4.0 5.0

5.2Backpropagation for a layered network

Define the error signal of layer $l$ as $\delta^{[l]}=\partial L/\partial z^{[l]}$. Then:

In words: take the error of the layer above, send it back through the weights (the transpose $W^\top$ routes each unit's share of blame to the units that fed it), and scale by how sensitive the activation is ($f'$). The gradient of a weight is then (error at its output) times (activation at its input). Notice $f'$ appears in every layer: this is exactly where sigmoid's 0.25 bites.

5.3Doing it by hand, and checking it

A tiny network, 2 inputs, 3 sigmoid hidden units, 1 sigmoid output, binary cross-entropy. Write forward and backward in NumPy, then check against a numerical gradient (nudge each weight by $\pm10^{-6}$ and measure the change in loss). This "gradient check" is how real bugs in hand-written backward passes are found:

import numpy as np

# one hidden layer: x(2) -> h(3, sigmoid) -> yhat(1, sigmoid), binary cross-entropy
rng = np.random.default_rng(0)
x = np.array([0.5, -1.0]); y = 1.0
W1 = rng.normal(scale=0.5, size=(3, 2)); b1 = np.zeros(3)
W2 = rng.normal(scale=0.5, size=(1, 3)); b2 = np.zeros(1)
sig = lambda z: 1 / (1 + np.exp(-z))

# ---- forward (keep everything: backward needs it)
z1 = W1 @ x + b1;  a1 = sig(z1)
z2 = W2 @ a1 + b2; a2 = sig(z2)
loss = -(y * np.log(a2) + (1 - y) * np.log(1 - a2))

# ---- backward
d2 = a2 - y                         # sigmoid + BCE: the sigmoid derivative cancels
dW2 = np.outer(d2, a1);  db2 = d2
d1 = (W2.T @ d2) * a1 * (1 - a1)    # push the error back through W2, times sigmoid'
dW1 = np.outer(d1, x);   db1 = d1
print("loss:", loss.round(5))
print("dW2:", dW2.round(5))
print("dW1:\n", dW1.round(5))

# ---- check against a numerical gradient (finite differences)
def loss_fn(W1_, W2_):
    a1_ = sig(W1_ @ x + b1); a2_ = sig(W2_ @ a1_ + b2)
    return (-(y * np.log(a2_) + (1 - y) * np.log(1 - a2_))).item()
eps = 1e-6
num = np.zeros_like(W1)
for i in range(3):
    for j in range(2):
        Wp = W1.copy(); Wp[i, j] += eps
        Wm = W1.copy(); Wm[i, j] -= eps
        num[i, j] = (loss_fn(Wp, W2) - loss_fn(Wm, W2)) / (2 * eps)
print("max |backprop - numerical| for W1:", np.abs(num - dW1).max())

loss: [0.49602]
dW2: [[-0.20505 -0.20604 -0.16501]]
dW1:
 [[-0.03179  0.06359]
 [-0.02308  0.04616]
 [ 0.01678 -0.03356]]
max |backprop - numerical| for W1: 8.295730768992371e-11

The difference is about $10^{-10}$: the analytic and numerical gradients agree. Look at the line d2 = a2 - y. The output error is simply prediction minus target: with a sigmoid output and cross-entropy loss, the sigmoid derivative $a(1-a)$ in $\delta^{[L]}$ cancels against the $1/(a(1-a))$ that comes from differentiating the loss. That cancellation is the real reason cross-entropy pairs with sigmoid and softmax: the gradient does not shrink when the neuron saturates.

import torch
torch.manual_seed(0)
x = torch.tensor([0.5, -1.0]); y = torch.tensor([1.0])
W1 = torch.randn(3, 2, requires_grad=True) * 0.5
W1.retain_grad()
W2 = (torch.randn(1, 3) * 0.5).requires_grad_()
a1 = torch.sigmoid(W1 @ x)
a2 = torch.sigmoid(W2 @ a1)
loss = torch.nn.functional.binary_cross_entropy(a2, y)
loss.backward()
print("loss:", round(loss.item(), 5))
print("dL/dW2:", W2.grad.round(decimals=5))

loss: 0.675
dL/dW2: tensor([[-0.3092, -0.1491, -0.2972]])

PyTorch does the same chain rule automatically (this is autograd, Week 2). You write the forward pass; loss.backward() fills every .grad. The two runs above use different random weights, so their numbers differ; the point is that you now know what backward() computes.

5.4Lecture summary

• Backprop = chain rule + cached forward values; cost about one forward pass.
• $\delta^{[l]}=(W^{[l+1]\top}\delta^{[l+1]})\odot f'(z^{[l]})$; $\partial L/\partial W^{[l]}=\delta^{[l]}a^{[l-1]\top}$.
• Sigmoid/softmax + cross-entropy gives output error $\hat y-y$.
• Always gradient-check hand-written backward passes.

XOR ("exclusive or") is the smallest problem that separates a network that can learn from one that cannot. It is also the cleanest demonstration of why hidden layers matter, so it appears in almost every exam on this week.

6.1Why one line cannot do it

A single neuron outputs 1 when $w_1x_1+w_2x_2+b\ge 0$, a straight line in the plane. Demand all four rows:

Add the middle two: $w_1+w_2+2b\ge0$, so $w_1+w_2+b\ge -b$. Since $b<0$, $-b>0$, hence $w_1+w_2+b>0$, which contradicts the last row. The four conditions are inconsistent. A brute-force search over a grid of weights agrees: the best any line does is 3 rows of 4.

# can ONE line w1*x1 + w2*x2 + b >= 0 produce XOR?   Try to satisfy all four rows:
#   (0,0) -> 0 :   b            <  0
#   (0,1) -> 1 :   w2 + b       >= 0
#   (1,0) -> 1 :   w1 + b       >= 0
#   (1,1) -> 0 :   w1 + w2 + b  <  0
# add rows 2 and 3:  w1 + w2 + 2b >= 0   ->  w1 + w2 + b >= -b > 0   (since b < 0)
# but row 4 says w1 + w2 + b < 0.   Contradiction.
import itertools, numpy as np
best = 0
for w1, w2, b in itertools.product(np.linspace(-5, 5, 41), repeat=3):
    ok = sum(int((w1 * x1 + w2 * x2 + b >= 0) == bool(t)) for (x1, x2), t in zip([(0, 0), (0, 1), (1, 0), (1, 1)], [0, 1, 1, 0]))
    best = max(best, ok)
print("best number of XOR rows any line gets right (out of 4):", best)

best number of XOR rows any line gets right (out of 4): 3

6.2The fix: a hidden layer

One neuron cannot, but two neurons plus a combiner can. XOR means "at least one is 1" and "not both are 1": that is OR AND NAND. Hidden unit 1 computes OR, hidden unit 2 computes NAND, the output neuron ANDs them. Weights that really work (step activation):

import numpy as np
step = lambda z: (z >= 0).astype(int)

# hidden unit 1 = OR(x1, x2), hidden unit 2 = NAND(x1, x2); output = AND(h1, h2)
W1 = np.array([[1, 1], [-1, -1]]); b1 = np.array([-0.5, 1.5])
W2 = np.array([[1, 1]]);           b2 = np.array([-1.5])

for x in [(0, 0), (0, 1), (1, 0), (1, 1)]:
    h = step(W1 @ np.array(x) + b1)
    y = step(W2 @ h + b2)
    print(f"x={x}  hidden (OR, NAND) = {tuple(int(v) for v in h)}  ->  output {int(y[0])}")

x=(0, 0)  hidden (OR, NAND) = (0, 1)  ->  output 0
x=(0, 1)  hidden (OR, NAND) = (1, 1)  ->  output 1
x=(1, 0)  hidden (OR, NAND) = (1, 1)  ->  output 1
x=(1, 1)  hidden (OR, NAND) = (1, 0)  ->  output 0

Each row confirms it. The hidden layer has re-described the input: in the (OR, NAND) space the four points are (0,1), (1,1), (1,1), (1,0), and the two XOR=1 points coincide at (1,1), which one line (the AND) now separates from the rest. That is what "a hidden layer is a feature transformation" means.

6.3Learning it instead of designing it

Nobody hand-designs weights in practice. Train the same 2-2-1 shape with sigmoid units on random initial weights, using the backprop you derived (NumPy, from scratch), for several seeds:

import numpy as np
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=float)
Y = np.array([[0], [1], [1], [0]], dtype=float)
sig = lambda z: 1 / (1 + np.exp(-z))

def train(seed, hidden=2, lr=1.0, epochs=5000):
    rng = np.random.default_rng(seed)
    W1 = rng.normal(size=(2, hidden)); b1 = np.zeros(hidden)
    W2 = rng.normal(size=(hidden, 1)); b2 = np.zeros(1)
    for _ in range(epochs):
        a1 = sig(X @ W1 + b1); a2 = sig(a1 @ W2 + b2)
        d2 = (a2 - Y) / len(X)                       # sigmoid + cross-entropy
        d1 = (d2 @ W2.T) * a1 * (1 - a1)
        W2 -= lr * a1.T @ d2;  b2 -= lr * d2.sum(0)
        W1 -= lr * X.T @ d1;   b1 -= lr * d1.sum(0)
    a2 = sig(sig(X @ W1 + b1) @ W2 + b2)
    return a2.ravel(), float(-np.mean(Y * np.log(a2) + (1 - Y) * np.log(1 - a2)))

for seed in range(4):
    p, loss = train(seed)
    print(f"seed {seed}: predictions {p.round(3)}  loss {loss:.4f}  rounded -> {(p > 0.5).astype(int)}")

seed 0: predictions [0.002 0.998 0.997 0.002]  loss 0.0023  rounded -> [0 1 1 0]
seed 1: predictions [0.002 0.998 0.998 0.002]  loss 0.0020  rounded -> [0 1 1 0]
seed 2: predictions [0.002 0.998 0.998 0.002]  loss 0.0020  rounded -> [0 1 1 0]
seed 3: predictions [0.002 0.998 0.998 0.002]  loss 0.0021  rounded -> [0 1 1 0]

All four seeds find a solution (outputs $\approx 0.002$ and $0.998$). Training can also fail. Same task in PyTorch (4 tanh hidden units, Adam), six seeds:

import torch, torch.nn as nn
X = torch.tensor([[0., 0.], [0., 1.], [1., 0.], [1., 1.]])
Y = torch.tensor([[0.], [1.], [1.], [0.]])

def fit(seed, epochs=1000):
    torch.manual_seed(seed)
    net = nn.Sequential(nn.Linear(2, 4), nn.Tanh(), nn.Linear(4, 1), nn.Sigmoid())
    opt = torch.optim.Adam(net.parameters(), lr=0.05)
    loss_fn = nn.BCELoss()
    for _ in range(epochs):
        opt.zero_grad()
        loss = loss_fn(net(X), Y)
        loss.backward()
        opt.step()
    return loss.item(), net(X).detach().squeeze().tolist()

for seed in range(6):
    loss, p = fit(seed)
    verdict = "solved" if loss < 0.05 else "STUCK on a plateau"
    print(f"seed {seed}: loss {loss:.4f}  predictions {[round(v, 2) for v in p]}  -> {verdict}")

seed 0: loss 0.0002  predictions [0.0, 1.0, 1.0, 0.0]  -> solved
seed 1: loss 0.3467  predictions [0.0, 0.5, 1.0, 0.5]  -> STUCK on a plateau
seed 2: loss 0.0004  predictions [0.0, 1.0, 1.0, 0.0]  -> solved
seed 3: loss 0.0004  predictions [0.0, 1.0, 1.0, 0.0]  -> solved
seed 4: loss 0.0001  predictions [0.0, 1.0, 1.0, 0.0]  -> solved
seed 5: loss 0.0003  predictions [0.0, 1.0, 1.0, 0.0]  -> solved

Seed 1 gets stuck on a plateau: two of four rows are right, the other two sit at 0.5, and the loss stays at $0.3467$. The loss surface is non-convex, so where you start matters. Real training copes with this using good initialisation, better optimisers (Week 5) and wider networks.

6.4A forward pass worked by hand (and a note)

The notes work one forward pass of a 2-2-1 sigmoid network on $x=[0,1]$ with $W_1=\begin{bmatrix}1&1\\1&1\end{bmatrix}, b_1=[0,-1], W_2=[1,-2], b_2=0$. Re-running it:

import numpy as np
sig = lambda z: 1 / (1 + np.exp(-z))
x = np.array([0, 1])
W1 = np.array([[1, 1], [1, 1]]); b1 = np.array([0, -1])
W2 = np.array([1, -2]);          b2 = 0
z1 = W1 @ x + b1; a1 = sig(z1)
z2 = W2 @ a1 + b2; a2 = sig(z2)
print("z1 =", z1, " a1 =", a1.round(3))
print("z2 =", round(z2, 3), " a2 =", round(a2, 3), " (target is 1: these weights are not trained yet)")

z1 = [1 0]  a1 = [0.731 0.5  ]
z2 = -0.269  a2 = 0.433  (target is 1: these weights are not trained yet)

6.5Lecture summary

• A single perceptron cannot learn XOR: the classes are not linearly separable (the four inequalities contradict).
• One hidden layer with a non-linear activation solves it: XOR = OR AND NAND.
• Training can find the weights by backprop, but from an unlucky start it can stall.

What is likely to be asked from Week 1, with 30-second answers, deeper explanations, a lecture-or-extra tag on each question, and code you can run.

Week 1 questions are mostly why questions: why an activation, why cross-entropy, why a hidden layer, why gradients vanish. Each answer starts with a 30-second version, then goes deeper. Questions are tagged From the notes or Extra.

7.1Interview heat-map: what is most likely

7.2Neurons, networks and activations

AND: weights=[2. 1.], bias=-3.0, converged after 6 epochs
OR: weights=[1. 1.], bias=-1.0, converged after 4 epochs
XOR: weights=[-1.  0.], bias=0.0, did NOT converge in 20 epochs

same output: True
collapsed W shape: (2, 3)
with ReLU still equals the single layer? False

Sigmoid              first layer = 2.25e-18   last layer = 1.60e+00   first/last = 1.41e-18
Tanh                 first layer = 2.28e-07   last layer = 8.98e-02   first/last = 2.54e-06
ReLU (default init)  first layer = 3.77e-10   last layer = 4.40e-02   first/last = 8.58e-09
ReLU (He init)       first layer = 1.90e-01   last layer = 7.72e-01   first/last = 2.45e-01

fraction of inputs with z > 0: 0.0
ReLU gradient for those inputs: 0.0 non-zero entries out of 1000
-> the neuron outputs 0 and passes 0 gradient: it cannot recover by itself
leaky ReLU gradient is still: 0.01 (small, but alive)

one-hot cat vs dog dot product: 0.0
one-hot cat vs cat dot product: 1.0
embedding cat~dog : 0.991
embedding cat~bird: 0.22

7.3Learning: loss, gradient descent, backprop

MSE: 0.16666666666666666
BCE label=1, p=0.9 : 0.1054
BCE label=1, p=0.1 : 2.3026
BCE label=1, p=0.01: 4.6052   <- a confident wrong answer is punished hard
MSE label=1, p=0.01: 0.9801   <- MSE can never exceed 1 here

lr = 0.01: [1.0, 1.04, 1.079, 1.118, 1.155, 1.192, 1.228, 1.264, 1.298]
lr = 0.1 : [1.0, 1.4, 1.72, 1.976, 2.181, 2.345, 2.476, 2.581, 2.664]
lr = 0.9 : [1.0, 4.6, 1.72, 4.024, 2.181, 3.655, 2.476, 3.419, 2.664]
lr = 1.0 : [1.0, 5.0, 1.0, 5.0, 1.0, 5.0, 1.0, 5.0, 1.0]
lr = 1.1 : [1.0, 5.4, 0.12, 6.456, -1.147, 7.977, -2.972, 10.166, -5.6]

batch (all 1000)   updates=    5  w=1.160  b=0.770
SGD (1 sample)     updates= 5000  w=3.023  b=2.129
mini-batch (32)    updates=  160  w=3.040  b=2.021

forward: 5.0 20.0
dL/da = 4.0  dL/db = 4.0  dL/dc = 5.0
autograd agrees: 4.0 4.0 5.0

probabilities: [0.659  0.2424 0.0986]  sum = 1.0
big logits, naive exp would overflow: [0.2689 0.7311]

loss: [0.49602]
dW2: [[-0.20505 -0.20604 -0.16501]]
dW1:
 [[-0.03179  0.06359]
 [-0.02308  0.04616]
 [ 0.01678 -0.03356]]
max |backprop - numerical| for W1: 8.295730768992371e-11

7.4XOR

x=(0, 0)  hidden (OR, NAND) = (0, 1)  ->  output 0
x=(0, 1)  hidden (OR, NAND) = (1, 1)  ->  output 1
x=(1, 0)  hidden (OR, NAND) = (1, 1)  ->  output 1
x=(1, 1)  hidden (OR, NAND) = (1, 0)  ->  output 0

seed 0: loss 0.0002  predictions [0.0, 1.0, 1.0, 0.0]  -> solved
seed 1: loss 0.3467  predictions [0.0, 0.5, 1.0, 0.5]  -> STUCK on a plateau
seed 2: loss 0.0004  predictions [0.0, 1.0, 1.0, 0.0]  -> solved
seed 3: loss 0.0004  predictions [0.0, 1.0, 1.0, 0.0]  -> solved
seed 4: loss 0.0001  predictions [0.0, 1.0, 1.0, 0.0]  -> solved
seed 5: loss 0.0003  predictions [0.0, 1.0, 1.0, 0.0]  -> solved

Week 1 on one page: a chapter-by-chapter glance, key terms, the pairs to tell apart, the training symptoms you can now diagnose, a checklist and corrections to the source notes.

8.1Week 1 at a glance

8.2Key terms

8.3The pairs you must tell apart

8.4Training symptoms you can now diagnose

8.5Last-minute checklist

8.6Corrections and additions to the source notes