Week 2 Notes

Deep Learning,
Week 2

Introduction to DL and GenAI, Week 2: PyTorch from the ground up. Tensors, broadcasting, the GPU, autograd, nn.Module, losses and optimisers, the training loop, the Fashion-MNIST workflow and the customer-churn case study. Every code block here was really run.

9 chapters17 worked examples~47 min read

TensorsBroadcastingAutogradnn.ModuleTraining loopFashion-MNISTChurnInterview prep

Part 1

Tensors

Week 1 was the theory; this week is the toolkit that turns it into running code. PyTorch gives you three things: tensors (NumPy-like arrays that can live on a GPU), autograd (automatic differentiation, so you never write backprop by hand again) and nn (ready-made layers, losses and optimisers). Everything in the rest of the course is built from these.

About these notes. The source notes only list the PyTorch topics for this week (tensors, operations, GPU, the workflow, Fashion-MNIST, customer churn); they contain no code. Everything below is written from that topic list, and every code block here was really run on PyTorch 2.x on a CPU. Fashion-MNIST needs a download, so the full pipeline is shown and a same-shape synthetic stand-in is run.

1.1A tensor is an n-dimensional array

Figure 1. From a single number to a batch of images, one more dimension each time

A tensor is a block of numbers of one type with a shape. Three attributes you will print constantly: .shape (the size of each dimension), .dtype (the number type) and .device (CPU or GPU).

main.py

import torch

a = torch.tensor([[1, 2, 3], [4, 5, 6]])           # from nested lists
print(a)
print("shape:", a.shape, " dtype:", a.dtype, " ndim:", a.ndim, " numel:", a.numel())

print(torch.zeros(2, 3))
print(torch.ones(2, 2))
print(torch.arange(0, 10, 3))                        # 0, 3, 6, 9
print(torch.linspace(0, 1, 5))                       # 5 points from 0 to 1
torch.manual_seed(0)
print(torch.rand(2, 2))                              # uniform in [0, 1)
print(torch.randn(2, 2))                             # standard normal

Output

tensor([[1, 2, 3],
        [4, 5, 6]])
shape: torch.Size([2, 3])  dtype: torch.int64  ndim: 2  numel: 6
tensor([[0., 0., 0.],
        [0., 0., 0.]])
tensor([[1., 1.],
        [1., 1.]])
tensor([0, 3, 6, 9])
tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])
tensor([[0.4963, 0.7682],
        [0.0885, 0.1320]])
tensor([[-2.1788,  0.5684],
        [-1.0845, -1.3986]])

torch.tensor(...) builds from Python data; integers become int64, decimals float32.
zeros, ones, arange, linspace make regular ones; rand is uniform on [0,1), randn is standard normal.
torch.manual_seed(0) makes random numbers repeatable, so your experiment can be reproduced.

1.2dtype: the number type matters

Neural networks run on float32. Class labels are int64 (also called long). Mixing them silently promotes (int + float gives float) or raises an error, so check .dtype first when something breaks. .float(), .long(), .half() convert; converting a float to an integer truncates toward zero (1.7 becomes 1, -1.7 becomes -1).

main.py

import torch
i = torch.tensor([1, 2, 3])
f = torch.tensor([1.0, 2.0, 3.0])
print(i.dtype, f.dtype)
print((i + f).dtype)                                # int + float -> float
print(i.float().dtype, f.long().dtype, f.half().dtype)
print(torch.tensor([1.7, -1.7]).long())            # conversion truncates toward zero

try:
    torch.tensor([1.0, 2.0]) @ torch.tensor([1, 2])  # float32 vs int64 matmul
except RuntimeError as e:
    print("RuntimeError:", str(e)[:70])

Output

torch.int64 torch.float32
torch.float32
torch.float32 torch.int64 torch.float16
tensor([ 1, -1])
RuntimeError: dot : expected both vectors to have same dtype, but found Float and Lo

1.3Indexing and slicing

Indexing is NumPy's: x[row, col], ranges with :, negative indices from the end, and boolean masks that pick (or assign) every element satisfying a condition. .item() turns a one-element tensor into a plain Python number.

main.py

import torch
x = torch.arange(12).reshape(3, 4)
print(x)
print("row 1        :", x[1])
print("column 2     :", x[:, 2])
print("block        :", x[0:2, 1:3].tolist())
print("last element :", x[-1, -1].item())
mask = x > 6
print("mask sum     :", mask.sum().item(), " selected:", x[mask].tolist())
x[x % 2 == 0] = -1                                   # assign through a mask
print(x)

Output

tensor([[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]])
row 1        : tensor([4, 5, 6, 7])
column 2     : tensor([ 2,  6, 10])
block        : [[1, 2], [5, 6]]
last element : 11
mask sum     : 5  selected: [7, 8, 9, 10, 11]
tensor([[-1,  1, -1,  3],
        [-1,  5, -1,  7],
        [-1,  9, -1, 11]])

1.4Reshaping: same numbers, new shape

Most shape bugs in deep learning are reshape bugs. reshape(a, b) re-slices the same numbers (one dimension may be -1: "work it out"). unsqueeze/squeeze add/remove a size-1 dimension, .T/permute reorder dimensions, and flatten(start_dim=1) turns an image batch into the (N, features) matrix a linear layer wants. A view shares memory with its source: edit one, you edit both.

main.py

import torch
x = torch.arange(12)
print(x.reshape(3, 4).shape, x.reshape(2, -1).shape, x.reshape(-1, 6).shape)   # -1 = infer
m = x.reshape(3, 4)
print("transpose :", m.T.shape, " permute:", m.unsqueeze(0).permute(2, 0, 1).shape)
print("unsqueeze :", m.unsqueeze(0).shape, "  squeeze:", m.unsqueeze(0).squeeze(0).shape)

# an image batch: (N, C, H, W) -> flatten everything but the batch dimension
batch = torch.zeros(32, 1, 28, 28)
print("flatten   :", batch.flatten(start_dim=1).shape)

# view shares memory with the original
v = x.view(3, 4)
v[0, 0] = 99
print("view shares storage, x[0] became:", x[0].item())

Output

torch.Size([3, 4]) torch.Size([2, 6]) torch.Size([2, 6])
transpose : torch.Size([4, 3])  permute: torch.Size([4, 1, 3])
unsqueeze : torch.Size([1, 3, 4])   squeeze: torch.Size([3, 4])
flatten   : torch.Size([32, 784])
view shares storage, x[0] became: 99

1.5Arithmetic: element-wise vs matrix

The one operator pair to never confuse: * multiplies element by element (the Hadamard product, the $\odot$ of Week 1) and @ is the matrix product (the layer $Wx$ ). Reductions (sum, mean, max, argmax) take a dim: the dimension that disappears.

main.py

import torch
a = torch.tensor([[1., 2.], [3., 4.]])
b = torch.tensor([[10., 20.], [30., 40.]])
print("a + b    :", (a + b).tolist())
print("a * b    :", (a * b).tolist(), " <- element-wise")
print("a @ b    :", (a @ b).tolist(), " <- matrix product")
print("sum      :", a.sum().item(), " rows:", a.sum(dim=0).tolist(), " cols:", a.sum(dim=1).tolist())
print("mean/std :", a.mean().item(), round(a.std().item(), 4))
print("argmax   :", a.argmax().item(), " max per row:", a.max(dim=1).values.tolist())
print("exp/log  :", torch.exp(torch.tensor([0., 1.])).tolist(), torch.log(torch.tensor([1., 2.718281828])).tolist())

Output

a + b    : [[11.0, 22.0], [33.0, 44.0]]
a * b    : [[10.0, 40.0], [90.0, 160.0]]  <- element-wise
a @ b    : [[70.0, 100.0], [150.0, 220.0]]  <- matrix product
sum      : 10.0  rows: [4.0, 6.0]  cols: [3.0, 7.0]
mean/std : 2.5 1.291
argmax   : 3  max per row: [2.0, 4.0]
exp/log  : [1.0, 2.7182817459106445] [0.0, 0.9999999403953552]

1.6Lecture summary

Tensor = typed block of numbers with a shape; check .shape .dtype .device first.
Labels are long, data is float32; conversions truncate.
* element-wise, @ matrix product; view shares memory.

Next · Part 2Broadcasting, NumPy and the GPU

Part 2

Broadcasting, NumPy and the GPU

Three small topics that cause a surprising share of real bugs: broadcasting (the rule behind "subtract the mean from every row"), the NumPy bridge, and keeping data and model on the same device.

2.1Broadcasting

Figure 2. A shape-(3,) row is stretched down to shape (2, 3) without copying it into memory

When two tensors have different shapes, PyTorch lines the shapes up from the right. In each position the two sizes must be equal, or one of them must be 1 (a missing dimension counts as 1). The size-1 side is stretched to match. This is why x + row adds the row to every row of x, and x + col adds the column to every column:

main.py

import torch
x = torch.tensor([[1., 2., 3.],
                  [4., 5., 6.]])            # shape (2, 3)
row = torch.tensor([10., 20., 30.])          # shape (3,)     -> stretched down the rows
col = torch.tensor([[100.], [200.]])         # shape (2, 1)   -> stretched across the columns
print(x + row)
print(x + col)
print((x + row + col).shape)

# the classic use: standardise every feature (column) of a data matrix
data = torch.tensor([[1., 100.], [2., 200.], [3., 300.], [4., 400.]])
z = (data - data.mean(dim=0)) / data.std(dim=0)
print(z.mean(dim=0).abs().max().item() < 1e-6, z.std(dim=0).tolist())

try:
    torch.ones(2, 3) + torch.ones(4, 3)       # 2 vs 4 in dimension 0: neither is 1
except RuntimeError as e:
    print("RuntimeError:", e)

Output

tensor([[11., 22., 33.],
        [14., 25., 36.]])
tensor([[101., 102., 103.],
        [204., 205., 206.]])
torch.Size([2, 3])
True [1.0, 1.0]
RuntimeError: The size of tensor a (2) must match the size of tensor b (4) at non-singleton dimension 0

The second half shows the idiom you will write constantly, standardising each feature: subtract the per-column mean and divide by the per-column standard deviation. The result has mean 0 and std 1 in every column. And when the shapes cannot be matched (2 against 4 in dimension 0) you get the error the notes' "shape mismatch" warnings are about.

Silent broadcasting bugs

Broadcasting never complains when it can stretch. A loss of shape (N,) minus a target of shape (N, 1) quietly becomes an (N, N) matrix and trains to nonsense. When a loss looks strange, print the shapes of everything that meets inside it.

2.2The NumPy bridge

torch.from_numpy(a) makes a tensor that shares memory with the array (edit one, the other changes); torch.tensor(a) copies. Going back, t.numpy() works only for a tensor on the CPU that is not tracking gradients.

main.py

import numpy as np, torch
a = np.array([1., 2., 3.])
t = torch.from_numpy(a)               # shares memory with the numpy array
a[0] = 100
print(t)
t2 = torch.tensor(a)                  # torch.tensor COPIES
a[1] = 200
print(t2, " <- unchanged by the second edit")
print(t.numpy() is not None, type(t.numpy()).__name__)

Output

tensor([100.,   2.,   3.], dtype=torch.float64)
tensor([100.,   2.,   3.], dtype=torch.float64)  <- unchanged by the second edit
True ndarray

2.3Devices: CPU and GPU

A GPU does thousands of multiply-adds in parallel, exactly what matrix products need, so training that takes hours on a CPU takes minutes. Data and model each live on a device, and an operation needs both on the same one. The standard pattern picks the device once and moves everything with .to(device):

main.py

import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
print("training on:", device)

x = torch.randn(3, 3).to(device)
print("tensor device:", x.device)

model = torch.nn.Linear(3, 1).to(device)          # a model moves with .to(), in place
out = model(x)
print("output device:", out.device)

# model and data must be on the SAME device. Mixing them is an error you will meet:
if device == "cuda":
    print("(on a GPU machine, model(x.cpu()) would raise: Expected all tensors to be on the same device)")
else:
    print("(this machine has no GPU, so everything here is CPU; on a GPU machine model(x.cpu()) would raise a device-mismatch error)")

Output

training on: cpu
tensor device: cpu
output device: cpu
(this machine has no GPU, so everything here is CPU; on a GPU machine model(x.cpu()) would raise a device-mismatch error)

This machine has no GPU, so it printed cpu everywhere. The same code on a GPU box prints cuda. In the training loop you will also move each batch (xb, yb = xb.to(device), yb.to(device)). Mixing a CUDA model with a CPU tensor raises "Expected all tensors to be on the same device".

2.4Lecture summary

Broadcasting: align shapes from the right, sizes equal or 1. Great for standardising; dangerous when silent.
from_numpy shares, tensor copies; .detach().numpy() to leave autograd.
Pick device once; model and data on the same one.

Next · Part 3Autograd

Part 3

Autograd

Last week you derived backpropagation and checked it against finite differences. Autograd does exactly that for you, for any combination of tensor operations: it records every operation on a tensor that asks for gradients, and .backward() replays the record in reverse applying the chain rule.

3.1requires_grad and backward()

Make a tensor a leaf that tracks gradients with requires_grad=True; every operation using it is recorded in a graph. Call .backward() on a scalar result and each tracked leaf gets a .grad holding $\partial\,\text{result}/\partial\,\text{leaf}$ . Here $y=wx+b$ , $L=(y-10)^2$ with $w=2,b=1,x=3$ :

main.py

import torch
w = torch.tensor(2.0, requires_grad=True)
b = torch.tensor(1.0, requires_grad=True)
x = torch.tensor(3.0)

y = w * x + b            # y = 7
loss = (y - 10) ** 2     # loss = 9
loss.backward()
print("loss =", loss.item())
print("dloss/dw =", w.grad.item(), "  (2 (y-10) x = 2*(-3)*3 = -18)")
print("dloss/db =", b.grad.item(), "  (2 (y-10)   = -6)")
print("x has no grad:", x.grad)

Output

loss = 9.0
dloss/dw = -18.0   (2 (y-10) x = 2*(-3)*3 = -18)
dloss/db = -6.0   (2 (y-10)   = -6)
x has no grad: None

Check by hand: $y=7$ , $L=9$ ; $\partial L/\partial w = 2(y-10)\cdot x = 2(-3)(3)=-18$ ; $\partial L/\partial b = 2(y-10)=-6$ . x did not ask for gradients, so x.grad is None.

3.2The graph, and why it vanishes after backward()

Each result remembers the operation that made it in .grad_fn. Inputs you created are leaves; everything computed from them is not. After backward() the graph is freed to save memory, so a second backward() on the same result fails (pass retain_graph=True only if you really need two passes).

main.py

import torch
x = torch.tensor(2.0, requires_grad=True)
y = x ** 2
z = y * 3
print("z.grad_fn         :", z.grad_fn.name())
print("y.grad_fn         :", y.grad_fn.name())
print("x is a leaf       :", x.is_leaf, "  y is a leaf:", y.is_leaf)

z.backward()                              # walks the graph, then frees it
print("dz/dx =", x.grad.item(), " (6x = 12)")
try:
    z.backward()                          # the graph is gone
except RuntimeError as e:
    print("RuntimeError:", str(e)[:75])

Output

z.grad_fn         : MulBackward0
y.grad_fn         : PowBackward0
x is a leaf       : True   y is a leaf: False
dz/dx = 12.0  (6x = 12)
RuntimeError: Trying to backward through the graph a second time (or directly access save

3.3Gradients accumulate: the reason for zero_grad()

backward() does not overwrite .grad, it adds to it. That is deliberate (it lets you sum gradients over several small batches to mimic a big one) but means that in an ordinary loop you must clear them each step:

main.py

import torch
w = torch.tensor(1.0, requires_grad=True)

for step in range(3):
    loss = (3 * w) ** 2
    loss.backward()                        # gradient is ADDED into w.grad
    print(f"step {step}: w.grad = {w.grad.item()}")

print("--- now zero the gradient every step ---")
w.grad = None
for step in range(3):
    if w.grad is not None:
        w.grad.zero_()
    loss = (3 * w) ** 2
    loss.backward()
    print(f"step {step}: w.grad = {w.grad.item()}")

Output

step 0: w.grad = 18.0
step 1: w.grad = 36.0
step 2: w.grad = 54.0
--- now zero the gradient every step ---
step 0: w.grad = 18.0
step 1: w.grad = 18.0
step 2: w.grad = 18.0

18, 36, 54: the gradient of the same loss keeps growing. What happens if you forget it in a real optimisation? On $L=(w-3)^2$ with $\eta=0.3$ , the correct loop slides to 3; the forgetful one overshoots and swings:

main.py

import torch

def run(zero, lr=0.3, steps=10):
    w = torch.tensor(0.0, requires_grad=True)          # minimise (w - 3)^2, optimum at w = 3
    opt = torch.optim.SGD([w], lr=lr)
    path = []
    for _ in range(steps):
        if zero:
            opt.zero_grad()
        loss = (w - 3) ** 2
        loss.backward()
        opt.step()
        path.append(round(w.item(), 2))
    return path

print("with    zero_grad:", run(True))
print("without zero_grad:", run(False))

Output

with    zero_grad: [1.8, 2.52, 2.81, 2.92, 2.97, 2.99, 3.0, 3.0, 3.0, 3.0]
without zero_grad: [1.8, 4.32, 6.05, 5.95, 4.08, 1.56, -0.09, 0.11, 2.05, 4.55]

The stored gradients behave like a momentum that never decays, so the walk oscillates instead of settling. Be careful with the opposite conclusion, though: on a small network the same bug can look faster:

main.py

import torch, torch.nn as nn
torch.manual_seed(0)
X = torch.randn(256, 2)
y = (X[:, 0] * X[:, 1] > 0).float()                  # quadrants: not linearly separable

def train(zero, lr=0.05, steps=200):
    torch.manual_seed(1)
    m = nn.Sequential(nn.Linear(2, 16), nn.Tanh(), nn.Linear(16, 1))
    opt = torch.optim.SGD(m.parameters(), lr=lr)
    for _ in range(steps):
        if zero:
            opt.zero_grad()
        loss = nn.functional.binary_cross_entropy_with_logits(m(X).squeeze(1), y)
        loss.backward()
        opt.step()
    return round(loss.item(), 3)

print("with    zero_grad, final loss:", train(True))
print("without zero_grad, final loss:", train(False), " (it can even look faster: the stored gradients act like momentum that never decays)")

Output

with    zero_grad, final loss: 0.675
without zero_grad, final loss: 0.089  (it can even look faster: the stored gradients act like momentum that never decays)

That lower final loss is an accident, not a feature. The update is no longer the gradient of anything you can reason about. It is still a bug.

3.4Turning tracking off: no_grad and detach

Recording a graph costs memory. For evaluation and inference you do not need gradients, so wrap the code in with torch.no_grad():. tensor.detach() gives a tensor with the same values cut off from the graph; you need it before .numpy() or plotting.

main.py

import torch
w = torch.tensor([1.0, 2.0], requires_grad=True)

with torch.no_grad():                      # nothing inside is recorded
    y = w * 2
print("inside no_grad:", y.requires_grad)

y = w * 2
print("normal        :", y.requires_grad)
d = y.detach()                              # same values, cut from the graph
print("detached      :", d.requires_grad)

try:
    y.numpy()                               # a tensor that requires grad cannot go to numpy directly
except RuntimeError as e:
    print("RuntimeError:", str(e)[:66])
print("detach().numpy():", y.detach().numpy())

Output

inside no_grad: False
normal        : True
detached      : False
RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach
detach().numpy(): [2. 4.]

3.5backward() needs a scalar

The loss is a single number, so loss.backward() is unambiguous. A vector output has one gradient per output element; PyTorch refuses to guess how you want them combined. Reduce first (.sum(), .mean()):

main.py

import torch
x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
y = x ** 2                                  # a vector, not a scalar
try:
    y.backward()
except RuntimeError as e:
    print("RuntimeError:", e)

y.sum().backward()                          # reduce to a scalar first
print("grad of sum(x^2):", x.grad.tolist(), " (2x)")

Output

RuntimeError: grad can be implicitly created only for scalar outputs
grad of sum(x^2): [2.0, 4.0, 6.0]  (2x)

3.6Autograd equals your hand-derived gradient

For linear regression with mean-squared error, $L=\frac1n\lVert Xw+b-y\rVert^2$ , calculus gives $\partial L/\partial w=\frac2n X^\top(Xw+b-y)$ . Compare it with autograd on random data:

main.py

import torch
torch.manual_seed(0)
X = torch.randn(100, 3)
true_w = torch.tensor([2.0, -1.0, 0.5])
y = X @ true_w + 0.3

w = torch.zeros(3, requires_grad=True)
b = torch.zeros(1, requires_grad=True)
loss = ((X @ w + b - y) ** 2).mean()
loss.backward()

# the same gradient written by hand: dL/dw = (2/n) X^T (Xw + b - y)
err = X @ w.detach() + b.detach() - y
manual_w = 2 * X.T @ err / len(X)
manual_b = 2 * err.mean()
print("autograd dw:", w.grad.tolist())
print("manual   dw:", manual_w.tolist())
print("equal?", torch.allclose(w.grad, manual_w), torch.allclose(b.grad, manual_b.reshape(1)))

Output

autograd dw: [-3.0667765140533447, 1.2619411945343018, -0.6028164625167847]
manual   dw: [-3.0667765140533447, 1.2619410753250122, -0.6028165221214294]
equal? True True

3.7A trap: in-place operations

Autograd saved the output of exp because its own derivative is itself. Editing that tensor in place (y += 1) destroys what backward needs:

main.py

import torch
x = torch.tensor([1.0, 2.0], requires_grad=True)
y = torch.exp(x)
y += 1                                    # in-place edit of a tensor autograd still needs
try:
    y.sum().backward()
except RuntimeError as e:
    print("RuntimeError:", str(e)[:78])

y2 = torch.exp(x)
y2 = y2 + 1                               # out of place: fine
y2.sum().backward()
print("out-of-place grad:", x.grad.tolist())

Output

RuntimeError: one of the variables needed for gradient computation has been modified by an i
out-of-place grad: [2.7182817459106445, 7.389056205749512]

3.8Lecture summary

requires_grad=True + loss.backward() fills .grad; it is the Week 1 chain rule, automated.
The graph is freed after backward(); gradients accumulate, so call zero_grad() every step.
no_grad for evaluation, detach before NumPy; avoid in-place edits.

Next · Part 4Modules, Losses and Optimisers

Part 4

Modules, Losses and Optimisers

You could build a network from raw tensors and requires_grad, but you will not. torch.nn packages the reusable parts: layers that own their parameters, loss functions, and (in torch.optim) the update rules. This chapter is the vocabulary.

4.1nn.Linear: one layer is two tensors

nn.Linear(n_in, n_out) creates a weight of shape (n_out, n_in) and a bias of shape (n_out,), both with requires_grad=True, and computes $y=xW^\top+b$ for a whole batch at once (the batch is the first dimension).

main.py

import torch, torch.nn as nn
torch.manual_seed(0)
layer = nn.Linear(3, 2)                     # y = x W^T + b
print("weight shape:", tuple(layer.weight.shape), " bias shape:", tuple(layer.bias.shape))
x = torch.tensor([[1., 2., 3.], [0., 1., 0.]])          # a batch of 2 samples
y = layer(x)
print("output shape:", tuple(y.shape))
manual = x @ layer.weight.T + layer.bias
print("same as the formula:", torch.allclose(y, manual))

Output

weight shape: (2, 3)  bias shape: (2,)
output shape: (2, 2)
same as the formula: True

4.2nn.Module: your network is a class

Every network is a subclass of nn.Module. In __init__ you create the layers (assigning them to self registers their parameters); in forward you describe the computation in plain Python. You never write backward: autograd derives it from forward. For a simple chain of layers nn.Sequential is a shortcut.

main.py

import torch, torch.nn as nn

class MLP(nn.Module):
    def __init__(self, n_in, n_hidden, n_out):
        super().__init__()
        self.fc1 = nn.Linear(n_in, n_hidden)
        self.fc2 = nn.Linear(n_hidden, n_out)
        self.act = nn.ReLU()

    def forward(self, x):                    # the forward pass is ordinary Python
        return self.fc2(self.act(self.fc1(x)))

model = MLP(4, 8, 3)
print(model)
print("parameters:", sum(p.numel() for p in model.parameters()))
print("trainable :", sum(p.numel() for p in model.parameters() if p.requires_grad))
print("output    :", tuple(model(torch.randn(5, 4)).shape))
print([name for name, _ in model.named_parameters()])

Output

MLP(
  (fc1): Linear(in_features=4, out_features=8, bias=True)
  (fc2): Linear(in_features=8, out_features=3, bias=True)
  (act): ReLU()
)
parameters: 67
trainable : 67
output    : (5, 3)
['fc1.weight', 'fc1.bias', 'fc2.weight', 'fc2.bias']

The parameter count $(4\cdot8+8)+(8\cdot3+3)=67$ matches the Week 1 formula $n_ln_{l-1}+n_l$ per layer. model.parameters() is what you hand to the optimiser.

4.3train() and eval()

Some layers behave differently while learning than while predicting: dropout randomly zeros units only in training mode, and batch norm (Week 4) uses batch statistics in training and stored ones in evaluation. model.train() and model.eval() flip a flag for every layer. Forgetting eval() is a classic reason test accuracy looks random.

main.py

import torch, torch.nn as nn
torch.manual_seed(0)
drop = nn.Dropout(p=0.5)
x = torch.ones(8)

drop.train()
print("train mode :", drop(x).tolist())     # about half zeroed, survivors scaled by 1/(1-p)
drop.eval()
print("eval mode  :", drop(x).tolist())     # identity
print("model.training flag:", drop.training)

Output

train mode : [0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 2.0, 2.0]
eval mode  : [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
model.training flag: False

Note the survivors in train mode are doubled (scaled by $1/(1-p)$ ). This is "inverted dropout": PyTorch compensates during training so that evaluation needs no change. (The notes describe the older convention of scaling by $1-p$ at inference; the effect, matching expected activations, is the same.)

4.4Loss functions

Task	Loss	Expects
Regression	`nn.MSELoss()`	predictions and targets, same shape
Binary classification	`nn.BCEWithLogitsLoss()`	logits (raw scores) and float targets 0/1
Multi-class	`nn.CrossEntropyLoss()`	logits of shape (N, C) and class indices (long) of shape (N,)

The two classification losses apply the sigmoid or softmax inside, in a numerically stable way, so the model must output raw logits. Feeding already-softmaxed probabilities into CrossEntropyLoss is a very common mistake: it still runs, but computes a different, wrong number:

main.py

import torch, torch.nn as nn
logits = torch.tensor([[2.0, 1.0, 0.1], [0.5, 2.5, 0.2]])      # raw scores, NOT probabilities
target = torch.tensor([0, 1])                                  # class indices

ce = nn.CrossEntropyLoss()
print("CrossEntropyLoss:", round(ce(logits, target).item(), 4))

# what it does inside: log_softmax, then pick the true class, negate, average
logp = torch.log_softmax(logits, dim=1)
manual = -logp[torch.arange(2), target].mean()
print("by hand         :", round(manual.item(), 4))

# a common mistake: feeding softmax outputs (probabilities) into CrossEntropyLoss
wrong = ce(torch.softmax(logits, dim=1), target)
print("with softmax first (wrong):", round(wrong.item(), 4))

bce = nn.BCEWithLogitsLoss()                                   # sigmoid + BCE in one stable op
print("BCEWithLogits:", round(bce(torch.tensor([2.0, -1.0]), torch.tensor([1.0, 0.0])).item(), 4))
print("MSELoss      :", nn.MSELoss()(torch.tensor([2.5, 0.0]), torch.tensor([3.0, 0.0])).item())

Output

CrossEntropyLoss: 0.3143
by hand         : 0.3143
with softmax first (wrong): 0.7425
BCEWithLogits: 0.2201
MSELoss      : 0.125

Two more traps: the target of CrossEntropyLoss must be integer class indices, and the shapes must be right.

main.py

import torch, torch.nn as nn
ce = nn.CrossEntropyLoss()
logits = torch.randn(4, 3)
good = torch.tensor([0, 2, 1, 1])                 # class indices, dtype long
print("ok:", round(ce(logits, good).item(), 3))
try:
    ce(logits, torch.tensor([0.0, 2.0, 1.0, 1.0]))     # float class labels
except RuntimeError as e:
    print("RuntimeError:", str(e)[:80])

Output

ok: 1.234
RuntimeError: expected target dtype to be Long or Byte, but got Float

main.py

import torch, torch.nn as nn
model = nn.Linear(4, 2)
x = torch.randn(5, 3)                 # 3 features, but the layer expects 4
try:
    model(x)
except RuntimeError as e:
    print("RuntimeError:", e)

Output

RuntimeError: mat1 and mat2 shapes cannot be multiplied (5x3 and 4x2)

4.5Optimisers

An optimiser holds the parameter list and the update rule. opt.zero_grad() clears, opt.step() applies $w\leftarrow w-\eta\,\text{grad}$ (for SGD). It reproduces the hand calculation from Week 1 exactly:

main.py

import torch
w = torch.tensor(1.0, requires_grad=True)
opt = torch.optim.SGD([w], lr=0.1)           # L(w) = (w - 3)^2 again
for step in range(1, 4):
    opt.zero_grad()
    loss = (w - 3) ** 2
    loss.backward()
    opt.step()                                # w <- w - lr * w.grad
    print(f"step {step}: w = {w.item():.4f}, loss = {loss.item():.4f}")
print("same numbers as the Week 1 hand calculation: 1.4, 1.72, 1.976")

Output

step 1: w = 1.4000, loss = 4.0000
step 2: w = 1.7200, loss = 2.5600
step 3: w = 1.9760, loss = 1.6384
same numbers as the Week 1 hand calculation: 1.4, 1.72, 1.976

SGD is the baseline; Adam (Week 5) is the common default and usually trains faster with lr=1e-3.

4.6Saving and loading

Save the state_dict (a dictionary of parameter tensors), not the Python object. To load, build the same architecture first, then load the weights into it:

main.py

import torch, torch.nn as nn, io
torch.manual_seed(0)
model = nn.Linear(3, 2)
buf = io.BytesIO()
torch.save(model.state_dict(), buf)                  # save the parameters, not the Python object
buf.seek(0)

fresh = nn.Linear(3, 2)                              # a differently initialised model...
print("before load equal:", torch.equal(fresh.weight, model.weight))
fresh.load_state_dict(torch.load(buf))
print("after  load equal:", torch.equal(fresh.weight, model.weight))
print("state_dict keys  :", list(model.state_dict().keys()))

Output

before load equal: False
after  load equal: True
state_dict keys  : ['weight', 'bias']

4.7Lecture summary

nn.Module = layers in __init__ + forward; autograd supplies backward.
Losses take logits; CrossEntropy wants long class indices.
train()/eval() switch dropout and batch norm; save state_dict.

Next · Part 5Data and the Training Loop

Part 5

Data and the Training Loop

Every PyTorch project, from this week's toy to a billion-parameter model, is the same loop. Learn its five lines and the data pipeline in front of it, and you can read any training script.

5.1The data pipeline: Dataset and DataLoader

Figure 3. Data flows left to right; the shape under each box is what leaves it

A Dataset answers "give me sample number i" (__len__ and __getitem__). TensorDataset(X, y) wraps tensors you already have. A DataLoader turns a Dataset into an iterator of mini-batches, shuffling the order every epoch. Here is a two-class "two moons" problem (a curved boundary a straight line cannot cut), made from a seeded generator:

main.py

import torch, math
from torch.utils.data import TensorDataset, DataLoader

def make_moons(n, noise, seed):
    g = torch.Generator().manual_seed(seed)
    t = torch.rand(n, generator=g) * math.pi
    upper = torch.stack([torch.cos(t), torch.sin(t)], 1)
    lower = torch.stack([1 - torch.cos(t), 0.5 - torch.sin(t)], 1)
    X = torch.cat([upper, lower]) + noise * torch.randn(2 * n, 2, generator=g)
    y = torch.cat([torch.zeros(n), torch.ones(n)])
    perm = torch.randperm(2 * n, generator=g)
    return X[perm], y[perm]

X, y = make_moons(500, 0.15, seed=0)
ds = TensorDataset(X, y)
print("dataset size:", len(ds), " one item:", [t.shape for t in ds[0]])
loader = DataLoader(ds, batch_size=64, shuffle=True)
xb, yb = next(iter(loader))
print("one batch:", tuple(xb.shape), tuple(yb.shape), " batches per epoch:", len(loader))

Output

dataset size: 1000  one item: [torch.Size([2]), torch.Size([])]
one batch: (64, 2) (64,)  batches per epoch: 16

1000 samples at 64 per batch give $\lceil 1000/64\rceil=16$ batches per epoch (the last one is smaller). One epoch = one full pass over the data = 16 weight updates here.

5.2The five-line training step

Figure 4. The loop body, in order. Memorise the order: clear, forward, loss, backward, step

Putting everything together: a small MLP, binary cross-entropy on logits, Adam, 30 epochs, and a held-out test set that the model never trains on. At the end of some epochs we switch to eval() and no_grad() and measure accuracy:

main.py

import torch, torch.nn as nn, math
from torch.utils.data import TensorDataset, DataLoader
torch.manual_seed(0)

def make_moons(n, noise, seed):
    g = torch.Generator().manual_seed(seed)
    t = torch.rand(n, generator=g) * math.pi
    upper = torch.stack([torch.cos(t), torch.sin(t)], 1)
    lower = torch.stack([1 - torch.cos(t), 0.5 - torch.sin(t)], 1)
    X = torch.cat([upper, lower]) + noise * torch.randn(2 * n, 2, generator=g)
    y = torch.cat([torch.zeros(n), torch.ones(n)])
    perm = torch.randperm(2 * n, generator=g)
    return X[perm], y[perm]

Xtr, ytr = make_moons(500, 0.15, seed=0)
Xte, yte = make_moons(200, 0.15, seed=1)
loader = DataLoader(TensorDataset(Xtr, ytr), batch_size=64, shuffle=True)

model = nn.Sequential(nn.Linear(2, 16), nn.ReLU(), nn.Linear(16, 16), nn.ReLU(), nn.Linear(16, 1))
loss_fn = nn.BCEWithLogitsLoss()
opt = torch.optim.Adam(model.parameters(), lr=0.01)

for epoch in range(1, 31):
    model.train()
    total = 0.0
    for xb, yb in loader:
        opt.zero_grad()                              # 1. clear old gradients
        logits = model(xb).squeeze(1)                # 2. forward
        loss = loss_fn(logits, yb)                   # 3. loss
        loss.backward()                              # 4. backward
        opt.step()                                   # 5. update
        total += loss.item() * len(xb)
    if epoch % 5 == 0 or epoch == 1:
        model.eval()
        with torch.no_grad():
            acc = ((model(Xte).squeeze(1) > 0).float() == yte).float().mean().item()
        print(f"epoch {epoch:2d}  train loss {total / len(Xtr):.4f}  test accuracy {acc:.3f}")

Output

epoch  1  train loss 0.6226  test accuracy 0.875
epoch  5  train loss 0.1361  test accuracy 0.947
epoch 10  train loss 0.0569  test accuracy 0.978
epoch 15  train loss 0.0375  test accuracy 0.988
epoch 20  train loss 0.0305  test accuracy 0.993
epoch 25  train loss 0.0285  test accuracy 0.988
epoch 30  train loss 0.0265  test accuracy 0.990

The training loss falls from 0.62 to 0.03 and test accuracy rises from 87.5% to about 99%. Because the curved boundary needs non-linearity, a plain logistic regression would stay far below this (the XOR lesson of Week 1 in a new costume).

Piece	Why it is there
`model.train()`	dropout/batch-norm in learning mode
`opt.zero_grad()`	gradients accumulate otherwise (Chapter 3)
`.squeeze(1)`	the model outputs (N,1); the target is (N,): make them match
`loss.item()`	a plain Python float for logging (the tensor would keep the graph alive)
`model.eval()` + `no_grad()`	evaluation: no dropout, no graph

5.3Train, validation and test

A model is judged on data it has not trained on. Keep a test set untouched until the end; in real projects also hold out a validation set to choose hyper-parameters (learning rate, depth), so the test set stays an honest final exam. If train loss keeps falling while validation loss rises, the model is overfitting: memorising instead of learning (Week 4 returns with dropout and augmentation).

5.4Lecture summary

Dataset gives one sample; DataLoader gives shuffled mini-batches; one pass = one epoch.
Step: zero_grad, forward, loss, backward, step.
Evaluate with eval() + no_grad() on data never trained on.

Next · Part 6Project: Fashion-MNIST

Part 6

Project: Fashion-MNIST

Fashion-MNIST is the course's first end-to-end image project: 70,000 greyscale 28x28 pictures of clothing in 10 classes (T-shirt, trouser, pullover, dress, coat, sandal, shirt, sneaker, bag, ankle boot). It is the "hello world" of image classification, harder than the digits dataset it replaces but small enough to train in a minute.

6.1The real data pipeline

torchvision downloads the dataset and ToTensor() converts each picture to a float tensor in [0, 1] of shape (1, 28, 28): 1 channel, 28 rows, 28 columns. This block needs a download (about 30 MB), so it is shown but not run here:

main.py

# This is the real data pipeline. It downloads about 30 MB, so it is shown, not run, in these notes.
import torch
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

tfm = transforms.ToTensor()                         # PIL image -> float tensor in [0, 1], shape (1, 28, 28)
train = datasets.FashionMNIST("data", train=True,  download=True, transform=tfm)
test  = datasets.FashionMNIST("data", train=False, download=True, transform=tfm)
train_loader = DataLoader(train, batch_size=64, shuffle=True)
test_loader  = DataLoader(test,  batch_size=256)
# ...then the same loop as before: model(xb) -> CrossEntropyLoss(logits, yb) -> backward -> step

6.2The model

The course's architecture is a plain MLP. Flatten unrolls the 28x28 image into 784 numbers, two hidden ReLU layers (128, then 64 units) learn features, and the last layer gives 10 logits, one per class, with no softmax because CrossEntropyLoss applies it. The printed parameter counts follow the $n_ln_{l-1}+n_l$ rule:

main.py

import torch.nn as nn

# the architecture the course uses for Fashion-MNIST: 28x28 grey images, 10 classes
model = nn.Sequential(
    nn.Flatten(),                  # (N, 1, 28, 28) -> (N, 784)
    nn.Linear(784, 128), nn.ReLU(),
    nn.Linear(128, 64),  nn.ReLU(),
    nn.Linear(64, 10),             # 10 logits, one per class (no softmax: the loss does it)
)
print(model)
for name, p in model.named_parameters():
    print(f"{name:10s} {tuple(p.shape)!s:12s} {p.numel():>7,}")
print("total:", f"{sum(p.numel() for p in model.parameters()):,}")

Output

Sequential(
  (0): Flatten(start_dim=1, end_dim=-1)
  (1): Linear(in_features=784, out_features=128, bias=True)
  (2): ReLU()
  (3): Linear(in_features=128, out_features=64, bias=True)
  (4): ReLU()
  (5): Linear(in_features=64, out_features=10, bias=True)
)
1.weight   (128, 784)   100,352
1.bias     (128,)           128
3.weight   (64, 128)      8,192
3.bias     (64,)             64
5.weight   (10, 64)         640
5.bias     (10,)             10
total: 109,386

Notice where the parameters are: 100,352 of 109,386 (92%) sit in the very first layer, because every one of the 784 pixels connects to every one of 128 units. That is the problem the next two weeks fix with convolution: a 3x3 filter reuses 9 weights at every position instead of learning separate weights per pixel.

6.3Running the full workflow on a same-shape stand-in

So that the whole pipeline is actually exercised here, this block builds 28x28 images of three simple "garments" (horizontal bar, vertical bar, square, with noise) in the same tensor layout as Fashion-MNIST, $(N,1,28,28)$ , and trains a small MLP exactly as you would on the real data. The only thing that changes on the real dataset is where the images come from.

main.py

import torch, torch.nn as nn
from torch.utils.data import TensorDataset, DataLoader
torch.manual_seed(0)

# Stand-in for Fashion-MNIST (no download needed): 28x28 images of three simple "garments".
def make_images(n_per_class):
    imgs, labels = [], []
    for cls in range(3):
        for _ in range(n_per_class):
            img = torch.zeros(28, 28)
            if cls == 0:                                   # horizontal bar
                r = torch.randint(6, 22, (1,)).item(); img[r:r + 3, 4:24] = 1
            elif cls == 1:                                 # vertical bar
                c = torch.randint(6, 22, (1,)).item(); img[4:24, c:c + 3] = 1
            else:                                          # filled square
                r, c = torch.randint(4, 16, (2,)).tolist(); img[r:r + 8, c:c + 8] = 1
            imgs.append(img + 0.2 * torch.randn(28, 28))
            labels.append(cls)
    X = torch.stack(imgs).unsqueeze(1)                     # (N, 1, 28, 28)  - same layout as Fashion-MNIST
    return X, torch.tensor(labels)

Xtr, ytr = make_images(200); Xte, yte = make_images(60)
print("image batch shape:", tuple(Xtr.shape), " labels:", tuple(ytr.shape))
loader = DataLoader(TensorDataset(Xtr, ytr), batch_size=32, shuffle=True)

model = nn.Sequential(nn.Flatten(), nn.Linear(784, 64), nn.ReLU(), nn.Linear(64, 3))
opt = torch.optim.Adam(model.parameters(), lr=0.001)
loss_fn = nn.CrossEntropyLoss()
for epoch in range(1, 6):
    model.train()
    for xb, yb in loader:
        opt.zero_grad(); loss_fn(model(xb), yb).backward(); opt.step()
    model.eval()
    with torch.no_grad():
        pred = model(Xte).argmax(dim=1)
    print(f"epoch {epoch}: test accuracy {(pred == yte).float().mean().item():.3f}")

Output

image batch shape: (600, 1, 28, 28)  labels: (600,)
epoch 1: test accuracy 0.794
epoch 2: test accuracy 0.867
epoch 3: test accuracy 0.939
epoch 4: test accuracy 0.961
epoch 5: test accuracy 0.989

What carries over to the real dataset

Same model code, same loss (CrossEntropyLoss on logits with long labels), same loop. Expect an MLP like the one above to reach roughly 88-89% test accuracy on real Fashion-MNIST (this figure is from general experience with this architecture, not measured in these notes); the CNNs of Weeks 3-4 push it to the low 90s.

6.4Lecture summary

Images are (N, 1, 28, 28) float tensors; Flatten then Linear layers; 10 logits out.
109,386 parameters, 92% in the first layer: the motivation for CNNs.
The workflow (data, model, loss, optimiser, loop, evaluate) never changes.

Next · Part 7Project: Customer Churn

Part 7

Project: Customer Churn

Customer churn prediction is the course's industry case study: given what we know about a telecom customer, will they leave next month? It is tabular data (rows of numbers and categories, not pixels), which changes the preprocessing, and it is imbalanced (most customers stay), which changes how you must judge the model.

Data note. The real telecom dataset is not available offline, so the examples use a synthetic customer table with the same kind of columns (tenure, monthly bill, contract type, support calls) and a churn rule written to behave plausibly: short tenure, high bills, month-to-month contracts and many support calls raise churn. The numbers below are real outputs on that synthetic data; do not read them as telecom facts.

7.1Preprocessing tabular data

Categories (contract type) are numerized with one-hot encoding (Week 1): three columns, exactly one is 1.
Numbers (tenure in months, bill in rupees, call count) live on very different scales, so each column is standardised (subtract the mean, divide by the std) using the broadcasting idiom of Chapter 2. Without it the large-scale column dominates the early gradients.
Compute the mean and std on the training rows only, then reuse them for test rows, so no test information leaks into training. (The code below standardises the whole table for brevity; in a real pipeline compute the statistics on the training split.)

main.py

import numpy as np, torch
rng = np.random.default_rng(0)
n = 2000
tenure   = rng.integers(1, 72, n)                       # months with the company
monthly  = rng.normal(65, 25, n).clip(20, 120)          # monthly bill
contract = rng.choice(["month-to-month", "one-year", "two-year"], n, p=[0.55, 0.25, 0.20])
support  = rng.poisson(1.5, n)                          # support calls

# churn probability: short tenure, high bill, month-to-month and many calls push it up
z = -1.2 - 0.04 * tenure + 0.015 * monthly + 1.1 * (contract == "month-to-month") - 0.9 * (contract == "two-year") + 0.25 * support
p = 1 / (1 + np.exp(-z))
churn = (rng.random(n) < p).astype(int)
print("customers:", n, "  churn rate:", churn.mean().round(3))

# preprocessing: one-hot for the category, standardise the numbers
cats = ["month-to-month", "one-year", "two-year"]
onehot = np.stack([(contract == c).astype(float) for c in cats], 1)
nums = np.stack([tenure, monthly, support], 1).astype(float)
nums = (nums - nums.mean(0)) / nums.std(0)
X = np.hstack([nums, onehot]).astype(np.float32)
print("feature matrix:", X.shape, "  first row:", [round(float(v), 2) for v in X[0]])

Output

customers: 2000   churn rate: 0.342
feature matrix: (2000, 6)   first row: [1.19, 0.1, 1.22, 0.0, 0.0, 1.0]

7.2Training the classifier

The network is a 6-16-8-1 MLP with a single output logit, BCEWithLogitsLoss and Adam, trained on the first 1500 customers and tested on the last 500. This data is small, so each epoch uses all 1500 rows at once (full-batch). Then we count the four outcomes of the confusion matrix by hand.

main.py

import numpy as np, torch, torch.nn as nn
rng = np.random.default_rng(0)
n = 2000
tenure = rng.integers(1, 72, n); monthly = rng.normal(65, 25, n).clip(20, 120)
contract = rng.choice(["month-to-month", "one-year", "two-year"], n, p=[0.55, 0.25, 0.20]); support = rng.poisson(1.5, n)
z = -1.2 - 0.04 * tenure + 0.015 * monthly + 1.1 * (contract == "month-to-month") - 0.9 * (contract == "two-year") + 0.25 * support
churn = (rng.random(n) < 1 / (1 + np.exp(-z))).astype(int)
onehot = np.stack([(contract == c).astype(float) for c in ["month-to-month", "one-year", "two-year"]], 1)
nums = np.stack([tenure, monthly, support], 1).astype(float); nums = (nums - nums.mean(0)) / nums.std(0)
X = torch.tensor(np.hstack([nums, onehot]), dtype=torch.float32); y = torch.tensor(churn, dtype=torch.float32)

split = 1500
Xtr, ytr, Xte, yte = X[:split], y[:split], X[split:], y[split:]
torch.manual_seed(0)
model = nn.Sequential(nn.Linear(6, 16), nn.ReLU(), nn.Linear(16, 8), nn.ReLU(), nn.Linear(8, 1))
opt = torch.optim.Adam(model.parameters(), lr=0.01)
loss_fn = nn.BCEWithLogitsLoss()
for epoch in range(1, 201):
    opt.zero_grad()
    loss = loss_fn(model(Xtr).squeeze(1), ytr)
    loss.backward(); opt.step()
    if epoch in (1, 50, 100, 200):
        print(f"epoch {epoch:3d}  train loss {loss.item():.4f}")

with torch.no_grad():
    pred = (model(Xte).squeeze(1) > 0).float()
tp = ((pred == 1) & (yte == 1)).sum().item(); fp = ((pred == 1) & (yte == 0)).sum().item()
fn = ((pred == 0) & (yte == 1)).sum().item(); tn = ((pred == 0) & (yte == 0)).sum().item()
acc = (tp + tn) / len(yte); prec = tp / max(tp + fp, 1); rec = tp / max(tp + fn, 1)
print(f"confusion: TP={tp} FP={fp} FN={fn} TN={tn}")
print(f"accuracy {acc:.3f}  precision {prec:.3f}  recall {rec:.3f}  F1 {2 * prec * rec / max(prec + rec, 1e-9):.3f}")
print("baseline 'nobody churns' accuracy:", round(1 - yte.mean().item(), 3))

Output

epoch   1  train loss 0.6500
epoch  50  train loss 0.5366
epoch 100  train loss 0.5248
epoch 200  train loss 0.4964
confusion: TP=86 FP=55 FN=88 TN=271
accuracy 0.714  precision 0.610  recall 0.494  F1 0.546
baseline 'nobody churns' accuracy: 0.652

Figure 5. The test-set confusion matrix from the run above

7.3Why accuracy alone lies

Read the last line of the output. 65.2% of these test customers stay, so a "model" that predicts nobody churns already scores 65.2% accuracy. Our network scores 71.4%: better, but far less impressive than the headline number, and its recall is 49%: it finds only about half of the customers who really leave. For a business whose job is to save those customers, recall is the number that matters.

The extreme version of the trap, with 2% churn:

main.py

# 2% of customers churn. A "model" that always says "will not churn":
n, churners = 1000, 20
tp, fn = 0, churners                      # catches none of the 20
tn, fp = n - churners, 0
print("accuracy :", (tp + tn) / n)
print("recall   :", tp / (tp + fn), "  <- it never finds a churner")
print("precision: undefined (no positive predictions)")

Output

accuracy : 0.98
recall   : 0.0   <- it never finds a churner
precision: undefined (no positive predictions)

Metric	Question it answers	Use when
Accuracy	what fraction was right?	classes are balanced
Precision	of those I flagged, how many were real?	false alarms are costly
Recall	of the real ones, how many did I find?	misses are costly (churn, disease)
F1	a balance of the two	one number for imbalanced data

7.4Handling the imbalance

The rare class is the one we care about, yet the loss is dominated by the common one. Common remedies: re-weight the rare class in the loss, resample (oversample the rare class or undersample the common), and move the decision threshold below 0.5 so that borderline cases count as churn. PyTorch's pos_weight multiplies the loss on positive examples:

main.py

import torch, torch.nn as nn
logits = torch.tensor([0.0, 0.0])                 # the model is unsure about both
target = torch.tensor([1.0, 0.0])                 # one positive (rare class), one negative
plain = nn.BCEWithLogitsLoss(reduction="none")(logits, target)
weighted = nn.BCEWithLogitsLoss(reduction="none", pos_weight=torch.tensor(9.0))(logits, target)
print("plain loss per sample   :", plain.tolist())
print("pos_weight=9 per sample :", [round(v, 3) for v in weighted.tolist()], " <- missing a churner now costs 9x")

Output

plain loss per sample   : [0.6931471824645996, 0.6931471824645996]
pos_weight=9 per sample : [6.238, 0.693]  <- missing a churner now costs 9x

7.5Lecture summary

Tabular preprocessing: one-hot categories, standardise numbers (statistics from the training rows).
Same workflow as before with a single logit and BCEWithLogitsLoss.
Judge imbalanced problems with precision, recall and F1, not accuracy alone; fix with weights, resampling or a lower threshold.

Next · InterviewInterview Question Bank

Interview

Interview Question Bank

What is likely to be asked from Week 2, with 30-second answers, deeper explanations, a lecture-or-extra tag on each question, and code you can run.

Week 2 questions mix concepts (what autograd does, why zero_grad) with code you are asked to write or debug (a training loop, a shape error). Each answer starts with a 30-second version. Questions are tagged From the notes or Extra; the source notes only list the topics, so most answers here are written from them and run.

1Definition

One clear sentence: what it is.

2Intuition or tiny example

Why it works, in one picture or three lines.

3The failure it prevents

What goes wrong without it.

8.1Interview heat-map: what is most likely

Very likely8 topics

Tensor vs NumPy array"How is a PyTorch tensor different from a NumPy array?"Notes 1.1, 2.2

Broadcasting"What is broadcasting? When does it fail?"Notes 2.1

requires_grad and backward"How does autograd work?"Notes 3.1-3.2

Why zero_grad"Why do we call optimizer.zero_grad()?"Notes 3.3

The training loop"Write a PyTorch training loop."Notes 5.2

train() vs eval(), no_grad"What do model.eval() and torch.no_grad() do?"Notes 4.3

CrossEntropyLoss inputs"What does nn.CrossEntropyLoss expect?"Notes 4.4

Imbalanced classification"A model has 98% accuracy on churn. Is it good?"Notes 7.3-7.4

Likely6 topics

view vs reshape and memory sharing"What is the difference between view and reshape?"Notes 1.4

detach vs no_grad"What is the difference between detach() and no_grad()?"Notes 3.4

nn.Module structure"How do you define a model in PyTorch?"Notes 4.2

DataLoader, batch, epoch"What is a batch, an epoch, an iteration?"Notes 5.1

Device handling"How do you train on a GPU?"Notes 2.3

BCEWithLogitsLoss"Why BCEWithLogitsLoss instead of Sigmoid + BCELoss?"Notes 4.4

Sometimes3 topics

In-place operations"What goes wrong with in-place operations in autograd?"Notes 3.7

Saving models"How do you save and load a PyTorch model?"Notes 4.6

Dtype pitfalls"Why do I get dtype errors in PyTorch?"Notes 1.2

8.2Tensors and autograd

Interview 1How is a PyTorch tensor different from a NumPy array?

Week 2Very likelyNotes 1.1, 2.2From the lecture

Two superpowers NumPy lacks.

Prints

tensor([100.,   2.,   3.], dtype=torch.float64)
tensor([100.,   2.,   3.], dtype=torch.float64)  <- unchanged by the second edit
True ndarray

Answer

Say this first (30 seconds)

Both are n-dimensional arrays with similar indexing and operations. A tensor adds two things: it can live on a GPU (.to("cuda")), and it can track operations for automatic differentiation (requires_grad=True). They convert cheaply: from_numpy shares memory, torch.tensor copies, .detach().numpy() goes back.

Go deeper

Shared memory is a trap: editing one changes the other.

Follow-up they may askWhen does .numpy() fail?

Interview 2What is broadcasting, and when does it fail?

Week 2Very likelyNotes 2.1Extra

Line the shapes up from the right.

Prints

tensor([[11., 22., 33.],
        [14., 25., 36.]])
tensor([[101., 102., 103.],
        [204., 205., 206.]])
torch.Size([2, 3])
True [1.0, 1.0]
RuntimeError: The size of tensor a (2) must match the size of tensor b (4) at non-singleton dimension 0

Answer

Say this first (30 seconds)

It lets operations work on tensors of different shapes without copying: shapes are aligned from the right, and in every position the sizes must be equal or one of them 1; the size-1 side is stretched. It fails when two sizes differ and neither is 1 (2 vs 4). The idiom: (X - X.mean(0)) / X.std(0) standardises every column.

Go deeper

Danger: a (N,) vs (N,1) mix silently becomes (N,N). Print shapes when a loss looks odd.

Follow-up they may askWhat shape is (5,1) + (1,4)?

Interview 3What is the difference between view and reshape?

Week 2LikelyNotes 1.4Extra

Think about memory.

Prints

torch.Size([3, 4]) torch.Size([2, 6]) torch.Size([2, 6])
transpose : torch.Size([4, 3])  permute: torch.Size([4, 1, 3])
unsqueeze : torch.Size([1, 3, 4])   squeeze: torch.Size([3, 4])
flatten   : torch.Size([32, 784])
view shares storage, x[0] became: 99

Answer

Say this first (30 seconds)

view never copies: it returns a tensor sharing memory with the original, and requires the data to be laid out contiguously. reshape returns a view when it can and a copy when it must. So after v = x.view(...), editing v edits x. When unsure, use reshape (or .contiguous().view).

Go deeper

Transposes are not contiguous: x.T.view(-1) fails, x.T.reshape(-1) works.

Follow-up they may askHow would you flatten an image batch for a linear layer?

Interview 4Why do you get dtype errors, and what are the usual types?

Week 2SometimesNotes 1.2Extra

Features float32, class labels int64.

Prints

torch.int64 torch.float32
torch.float32
torch.float32 torch.int64 torch.float16
tensor([ 1, -1])
RuntimeError: dot : expected both vectors to have same dtype, but found Float and Lo

Answer

Say this first (30 seconds)

Operations need compatible types: float32 @ int64 raises an error. Rules of thumb: inputs and weights are float32; class labels for CrossEntropyLoss are int64 (long); targets for MSELoss and BCEWithLogitsLoss are float. Convert with .float()/.long(); float to int truncates toward zero.

Go deeper

NumPy defaults to float64; torch.from_numpy keeps it, which can mismatch float32 weights.

Follow-up they may askWhat is float16 used for?

Interview 5How does PyTorch autograd work?

Week 2Very likelyNotes 3.1-3.2Extra

It records, then replays in reverse.

Prints

loss = 9.0
dloss/dw = -18.0   (2 (y-10) x = 2*(-3)*3 = -18)
dloss/db = -6.0   (2 (y-10)   = -6)
x has no grad: None

Prints (non-scalar)

RuntimeError: grad can be implicitly created only for scalar outputs
grad of sum(x^2): [2.0, 4.0, 6.0]  (2x)

Answer

Say this first (30 seconds)

Tensors with requires_grad=True make every operation on them be recorded in a graph (each result stores its grad_fn). loss.backward() walks that graph in reverse applying the chain rule and stores $\partial L/\partial w$ in each leaf's .grad. It is the same chain rule as hand backprop; it gave identical numbers to the manual formula in the example. After backward() the graph is freed.

Go deeper

Only a scalar can be backpropagated directly; reduce a vector with .sum() or .mean() first.

Follow-up they may askWhy does a second backward() raise an error?

Interview 6Why do we call optimizer.zero_grad() every step?

Week 2Very likelyNotes 3.3Extra

What does backward() do to .grad?

Prints

step 0: w.grad = 18.0
step 1: w.grad = 36.0
step 2: w.grad = 54.0
--- now zero the gradient every step ---
step 0: w.grad = 18.0
step 1: w.grad = 18.0
step 2: w.grad = 18.0

Prints

with    zero_grad: [1.8, 2.52, 2.81, 2.92, 2.97, 2.99, 3.0, 3.0, 3.0, 3.0]
without zero_grad: [1.8, 4.32, 6.05, 5.95, 4.08, 1.56, -0.09, 0.11, 2.05, 4.55]

Answer

Say this first (30 seconds)

backward() adds to .grad instead of overwriting it. Without clearing, each step uses the sum of all previous gradients: in the example the same loss gave gradients 18, 36, 54. Accumulation is deliberate (it lets you sum gradients over several small batches to simulate a large batch), but in a normal loop you must clear first, or the update is not the gradient of anything meaningful.

Go deeper

On $(w-3)^2$ with $\eta=0.3$ the correct loop settles at 3; the forgetful one swings between -0.09 and 6.05.

Follow-up they may askHow do you simulate a batch of 256 on a GPU that fits 64?

Interview 7What is the difference between detach() and torch.no_grad()?

Week 2LikelyNotes 3.4Extra

One acts on a tensor, one on a block of code.

Prints

inside no_grad: False
normal        : True
detached      : False
RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach
detach().numpy(): [2. 4.]

Answer

Say this first (30 seconds)

no_grad() is a context: nothing computed inside is recorded. detach() is a method on one tensor: it returns the same values cut from the graph (sharing memory), typically before .numpy(), plotting or logging.

Go deeper

Detaching a target inside a loss stops gradients flowing into it (used in some RL and distillation setups).

Follow-up they may askWhy does y.numpy() raise on a tracked tensor?

Interview 8What can go wrong with in-place operations (+=, relu_()) under autograd?

Week 2SometimesNotes 3.7Extra

Backward may need the old value.

Prints

RuntimeError: one of the variables needed for gradient computation has been modified by an i
out-of-place grad: [2.7182817459106445, 7.389056205749512]

Answer

Say this first (30 seconds)

Backward may need a value that the forward pass computed; editing that tensor in place overwrites it, and PyTorch raises "one of the variables needed for gradient computation has been modified by an inplace operation". Prefer out-of-place (y = y + 1). In-place edits on leaf tensors that require grad are not allowed at all.

Go deeper

Inside torch.no_grad() (e.g. weight updates) in-place edits are fine.

Follow-up they may askHow does an optimiser update weights in place without breaking autograd?

8.3Models, losses and the training loop

Interview 9How do you define a model in PyTorch?

Week 2LikelyNotes 4.2Extra

A class with two methods.

main.py

class MLP(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(4, 8)
        self.fc2 = nn.Linear(8, 3)
    def forward(self, x):
        return self.fc2(torch.relu(self.fc1(x)))

Prints

parameters: 67
trainable : 67
output    : (5, 3)
['fc1.weight', 'fc1.bias', 'fc2.weight', 'fc2.bias']

Answer

Say this first (30 seconds)

Subclass nn.Module. In __init__ create the layers and assign them to self (that is what registers their parameters); in forward write the computation. You do not write backward: autograd derives it. Call the model like a function, model(x), never model.forward(x) directly. nn.Sequential is a shortcut for a plain chain.

Go deeper

Layers kept in a plain Python list are NOT registered; use nn.ModuleList.

Follow-up they may askHow do you count a model's parameters?

Interview 10What does nn.CrossEntropyLoss expect, and what is the common mistake?

Week 2Very likelyNotes 4.4Extra

Logits and class indices.

Prints

CrossEntropyLoss: 0.3143
by hand         : 0.3143
with softmax first (wrong): 0.7425
BCEWithLogits: 0.2201
MSELoss      : 0.125

Prints (float labels)

ok: 1.234
RuntimeError: expected target dtype to be Long or Byte, but got Float

Answer

Say this first (30 seconds)

Raw logits of shape (N, C) and integer class indices (long) of shape (N,). It applies log-softmax itself, in a stable way. Mistakes: applying softmax first (it still runs but computes a different number: 0.743 instead of 0.314 in the example), or passing float or one-hot labels.

Go deeper

For a binary output use BCEWithLogitsLoss with one logit and float targets.

Follow-up they may askWhy is BCEWithLogitsLoss better than sigmoid then BCELoss?

Interview 11Why prefer BCEWithLogitsLoss to a sigmoid followed by BCELoss?

Week 2LikelyNotes 4.4Extra

Think of $\log(\sigma(z))$ for very negative $z$ .

Prints

plain loss per sample   : [0.6931471824645996, 0.6931471824645996]
pos_weight=9 per sample : [6.238, 0.693]  <- missing a churner now costs 9x

Answer

Say this first (30 seconds)

It fuses the sigmoid and the log into one numerically stable expression (the log-sum-exp trick). Doing them separately makes $\sigma(z)$ round to exactly 0 or 1 for large $|z|$ and then $\log 0=-\infty$ gives nan. It also takes pos_weight for imbalanced data.

Go deeper

Consequence: the model's last layer outputs logits and has no activation; apply torch.sigmoid only when you want probabilities.

Follow-up they may askHow do you turn logits into class predictions?

Interview 12Write the PyTorch training loop and explain each line.

Week 2Very likelyNotes 5.2Extra

Five lines inside the batch loop.

main.py

for epoch in range(epochs):
    model.train()
    for xb, yb in loader:
        xb, yb = xb.to(device), yb.to(device)
        opt.zero_grad()                 # 1 clear
        loss = loss_fn(model(xb), yb)   # 2 forward, 3 loss
        loss.backward()                 # 4 backward
        opt.step()                      # 5 update
    model.eval()
    with torch.no_grad():
        ...evaluate on validation data...

Prints (two moons)

epoch  1  train loss 0.6226  test accuracy 0.875
epoch  5  train loss 0.1361  test accuracy 0.947
epoch 10  train loss 0.0569  test accuracy 0.978
epoch 15  train loss 0.0375  test accuracy 0.988
epoch 20  train loss 0.0305  test accuracy 0.993
epoch 25  train loss 0.0285  test accuracy 0.988
epoch 30  train loss 0.0265  test accuracy 0.990

Answer

Say this first (30 seconds)

For each batch: zero_grad (clear old gradients), forward (predictions), loss, backward (fill every .grad), step (update the weights). Around it: model.train() before training, model.eval() and torch.no_grad() for evaluation, .to(device) for each batch.

Go deeper

The moons example went from 87.5% to 99% test accuracy in 30 epochs with this loop.

Follow-up they may askWhere do the validation and test sets enter?

Interview 13What do model.eval() and torch.no_grad() do, and are they the same?

Week 2Very likelyNotes 4.3Extra

Two different switches.

Prints

train mode : [0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 2.0, 2.0]
eval mode  : [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
model.training flag: False

Prints

with grad tracking : True   (keeps the graph in memory)
under no_grad      : False   (no graph: use this for evaluation / inference)

Answer

Say this first (30 seconds)

Not the same. model.eval() changes the behaviour of layers: dropout turns off and batch norm uses stored statistics. torch.no_grad() stops recording the graph, saving memory and time. For evaluation you use both.

Go deeper

Forgetting eval() with dropout gives noisy, lower test accuracy.

Follow-up they may askWhy does PyTorch dropout scale the survivors by

1/(1-p)

Interview 14What are a batch, an epoch and an iteration?

Week 2LikelyNotes 5.1Extra

1000 samples, batch 64.

Prints

dataset size: 1000  one item: [torch.Size([2]), torch.Size([])]
one batch: (64, 2) (64,)  batches per epoch: 16

Answer

Say this first (30 seconds)

A batch is the group of samples used for one weight update. An iteration is one such update. An epoch is one full pass over the dataset. With 1000 samples and batch 64 there are $\lceil1000/64\rceil=16$ iterations per epoch. DataLoader(shuffle=True) reshuffles every epoch.

Go deeper

Do not shuffle the test loader: it does not matter, and fixed order helps debugging.

Follow-up they may askWhy not train on the whole dataset in one batch?

Interview 15How do you train on a GPU?

Week 2LikelyNotes 2.3Extra

Model and data on the same device.

Prints

training on: cpu
tensor device: cpu
output device: cpu
(this machine has no GPU, so everything here is CPU; on a GPU machine model(x.cpu()) would raise a device-mismatch error)

Answer

Say this first (30 seconds)

Choose the device once, device = "cuda" if torch.cuda.is_available() else "cpu", then move the model (model.to(device)) and every batch (xb.to(device)). A GPU runs thousands of multiply-adds in parallel, which is what matrix products need. If model and data are on different devices you get "Expected all tensors to be on the same device".

Go deeper

Move the model before creating the optimiser.

Follow-up they may askWhy is it slow to move tensors back and forth every step?

Interview 16How do you save and load a PyTorch model?

Week 2SometimesNotes 4.6Extra

Weights, not the object.

Prints

before load equal: False
after  load equal: True
state_dict keys  : ['weight', 'bias']

Answer

Say this first (30 seconds)

torch.save(model.state_dict(), path) saves only the parameter tensors. To load, build the same architecture and call model.load_state_dict(torch.load(path)), then model.eval() for inference. Saving the whole model object with pickle is fragile: it breaks when your code changes.

Go deeper

For resuming training also save the optimiser's state_dict and the epoch.

Follow-up they may askWhat does map_location do in torch.load?

8.4Case study

Interview 17A churn model has 98% accuracy. Is it good?

Week 2Very likelyNotes 7.3-7.4Extra

How many customers churn?

Prints

accuracy : 0.98
recall   : 0.0   <- it never finds a churner
precision: undefined (no positive predictions)

Prints (the confusion matrix and metrics)

confusion: TP=86 FP=55 FN=88 TN=271
accuracy 0.714  precision 0.610  recall 0.494  F1 0.546
baseline 'nobody churns' accuracy: 0.652

Answer

Say this first (30 seconds)

Not necessarily. If only 2% of customers churn, predicting "nobody churns" already scores 98% and finds 0 churners (recall 0). Report precision, recall and F1 and read the confusion matrix. Fixes: class weights (pos_weight), resampling, a lower decision threshold.

Go deeper

In our synthetic run the baseline was 65.2% and the network 71.4% accuracy but only 49% recall.

Follow-up they may askPrecision or recall: which matters more for churn, and for a spam filter?

Next · SummaryWeek 2 Summary Sheet

Summary

Week 2 Summary Sheet

Week 2 on one page: a chapter-by-chapter glance, key terms, the pairs to tell apart, the error messages you can now read, a checklist and notes on the source.

9.1Week 2 at a glance

C1Tensors

Typed n-d arrays: .shape .dtype .device. * element-wise, @ matrix. view shares memory. Labels long, data float32.

C2Broadcasting, bridge, GPU

Shapes align from the right, sizes equal or 1. from_numpy shares, tensor copies. One device for model and data.

C3Autograd

requires_grad + backward(); graph freed after; gradients accumulate; no_grad, detach.

C4nn building blocks

nn.Module: layers in __init__, forward. Losses take logits. train()/eval(). Save state_dict.

C5Data and the loop

Dataset -> DataLoader -> model -> loss -> optimiser. zero_grad, forward, loss, backward, step. Evaluate on unseen data.

C6Fashion-MNIST

Images (N,1,28,28); Flatten + 784-128-64-10 = 109,386 params, 92% in layer 1 (motivates CNNs).

C7Customer churn

Tabular: one-hot + standardise. One logit + BCEWithLogitsLoss. Imbalance: use precision/recall/F1, weights, threshold.

9.2Key terms

C1-C2

tensor: n-d array that can use a GPU and track gradients.
broadcasting: stretching size-1 dimensions to match.
device: cpu or cuda.

C3-C4

leaf: a tensor you created that tracks gradients.
logits: raw scores before sigmoid/softmax.
state_dict: the model parameters as a dictionary.
inverted dropout: survivors scaled by 1/(1-p) in training.

C5-C7

epoch: one pass over the dataset.
batch: samples per update.
confusion matrix: TP, FP, FN, TN counts.
recall: found / all real positives.

9.3The pairs you must tell apart

*element-wise

@matrix product

from_numpyshares memory

torch.tensorcopies

viewno copy, contiguous only

reshapeview or copy

model.eval()layer behaviour

no_grad()no graph recorded

detach()one tensor

no_grad()a block of code

CrossEntropyLosslogits + long labels

BCEWithLogitsLoss1 logit + float labels

accuracybalanced classes

recall / F1imbalanced classes

train setfit the weights

test setuntouched until the end

9.4Error messages you can now read

Message	Cause	Fix
`mat1 and mat2 shapes cannot be multiplied (5x3 and 4x2)`	layer expects 4 features, got 3	print the shape; fix `in_features` or the data
`The size of tensor a (2) must match the size of tensor b (4) at non-singleton dimension 0`	broadcasting impossible	align the shapes (unsqueeze / reshape)
`expected target dtype to be Long or Byte, but got Float`	float class labels for CrossEntropy	`y.long()`
`Trying to backward through the graph a second time`	graph already freed	recompute the forward pass, or `retain_graph=True`
`grad can be implicitly created only for scalar outputs`	backward() on a vector	`.sum()` / `.mean()`
`Can't call numpy() on Tensor that requires grad`	tracked tensor	`.detach().numpy()`
`one of the variables needed for gradient computation has been modified by an inplace operation`	in-place edit	use `y = y + 1`
`Expected all tensors to be on the same device`	model on cuda, data on cpu (or reverse)	`.to(device)`

9.5Last-minute checklist

C1-C4

check shape, dtype, device first
labels long, features float32
zero_grad every step
losses take logits

five lines: zero, forward, loss, backward, step
eval() + no_grad() to evaluate
never tune on the test set
epoch = pass, batch = update group

C6-C7

(N,1,28,28) image layout
one-hot + standardise (train statistics)
imbalance: recall, F1, weights
baseline accuracy first

9.6Notes on the source

Where	The source says	Better
Whole week	PyTorch topics listed (tensors, operations, GPU, workflow, Fashion-MNIST, churn)	no code in the source: everything here was written and run on PyTorch 2.x
Dropout	inference scales by $1-p$ (Week 3-4 notes)	PyTorch uses inverted dropout: survivors scaled by $1/(1-p)$ in training, nothing at inference (measured: survivors became 2.0 at $p=0.5$ )
Fashion-MNIST	dataset download	needs the internet; a synthetic stand-in with the same tensor layout was run instead
Customer churn	telecom dataset	synthetic data of the same shape; the numbers are real outputs but not telecom facts

Revision overview

Tensorsshape, dtype, device
broadcasting
view vs reshape

Autogradrequires_grad
backward, accumulate
no_grad, detach

nnModule, Linear
losses on logits
optimisers

Week 2PyTorch toolkit

Loopzero, forward, loss
backward, step
train / eval

Images(N,1,28,28)
Flatten + MLP
109,386 params

Tabularone-hot, standardise
imbalance
precision / recall

Revision: all of Week 2 on one page

Deep Learning,Week 2

Chapters

Tensors

1.1A tensor is an n-dimensional array

1.2dtype: the number type matters

1.3Indexing and slicing

1.4Reshaping: same numbers, new shape

1.5Arithmetic: element-wise vs matrix

1.6Lecture summary

Broadcasting, NumPy and the GPU

2.1Broadcasting

2.2The NumPy bridge

2.3Devices: CPU and GPU

2.4Lecture summary

Autograd

3.1requires_grad and backward()

3.2The graph, and why it vanishes after backward()

3.3Gradients accumulate: the reason for zero_grad()

3.4Turning tracking off: no_grad and detach

3.5backward() needs a scalar

3.6Autograd equals your hand-derived gradient

3.7A trap: in-place operations

3.8Lecture summary

Modules, Losses and Optimisers

4.1nn.Linear: one layer is two tensors

4.2nn.Module: your network is a class

4.3train() and eval()

4.4Loss functions

4.5Optimisers

4.6Saving and loading

4.7Lecture summary

Data and the Training Loop

5.1The data pipeline: Dataset and DataLoader

5.2The five-line training step

5.3Train, validation and test

5.4Lecture summary

Project: Fashion-MNIST

6.1The real data pipeline

6.2The model

6.3Running the full workflow on a same-shape stand-in

6.4Lecture summary

Project: Customer Churn

7.1Preprocessing tabular data

7.2Training the classifier

7.3Why accuracy alone lies

7.4Handling the imbalance

7.5Lecture summary

Interview Question Bank

8.1Interview heat-map: what is most likely

8.2Tensors and autograd

Go deeper

Go deeper

Go deeper

Go deeper

Go deeper

Go deeper

Go deeper

Go deeper

8.3Models, losses and the training loop

Go deeper

Go deeper

Go deeper

Go deeper

Go deeper

Go deeper

Go deeper

Go deeper

8.4Case study

Go deeper

Week 2 Summary Sheet

9.1Week 2 at a glance

9.2Key terms

9.3The pairs you must tell apart

9.4Error messages you can now read

9.5Last-minute checklist

9.6Notes on the source

Deep Learning,
Week 2