Deep Learning,
Week 7

Images are grids; text, speech, music, stock prices and sensor logs are sequences: ordered lists where the order carries the meaning. "Dog bites man" and "man bites dog" use the same three words and mean opposite things. Weeks 7 and 8 build networks for this kind of data, and this chapter explains why the networks of earlier weeks are the wrong tool.

1.1Three kinds of sequence task

• Sequence classification: a whole sequence in, one label out (is this email spam? is this review positive?).
• Sequence generation: produce a sequence one element at a time (continue a sentence, compose a melody).
• Sequence to sequence: a sequence in, a sequence out, possibly of a different length (translate English into Hindi; Week 8).

1.2Why an MLP or a CNN is the wrong tool

• Fixed-size input. An MLP has one input neuron per position, so sequences must all have the same length. Sentences do not.
• No sharing across positions. The word "cat" at position 1 and at position 100 meets completely different weights, so everything must be learned separately at every position.
• No memory. The output depends only on the current input window, with no running summary of the past.

A CNN (Week 3) does share weights and can slide over text, but its view is limited to a small window and it has no state carried forward. The three failures show up directly in numbers:

import torch, torch.nn as nn
torch.manual_seed(0)
vocab = {"dog": 0, "bites": 1, "man": 2}
a, b = ["dog", "bites", "man"], ["man", "bites", "dog"]
bow = lambda s: torch.bincount(torch.tensor([vocab[w] for w in s]), minlength=3)        # bag of words: counts only
print("bag of words  'dog bites man':", bow(a).tolist(), "  'man bites dog':", bow(b).tolist(), " <- identical, order is lost")

emb = nn.Embedding(3, 4)
rnn = nn.RNN(4, 6, batch_first=True)
last = lambda s: rnn(emb(torch.tensor([[vocab[w] for w in s]])))[0][0, -1]
print("RNN final hidden state, same words in a different order differ by:", round((last(a) - last(b)).norm().item(), 3))

# parameters needed to read a sequence of length T with d features per step into h hidden units
d, h = 50, 100
print("\nlength T   fixed-input MLP weights (T*d*h)   RNN weights (d*h + h*h)")
for T in (10, 100, 1000):
    print(f"{T:8d}   {T * d * h:>28,}   {d * h + h * h:>22,}")
print("the MLP needs a fixed T and a separate weight for 'cat' at every position; the RNN reuses one set of weights at every step")

bag of words  'dog bites man': [1, 1, 1]   'man bites dog': [1, 1, 1]  <- identical, order is lost
RNN final hidden state, same words in a different order differ by: 0.997

length T   fixed-input MLP weights (T*d*h)   RNN weights (d*h + h*h)
      10                         50,000                   15,000
     100                        500,000                   15,000
    1000                      5,000,000                   15,000
the MLP needs a fixed T and a separate weight for 'cat' at every position; the RNN reuses one set of weights at every step

A bag of words (counting words, ignoring order) gives [1, 1, 1] for both sentences: all information about who bit whom is gone. A recurrent network reading them in order ends in different final states (they differ by 0.997). And for a sequence of length 1000 with 50 features per step and 100 hidden units, a fixed-input MLP needs 5 million weights against the recurrent network's 15,000, which also does not change when the sequence gets longer. That is the idea of this week: read the sequence one step at a time with the same weights, carrying a memory forward.

1.3Chapter summary

• Sequence data: order matters. Tasks: classification, generation, sequence to sequence.
• MLPs/CNNs: fixed input size, no sharing across positions (MLP), no memory.
• Measured: bag of words cannot tell the two sentences apart; a fixed MLP needs 5,000,000 weights where an RNN needs 15,000.

The most basic sequence task is language modelling: give a sentence a probability, or, equivalently, predict the next word. It is the task behind autocomplete, speech recognition, translation, and (Weeks 9-12) every large language model.

2.1The chain rule

This is exact, not an approximation: the probability of a sentence is the product of "probability of each word given everything before it". A language model is therefore any machine that outputs $P(x_t\mid x_{\lt t})$, a distribution over the vocabulary. The difficulty is the conditioning on the whole history.

2.2The Markov assumption and n-grams

The classical shortcut is the Markov assumption: only the last $n-1$ words matter. A bigram model uses $P(x_t\mid x_{t-1})$, a trigram $P(x_t\mid x_{t-2},x_{t-1})$. The probabilities are just counts: $P(w\mid h)=\text{count}(h,w)/\text{count}(h)$. Build all three from ten tiny sentences and test on three unseen ones:

import math
from collections import Counter
train = ["the cat sat on the mat", "the dog sat on the rug", "the cat saw the dog", "the dog saw the cat",
         "a cat sat on the rug", "the dog ran to the park", "the cat ran to the mat", "a dog sat on the mat",
         "the cat sat on the rug", "the dog ran on the mat"]
test = ["the dog sat on the mat", "the cat ran to the park", "a cat saw a dog"]
V = sorted({w for s in train + test for w in s.split()} | {"</s>"})
pad = lambda s, n: ["<s>"] * (n - 1) + s.split() + ["</s>"]          # n-1 start symbols, then the words, then an end symbol

def fit(n):
    ctx, full = Counter(), Counter()
    for s in train:
        t = pad(s, n)
        for i in range(n - 1, len(t)):
            ctx[tuple(t[i - n + 1:i])] += 1; full[tuple(t[i - n + 1:i + 1])] += 1
    return ctx, full

def logprob(sent, n, ctx, full, smooth):
    t, lp = pad(sent, n), 0.0
    for i in range(n - 1, len(t)):                                   # P(word | previous n-1 words) = count(history+word) / count(history)
        c = tuple(t[i - n + 1:i])
        num = full[c + (t[i],)] + (1 if smooth else 0); den = ctx[c] + (len(V) if smooth else 0)
        if num == 0: return float("-inf")
        lp += math.log(num / den)
    return lp

n_pred = sum(len(s.split()) + 1 for s in test)                         # predicted tokens in the test set (words + end symbol)
for n, name in ((1, "unigram"), (2, "bigram "), (3, "trigram")):
    ctx, full = fit(n)
    smooth = [logprob(s, n, ctx, full, True) for s in test]
    raw = [math.exp(logprob(s, n, ctx, full, False)) for s in test]
    print(f"{name}: P('the dog sat on the mat') = {math.exp(smooth[0]):.2e}  | perplexity on the 3 test sentences {math.exp(-sum(smooth) / n_pred):5.1f} (add-one smoothing)  | without smoothing the 3 sentences get {[round(r, 5) for r in raw]}")
print("vocabulary", len(V), "-> possible bigrams", len(V) ** 2, ", trigrams", len(V) ** 3, "; the corpus has only", sum(len(s.split()) + 1 for s in train), "word positions")

unigram: P('the dog sat on the mat') = 2.55e-07  | perplexity on the 3 test sentences  11.8 (add-one smoothing)  | without smoothing the 3 sentences get [0.0, 0.0, 0.0]
bigram : P('the dog sat on the mat') = 6.97e-05  | perplexity on the 3 test sentences   5.9 (add-one smoothing)  | without smoothing the 3 sentences get [0.01646, 0.00137, 0.0]
trigram: P('the dog sat on the mat') = 4.27e-05  | perplexity on the 3 test sentences   6.3 (add-one smoothing)  | without smoothing the 3 sentences get [0.04, 0.04, 0.0]
vocabulary 13 -> possible bigrams 169 , trigrams 2197 ; the corpus has only 68 word positions

Read it in three steps. (1) Looking at the previous word helps a lot: perplexity falls from 11.8 (unigram, no context) to 5.9 (bigram). (2) Looking at two previous words does not help here: trigram perplexity is 6.3, slightly worse, because there are 2,197 possible trigrams and only 68 word positions of training text. (3) Without smoothing, the unseen combination in "a cat saw a dog" makes every n-gram model give the whole sentence probability exactly 0, and that is why add-one (or better) smoothing is essential. This is the sparsity problem: the number of possible contexts grows as $V^{n-1}$ while the data stays finite, so n-grams cannot look far back, and they treat "cat" and "dog" as unrelated symbols. A neural language model fixes both: it generalises across similar words and can carry a long context in a hidden state.

2.3Perplexity

How good is a language model? Not by accuracy (predicting the exact next word is hopeless for a good model), but by how much probability it gave the words that actually came next. The standard measure is perplexity:

import math, torch
# perplexity = exp(average negative log-likelihood) = exp(cross-entropy)
p_correct = [0.1] * 8                                   # the model gives the true next word probability 0.1 at each of 8 steps
nll = -sum(math.log(p) for p in p_correct) / len(p_correct)
print("average -log P =", round(nll, 3), "-> perplexity", round(math.exp(nll), 2), "(as unsure as a uniform choice among 10 words)")

for k in (2, 10, 1000):
    logits = torch.zeros(5, k)                          # a model that spreads probability evenly over k words
    targets = torch.randint(0, k, (5,))
    ce = torch.nn.functional.cross_entropy(logits, targets)
    print(f"uniform over {k:4d} words: cross-entropy {ce.item():.4f}, exp = {math.exp(ce.item()):.1f}")

# a better model gives the right words more probability
logits = torch.zeros(5, 10); targets = torch.arange(5)
for strength in (0.0, 1.0, 3.0, 6.0):
    l = logits.clone(); l[torch.arange(5), targets] = strength
    print(f"logit boost {strength:3.1f} on the correct word: perplexity {math.exp(torch.nn.functional.cross_entropy(l, targets).item()):.2f}")
print("lower perplexity = better; perplexity 1 would mean the model is certain of every word")

average -log P = 2.303 -> perplexity 10.0 (as unsure as a uniform choice among 10 words)
uniform over    2 words: cross-entropy 0.6931, exp = 2.0
uniform over   10 words: cross-entropy 2.3026, exp = 10.0
uniform over 1000 words: cross-entropy 6.9078, exp = 1000.0
logit boost 0.0 on the correct word: perplexity 10.00
logit boost 1.0 on the correct word: perplexity 4.31
logit boost 3.0 on the correct word: perplexity 1.45
logit boost 6.0 on the correct word: perplexity 1.02
lower perplexity = better; perplexity 1 would mean the model is certain of every word

Read perplexity as the effective number of equally likely choices the model is torn between: a model that gives each correct word probability 0.1 has perplexity 10 ("as uncertain as a uniform choice among 10 words"). The model that spreads probability evenly over $k$ words has perplexity exactly $k$ (2, 10, 1000 above). As the correct word gets more probability the perplexity falls to 1.02. Lower is better, and 1 is the floor. Because PyTorch's cross_entropy is exactly $-\log P$ averaged over positions, perplexity is simply exp(loss).

2.4Chapter summary

• Chain rule: $P(x_{1:T})=\prod_tP(x_t\mid x_{\lt t})$, exact. Markov/n-gram: keep only the last $n-1$ words.
• n-grams hit sparsity: 2,197 possible trigrams against 68 training positions; unsmoothed unseen n-grams give probability 0. Measured perplexity unigram 11.8, bigram 5.9, trigram 6.3.
• Perplexity $=\exp(\text{cross-entropy})$ = effective number of equally likely choices; lower is better.

A recurrent neural network (RNN) answers both problems of the last chapter with one idea: keep a vector $h_t$, the hidden state, that is the network's memory of everything read so far, and update it with the same function at every step.

3.1The equations

• $W_{xh}$ (input to hidden): how much the current input matters.
• $W_{hh}$ (hidden to hidden): how much the past memory matters. This feedback loop is what makes the network recurrent.
• $W_{hy}$ (hidden to output): turns the memory into a prediction, e.g. a distribution over the next word.
• The same $W_{xh},W_{hh},W_{hy}$ are used at every time step: parameter sharing across time.

3.2Unfolding, and a check against PyTorch

Unfold the loop in time and an RNN looks like a very deep feed-forward network with one layer per time step, all layers sharing the same weights. Computing it costs $O(T)$ for a sequence of length $T$. Write the loop yourself and compare it with nn.RNN:

import torch, torch.nn as nn
torch.manual_seed(0)
d, h, T = 3, 4, 5
rnn = nn.RNN(d, h, batch_first=True)                                  # tanh RNN
x = torch.randn(1, T, d)
Wxh, Whh = rnn.weight_ih_l0, rnn.weight_hh_l0
bh = rnn.bias_ih_l0 + rnn.bias_hh_l0

hs, h_t = [], torch.zeros(h)
for t in range(T):                                                    # h_t = tanh(Wxh x_t + Whh h_{t-1} + b): the SAME weights every step
    h_t = torch.tanh(Wxh @ x[0, t] + Whh @ h_t + bh)
    hs.append(h_t)
by_hand = torch.stack(hs)
out, h_last = rnn(x)
print("hand-written loop matches nn.RNN:", torch.allclose(by_hand, out[0], atol=1e-6))
print("x:", tuple(x.shape), " hidden states:", tuple(out.shape), " Wxh:", tuple(Wxh.shape), " Whh:", tuple(Whh.shape))
n = sum(p.numel() for p in rnn.parameters())
print("parameters:", n, "= h*(d+h) + 2h =", h * (d + h) + 2 * h, "  (independent of the sequence length T =", T, ")")
for T2 in (5, 50, 500):
    print(f"  T = {T2:3d}: the same network reads it, output {tuple(rnn(torch.randn(1, T2, d))[0].shape)}")

hand-written loop matches nn.RNN: True
x: (1, 5, 3)  hidden states: (1, 5, 4)  Wxh: (4, 3)  Whh: (4, 4)
parameters: 36 = h*(d+h) + 2h = 36   (independent of the sequence length T = 5 )
  T =   5: the same network reads it, output (1, 5, 4)
  T =  50: the same network reads it, output (1, 50, 4)
  T = 500: the same network reads it, output (1, 500, 4)

The hand-written loop reproduces PyTorch exactly. The parameter count is $h(d+h)+2h$ (the two bias vectors are how PyTorch splits $b_h$) and does not depend on the length of the sequence: the very same 36 numbers read 5 steps or 500.

3.3Training a language model with an RNN

To turn this into a language model, make $y_t$ a distribution over the next character or word, and at every position score it with cross-entropy against the actual next token. Below, a small character-level LSTM (the gated RNN of Chapter 5; the training loop is identical for a plain RNN) learns a toy language of sentences like "the cat chased the fish." The toy language is built so that its best possible perplexity can be computed exactly: each sentence hides three independent 3-way choices, so no model can do better than $\exp(3\ln3/\text{avg length})=1.152$.

import math, random, torch, torch.nn as nn
torch.manual_seed(0); random.seed(0); torch.set_num_threads(1)
# a toy language: "the <animal> <verb> the <thing>."
animals, verbs, things = ["cat", "dog", "bird"], ["saw", "chased", "ate"], ["fish", "ball", "bone"]
def sentence(): return f"the {random.choice(animals)} {random.choice(verbs)} the {random.choice(things)}. "
text = "".join(sentence() for _ in range(1500))
chars = sorted(set(text)); idx = {c: i for i, c in enumerate(chars)}
data = torch.tensor([idx[c] for c in text])
V = len(chars); print("characters:", V, " corpus length:", len(text))
avg_len = len(text) / 1500
print(f"each sentence hides 3 independent 3-way choices = {3 * math.log(3):.2f} nats over {avg_len:.1f} characters, so NO model can beat perplexity {math.exp(3 * math.log(3) / avg_len):.3f}")

class CharLM(nn.Module):
    def __init__(self):
        super().__init__()
        self.emb = nn.Embedding(V, 16); self.lstm = nn.LSTM(16, 64, batch_first=True); self.out = nn.Linear(64, V)
    def forward(self, x, state=None):
        o, state = self.lstm(self.emb(x), state); return self.out(o), state
model = CharLM(); opt = torch.optim.Adam(model.parameters(), lr=3e-3)
L = 40
def batch(B=64):
    starts = torch.randint(0, len(data) - L - 1, (B,))
    return torch.stack([data[s:s + L] for s in starts]), torch.stack([data[s + 1:s + L + 1] for s in starts])
def perplexity():
    with torch.no_grad():
        x, y = batch(256); return math.exp(nn.functional.cross_entropy(model(x)[0].reshape(-1, V), y.reshape(-1)).item())
print(f"before training: perplexity {perplexity():.1f}  (a uniform guess over {V} characters would be {V})")
for step in range(1, 401):
    x, y = batch()
    loss = nn.functional.cross_entropy(model(x)[0].reshape(-1, V), y.reshape(-1))
    opt.zero_grad(); loss.backward(); nn.utils.clip_grad_norm_(model.parameters(), 1.0); opt.step()
    if step in (25, 100, 400): print(f"step {step:3d}: perplexity {perplexity():.2f}")

# generate: feed the sampled character back in (this is inference: no teacher forcing)
model.eval(); prime, state = ". the ", None
for ch in prime[:-1]: _, state = model(torch.tensor([[idx[ch]]]), state)          # read the prompt
x, out = torch.tensor([[idx[prime[-1]]]]), prime
for _ in range(110):
    logits, state = model(x, state)
    x = torch.multinomial(torch.softmax(logits[0, -1], -1), 1).unsqueeze(0); out += chars[x.item()]
print("sample:", out)

characters: 18  corpus length: 34972
each sentence hides 3 independent 3-way choices = 3.30 nats over 23.3 characters, so NO model can beat perplexity 1.152
before training: perplexity 18.6  (a uniform guess over 18 characters would be 18)
step  25: perplexity 6.75
step 100: perplexity 1.35
step 400: perplexity 1.19
sample: . the cat chased the ball. the bird chased the bone. the cat ate the bone. the bird chased the fish. the bird saw th

Perplexity falls from 18.6 (a random guess among 18 characters) to 6.75 after 25 steps, 1.35 after 100 and 1.19 after 400, close to the theoretical floor 1.152. The sample the trained network writes is grammatical, "the cat chased the ball. the bird saw the bone.", and it chose every word itself. Two things to notice about how this was trained and used:

• Teacher forcing in training. At each position the network is fed the true previous character and asked to predict the next one, not its own earlier guess. This keeps training stable and fast.
• No teacher at inference. To generate, the network's own sampled character is fed back in as the next input. A model that was never shown its own mistakes can drift, a mismatch ("exposure bias") that Week 8 returns to.

3.4Chapter summary

• $h_t=\tanh(W_{xh}x_t+W_{hh}h_{t-1}+b)$, $y_t=\mathrm{softmax}(W_{hy}h_t+b_y)$; the same weights at every step.
• Parameters $h(d+h)+2h$, independent of the sequence length; cost $O(T)$; unfolding gives a deep network sharing weights.
• Measured: a character LSTM reached perplexity 1.19 against a true floor of 1.152. Train with teacher forcing; generate by feeding back your own samples.

Training an RNN means computing the gradient of a loss that depends on the weights through every time step. The algorithm is backpropagation applied to the unfolded network, and it comes with two famous troubles.

4.1Backpropagation through time (BPTT)

Because the same $W_{hh}$ is used at every step, its gradient is the sum of the gradients from each step: $\dfrac{\partial L}{\partial W_{hh}}=\sum_t\dfrac{\partial L}{\partial W_{hh}}\Big|_{\text{step }t}$. Check it by unfolding explicitly: give every step its own copy of $W$, run the same computation, and add up the copies' gradients.

import torch
torch.manual_seed(0)
h, T = 3, 6
W = torch.randn(h, h) * 0.5; U = torch.randn(h, 2) * 0.5; xs = torch.randn(T, 2); target = torch.randn(h)

def loss_fn(Ws):                                          # Ws: one weight matrix per step (equal values or independent copies)
    state = torch.zeros(h)
    for t in range(T):
        state = torch.tanh(Ws[t] @ state + U @ xs[t])
    return ((state - target) ** 2).sum()

shared = W.clone().requires_grad_()
loss_fn([shared] * T).backward()                          # ONE matrix used at every step (a real RNN)
copies = [W.clone().requires_grad_() for _ in range(T)]   # unfold in time: a separate copy per step
loss_fn(copies).backward()
total = sum(c.grad for c in copies)
print("gradient of the shared W:\n", shared.grad.numpy().round(4))
print("sum over time steps of the per-step gradients:\n", total.numpy().round(4))
print("equal:", torch.allclose(shared.grad, total, atol=1e-6), " -> BPTT = backprop on the unrolled network, then add up the contribution of every step")
print("per-step contributions (norm), step 1 ... step", T, ":", [round(c.grad.norm().item(), 4) for c in copies])

gradient of the shared W:
 [[-0.2379 -0.0027  0.0718]
 [-0.1086  0.0422  0.7295]
 [-0.5028 -0.8557  0.5535]]
sum over time steps of the per-step gradients:
 [[-0.2379 -0.0027  0.0718]
 [-0.1086  0.0422  0.7295]
 [-0.5028 -0.8557  0.5535]]
equal: True  -> BPTT = backprop on the unrolled network, then add up the contribution of every step
per-step contributions (norm), step 1 ... step 6 : [0.0, 0.0208, 0.04, 0.679, 0.6575, 1.1105]

The gradient of the shared matrix equals the sum of the per-step gradients, entry by entry. That is all BPTT is: ordinary backpropagation on the unrolled graph, with the gradients of the shared weights added up. (The first step's contribution is 0.0 because the initial state is zero, so $W_{hh}$ multiplied nothing at that step.) The later steps contribute more here, a first hint of the problem below: the influence of a weight use shrinks with its distance from the loss.

4.2Vanishing and exploding gradients

The gradient that travels from step $T$ back to step $0$ passes through $T$ copies of the same Jacobian, roughly $\big(W_{hh}^{\top}\,\mathrm{diag}(\tanh')\big)^{T}$. A repeated product behaves like $\rho^T$, where $\rho$ is the size of the matrix (its largest singular value, or the spectral radius): $\rho<1$ makes it vanish exponentially, $\rho>1$ makes it explode exponentially. To see this cleanly, give the RNN a recurrent matrix that is an orthogonal matrix times $\rho$, so every singular value equals $\rho$:

import torch
torch.manual_seed(0)
h = 32
Q, _ = torch.linalg.qr(torch.randn(h, h))                   # an orthogonal matrix: every singular value is exactly 1

def jacobian_norm(rho, T, activation):
    W = rho * Q                                              # all singular values equal rho
    h0 = torch.randn(h, requires_grad=True)
    state = h0
    for _ in range(T):
        state = activation(W @ state)
    state.sum().backward()                                   # gradient of the final state w.r.t. the first
    return h0.grad.norm().item()

print("gradient size reaching step 0 from step T   (linear RNN: exactly rho^T times the starting size)")
print("   rho      T=10        T=50       T=100")
for rho in (0.5, 0.9, 1.0, 1.1, 1.5):
    row = [jacobian_norm(rho, T, lambda z: z) for T in (10, 50, 100)]
    print(f"  {rho:4.1f}   " + "  ".join(f"{v:9.2e}" for v in row))
print("same with tanh (its derivative is at most 1, and tanh also saturates):")
for rho in (0.5, 0.9, 1.5, 3.0):
    row = [jacobian_norm(rho, T, torch.tanh) for T in (10, 50, 100)]
    print(f"  {rho:4.1f}   " + "  ".join(f"{v:9.2e}" for v in row))

gradient size reaching step 0 from step T   (linear RNN: exactly rho^T times the starting size)
   rho      T=10        T=50       T=100
   0.5    5.52e-03   5.02e-15   0.00e+00
   0.9    1.97e+00   2.92e-02   1.50e-04
   1.0    5.66e+00   5.66e+00   5.66e+00
   1.1    1.47e+01   6.64e+02   7.80e+04
   1.5    3.26e+02   3.61e+09   2.30e+18
same with tanh (its derivative is at most 1, and tanh also saturates):
   0.5    4.36e-03   3.37e-15   0.00e+00
   0.9    9.16e-01   1.15e-02   5.31e-05
   1.5    1.00e+01   2.42e+01   1.10e+03
   3.0    4.99e+01   2.55e+07   5.17e+10

For the linear RNN the table is exactly $\rho^T$ times 5.66: with $\rho=0.9$ the gradient after 100 steps is $1.5\times10^{-4}$; with $\rho=1.1$ it is $7.8\times10^{4}$; with $\rho=1.5$ it is $2.3\times10^{18}$. Only $\rho=1$ exactly stays put (5.66 everywhere), a knife edge no learned matrix will stay on. With tanh the picture is the same but softer: its derivative is at most 1 and shrinks when it saturates, which makes vanishing easier and explosion harder ($\rho=1.5$ still grows, $\rho=3$ explodes). The consequence in practice: a plain RNN cannot learn from something that happened 50 or 100 steps ago, because by then the learning signal has either vanished or blown up.

4.3Gradient clipping

Exploding gradients have a cheap, reliable cure. If the gradient's norm exceeds a threshold, rescale it to the threshold, keeping its direction: $g\leftarrow g\cdot\dfrac{\text{threshold}}{\|g\|}$ whenever $\|g\|>\text{threshold}$. The step stays a sensible size and still points downhill.

import torch
torch.manual_seed(0)
w = torch.nn.Parameter(torch.randn(3))
# a gradient that exploded
w.grad = torch.tensor([300.0, -400.0, 0.0])
print("gradient norm before clipping:", w.grad.norm().item())
threshold = 5.0
manual = w.grad * (threshold / w.grad.norm()) if w.grad.norm() > threshold else w.grad      # g <- g * threshold / ||g||
total_norm = torch.nn.utils.clip_grad_norm_([w], max_norm=threshold)                       # PyTorch does it in place
print("torch returned the ORIGINAL norm:", total_norm.item(), " new norm:", round(w.grad.norm().item(), 4))
print("matches the formula:", torch.allclose(w.grad, manual), " direction unchanged:", (w.grad / w.grad.norm()).tolist())

# why it matters: a linear recurrence with spectral radius > 1 has an enormous gradient, and one step can wreck the weights
torch.manual_seed(1)
W = (1.3 * torch.linalg.qr(torch.randn(8, 8))[0]).requires_grad_()
x = torch.randn(8)
state = x
for _ in range(40): state = W @ state
state.sum().backward()
print("\n40-step linear RNN with spectral radius 1.3: gradient norm", f"{W.grad.norm().item():.2e}")
torch.nn.utils.clip_grad_norm_([W], max_norm=1.0)
print("after clipping to 1.0:", round(W.grad.norm().item(), 4), "(a sane step, in the same direction)")

gradient norm before clipping: 500.0
torch returned the ORIGINAL norm: 500.0  new norm: 5.0
matches the formula: True  direction unchanged: [0.5999999642372131, -0.800000011920929, 0.0]

40-step linear RNN with spectral radius 1.3: gradient norm 2.60e+06
after clipping to 1.0: 1.0 (a sane step, in the same direction)

A gradient of norm 500 became norm 5 with the direction untouched (0.6, $-0.8$), and PyTorch's clip_grad_norm_ matches the formula (it returns the original norm, handy for logging). In the second experiment a 40-step linear RNN with $\rho=1.3$ has gradient norm $2.6\times10^6$, which one update would turn into garbage; clipped, it is 1.0. In code, call it between loss.backward() and optimizer.step(). Clipping fixes only explosion: it cannot bring back a vanished gradient.

4.4Truncated BPTT

For a sequence of length 100,000 (a book) full BPTT would unroll a 100,000-layer network in memory. Truncated BPTT processes the sequence in chunks of $k$ steps: the hidden state is carried forward across chunks, but it is detached at each boundary so the gradient flows back at most $k$ steps.

import torch, torch.nn as nn
torch.manual_seed(0)
d, h, T, k = 2, 8, 12, 4
rnn = nn.RNN(d, h, batch_first=True)
x = torch.randn(1, T, d, requires_grad=True)

def run(truncate):
    state = torch.zeros(1, 1, h)
    outs = []
    for t in range(T):
        if truncate and t % k == 0:
            state = state.detach()                       # cut the graph: no gradient flows back past this point
        out, state = rnn(x[:, t:t + 1], state)
        outs.append(out)
    return torch.cat(outs, 1)[0, -1].sum()

for truncate in (False, True):
    x.grad = None
    run(truncate).backward()
    g = x.grad[0].norm(dim=1)
    label = "truncated every 4 steps" if truncate else "full BPTT              "
    print(label, " gradient norm reaching input step 0..11:", [f"{v:.1e}" if v > 0 else "0" for v in g.tolist()])
print("truncated BPTT only lets the loss reach the last", k, "steps (here the inputs 8..11): earlier steps get exactly zero gradient")
print("the hidden STATE is still carried forward, so the forward pass still remembers; only the learning signal is cut")

full BPTT                gradient norm reaching input step 0..11: ['1.5e-04', '2.3e-04', '3.0e-04', '5.9e-04', '1.4e-03', '4.2e-03', '4.0e-03', '5.5e-03', '7.3e-03', '7.7e-02', '1.0e-01', '1.0e+00']
truncated every 4 steps  gradient norm reaching input step 0..11: ['0', '0', '0', '0', '0', '0', '0', '0', '7.3e-03', '7.7e-02', '1.0e-01', '1.0e+00']
truncated BPTT only lets the loss reach the last 4 steps (here the inputs 8..11): earlier steps get exactly zero gradient
the hidden STATE is still carried forward, so the forward pass still remembers; only the learning signal is cut

With full BPTT every input step receives some gradient (tiny for the early ones). With truncation every four steps the first 8 inputs get exactly zero and only the last block (inputs 8 to 11) learns from this loss. It is the standard compromise: memory and time stay bounded, at the price that dependencies longer than $k$ cannot be learned (they can still be used, since the state carries the information forward).

4.5Chapter summary

• BPTT = backprop on the unrolled network; the gradient of shared weights is the sum over time steps (verified: identical).
• Gradient over $T$ steps $\sim\rho^T$: vanishes for $\rho<1$ ($1.5\times10^{-4}$ at $T=100$, $\rho=0.9$), explodes for $\rho>1$ ($2.3\times10^{18}$ at $\rho=1.5$).
• Clip to fix explosion: $g\leftarrow g\cdot\text{thr}/\|g\|$; truncate BPTT (detach every $k$ steps) to bound cost; gated cells fix vanishing.

The previous chapter ended on a problem: a plain RNN rewrites its entire hidden state at every step and multiplies the gradient by $W_{hh}$ each time, so it cannot hold a fact for long. The Long Short-Term Memory (LSTM) network (Hochreiter and Schmidhuber, 1997) fixes this by separating long-term memory from working memory, and giving the network gates to decide what to keep, add and show.

5.1Two states and four blocks

• Cell state $c_t$: the long-term memory, a conveyor belt that runs through time with only small, additive changes.
• Hidden state $h_t$: the short-term, working memory and the cell's output.

Memory aid, "FICO": Forget, Input, Candidate, Output. All four read the same concatenation $[h_{t-1},x_t]$. The three gates use the sigmoid (values in 0 to 1, a dimmer switch); the candidate uses tanh (values in $-1$ to 1, the actual content). The gates are modulators: they are multiplied with a state ($\odot$ is element-wise multiplication), they are not outputs themselves. (The source notes call the candidate $\tilde c$; the code and PyTorch call it $g$.) Write the cell by hand and compare it with nn.LSTM:

import torch, torch.nn as nn
torch.manual_seed(0)
d, h, T = 3, 4, 5
lstm = nn.LSTM(d, h, batch_first=True)
x = torch.randn(1, T, d)
W, U, b = lstm.weight_ih_l0, lstm.weight_hh_l0, lstm.bias_ih_l0 + lstm.bias_hh_l0     # the four gates stacked: i, f, g, o
sig = torch.sigmoid

hs, h_t, c_t = [], torch.zeros(h), torch.zeros(h)
for t in range(T):
    z = W @ x[0, t] + U @ h_t + b
    i, f, g, o = sig(z[:h]), sig(z[h:2 * h]), torch.tanh(z[2 * h:3 * h]), sig(z[3 * h:])
    c_t = f * c_t + i * g                  # cell state: forget some of the old, add some of the new  (ADDITIVE)
    h_t = o * torch.tanh(c_t)              # hidden state: a filtered view of the cell
    hs.append(h_t)
print("hand-written LSTM matches nn.LSTM:", torch.allclose(torch.stack(hs), lstm(x)[0][0], atol=1e-6))
print("gate values at the last step: forget", [round(v, 2) for v in f.tolist()], " input", [round(v, 2) for v in i.tolist()], " output", [round(v, 2) for v in o.tolist()])

d, h = 10, 20
count = lambda m: sum(p.numel() for p in m.parameters())
print(f"\nparameters for d={d}, h={h}:  RNN {count(nn.RNN(d, h))},  GRU {count(nn.GRU(d, h))},  LSTM {count(nn.LSTM(d, h))}")
print("RNN = h(d+h)+2h =", h * (d + h) + 2 * h, "; LSTM = 4x that (4 weight blocks: input, forget, candidate, output) =", 4 * (h * (d + h) + 2 * h), "; GRU = 3x =", 3 * (h * (d + h) + 2 * h))

hand-written LSTM matches nn.LSTM: True
gate values at the last step: forget [0.49, 0.48, 0.61, 0.33]  input [0.51, 0.52, 0.65, 0.4]  output [0.61, 0.51, 0.55, 0.51]

parameters for d=10, h=20:  RNN 640,  GRU 1920,  LSTM 2560
RNN = h(d+h)+2h = 640 ; LSTM = 4x that (4 weight blocks: input, forget, candidate, output) = 2560 ; GRU = 3x = 1920

The hand-written cell reproduces PyTorch. Note the gate values: at random initialisation they sit near 0.5, "half open", and they only become meaningful through training. The cell has four times the weights of a plain RNN (four blocks: forget, input, candidate, output), 2,560 against 640 for $d=10,h=20$; a GRU (Week 8) has three times.

5.2Why it helps: the additive highway

Look at $c_t=f_t\odot c_{t-1}+i_t\odot g_t$. The path from $c_{t-1}$ to $c_t$ has derivative $f_t$: no matrix multiplication and no squashing function. Over $T$ steps the gradient along the cell state is a product of forget gates, $\prod f_t$. If the gates are near 1 the product survives; if they are 0.5 it does not. Measure $f^T$ with autograd for a constant forget gate:

import torch
# The LSTM cell-state recurrence  c_t = f * c_{t-1} + (new stuff)  has derivative  dc_t/dc_{t-1} = f.
# Over T steps the gradient from c_T back to c_0 is  f^T. Measure it with autograd for a constant forget gate.
def cell_gradient(f, T):
    c0 = torch.tensor(1.0, requires_grad=True)
    c = c0
    for _ in range(T):
        c = f * c + 0.0                       # the additive "input" term does not affect the path back to c_0
    c.backward()
    return c0.grad.item()

print("forget gate   T=10        T=50        T=100       (and f^100 from the formula)")
for f in (0.5, 0.9, 0.99, 1.0):
    print(f"  {f:5.2f}     " + "   ".join(f"{cell_gradient(f, T):9.2e}" for T in (10, 50, 100)) + f"    {f ** 100:9.2e}")
print("with f close to 1 the gradient survives 100 steps; a plain RNN multiplies by W and (1 - h^2) at every step instead")
print("the forget gate is not magic: if it sits at 0.5 the highway is as lossy as a plain RNN (the first row)")

forget gate   T=10        T=50        T=100       (and f^100 from the formula)
   0.50      9.77e-04    8.88e-16    7.89e-31     7.89e-31
   0.90      3.49e-01    5.15e-03    2.66e-05     2.66e-05
   0.99      9.04e-01    6.05e-01    3.66e-01     3.66e-01
   1.00      1.00e+00    1.00e+00    1.00e+00     1.00e+00
with f close to 1 the gradient survives 100 steps; a plain RNN multiplies by W and (1 - h^2) at every step instead
the forget gate is not magic: if it sits at 0.5 the highway is as lossy as a plain RNN (the first row)

With $f=0.99$ the gradient is still 0.37 after 100 steps and with $f=1$ it is exactly 1; with $f=0.5$ it is $10^{-31}$, as bad as a plain RNN. This is the real meaning of "the cell state is a gradient highway" and of the notes' "forget gate near 1 gives an identity connection": the LSTM does not remove the vanishing-gradient problem, it makes a path on which it can be avoided, and the gates have to learn to use it.

5.3An honest test: does a default LSTM already solve it?

Measure the gradient of the last output with respect to the very first input, for four recurrent nets at their default PyTorch initialisation, plus an LSTM whose forget-gate bias is set to 5 (so $f\approx0.99$ at the start):

import torch, torch.nn as nn
torch.set_num_threads(1)
def first_input_gradient(kind, T, forget_bias=None, d=8, h=32, seed=0):
    torch.manual_seed(seed)
    net = {"RNN": nn.RNN, "LSTM": nn.LSTM, "GRU": nn.GRU}[kind](d, h, batch_first=True)
    if forget_bias is not None:
        with torch.no_grad(): net.bias_hh_l0[h:2 * h].fill_(forget_bias)           # the forget-gate block of the LSTM bias
    x = torch.randn(1, T, d, requires_grad=True)
    net(x)[0][0, -1].sum().backward()                                              # loss depends on the LAST output only
    return x.grad[0, 0].norm().item()                                              # how much does the FIRST input matter?

print("gradient norm of the last output with respect to the FIRST input (default PyTorch initialisation)")
print("  T      RNN         GRU         LSTM        LSTM, forget bias 5")
for T in (10, 30, 60, 100):
    print(f"{T:4d}   " + "   ".join(f"{first_input_gradient(k, T):9.2e}" for k in ("RNN", "GRU", "LSTM")) + f"   {first_input_gradient('LSTM', T, forget_bias=5.0):9.2e}")
print("at default initialisation NONE of them keeps the gradient alive; the LSTM only does once its forget gate starts near 1")

gradient norm of the last output with respect to the FIRST input (default PyTorch initialisation)
  T      RNN         GRU         LSTM        LSTM, forget bias 5
  10    2.90e-03    2.25e-03    2.20e-03    2.84e-01
  30    8.50e-09    2.26e-07    2.00e-07    2.38e-01
  60    5.86e-17    4.13e-14    4.02e-13    1.17e-01
 100    0.00e+00    3.80e-22    1.12e-21    5.45e-02
at default initialisation NONE of them keeps the gradient alive; the LSTM only does once its forget gate starts near 1

The result is humbling and important. At default initialisation the RNN, GRU and LSTM all lose the gradient ($10^{-21}$ or less at 100 steps; the LSTM is no better than the plain RNN). Only the LSTM with its forget gate pushed toward 1 keeps a usable gradient (0.05 at 100 steps). The forget-gate bias initialisation is part of what makes an LSTM work; PyTorch's default of 0 (so $f\approx0.5$) is not it.

Does it matter for learning? A pure memory task: a sequence of $T$ numbers where the first is $\pm1$ and all the rest are noise, and the label is the sign of the first number. The only way to answer is to remember it for $T$ steps. Same size networks, 1,500 training steps each, three random seeds:

import torch, torch.nn as nn
torch.set_num_threads(1)

def run(kind, T, seed, forget_bias=None, steps=1500, h=16):
    # the label is the SIGN of the very first number; every later number is noise (a pure long-memory task)
    torch.manual_seed(seed)
    net = nn.RNN(1, h, batch_first=True) if kind == "RNN" else nn.LSTM(1, h, batch_first=True)
    if forget_bias is not None:
        with torch.no_grad(): net.bias_hh_l0[h:2 * h].fill_(forget_bias)
    head = nn.Linear(h, 1)
    params = list(net.parameters()) + list(head.parameters())
    opt = torch.optim.Adam(params, lr=3e-3)
    def batch(B):
        x = torch.randn(B, T, 1) * 0.5
        s = (torch.rand(B) < 0.5).float() * 2 - 1
        x[:, 0, 0] = s
        return x, (s > 0).float()
    for _ in range(steps):
        x, y = batch(64)
        loss = nn.functional.binary_cross_entropy_with_logits(head(net(x)[0][:, -1]).squeeze(1), y)
        opt.zero_grad(); loss.backward(); nn.utils.clip_grad_norm_(params, 1.0); opt.step()
    with torch.no_grad():
        x, y = batch(2000)
        return ((head(net(x)[0][:, -1]).squeeze(1) > 0).float() == y).float().mean().item()

print("accuracy at remembering the first number across T steps (3 seeds each, chance = 0.50)")
print("  model                       T=20                  T=40")
for name, kind, fb in (("tanh RNN", "RNN", None), ("LSTM, default init", "LSTM", None), ("LSTM, forget bias 3", "LSTM", 3.0)):
    cells = []
    for T in (20, 40):
        accs = [run(kind, T, s, fb) for s in (0, 1, 2)]
        cells.append(" ".join(f"{a:.2f}" for a in accs) + f"  ({sum(a > 0.9 for a in accs)}/3 learned)")
    print(f"  {name:24s}  {cells[0]:36s}  {cells[1]}")

accuracy at remembering the first number across T steps (3 seeds each, chance = 0.50)
  model                       T=20                  T=40
  tanh RNN                  1.00 1.00 1.00  (3/3 learned)         0.50 0.48 0.52  (0/3 learned)
  LSTM, default init        0.51 0.50 1.00  (1/3 learned)         0.50 0.49 0.51  (0/3 learned)
  LSTM, forget bias 3       1.00 1.00 1.00  (3/3 learned)         1.00 1.00 1.00  (3/3 learned)

At 20 steps the plain tanh RNN learns it in all three runs, and the default LSTM in just one of three. At 40 steps the plain RNN and the default LSTM both fail (accuracy 0.50, pure chance), while the LSTM with forget bias 3 learns it in every run, at both lengths. Two lessons. The idea behind LSTM is real: when the forget gate starts open, a signal can be carried across 40 steps and learned. But "use an LSTM" is not by itself a cure for long memory; initialisation and training matter, and a default LSTM can do worse than a plain RNN on a short task. (This is one small task with a small network, not a benchmark.)

5.4Chapter summary

• LSTM: cell state $c$ (long-term) and hidden state $h$ (working). Gates $f,i,o$ use sigmoid and modulate; candidate $g$ uses tanh. $c_t=f\odot c_{t-1}+i\odot g$, $h_t=o\odot\tanh c_t$.
• 4 times the parameters of an RNN (2,560 vs 640). The additive path has gradient $\prod f_t$: 0.37 after 100 steps at $f=0.99$, $10^{-31}$ at $f=0.5$.
• Measured: at default initialisation LSTM also vanishes; with forget bias 3-5 it learns a 40-step memory task in 3 of 3 runs where RNN and default LSTM fail.

What is likely to be asked from Week 7, with 30-second answers, deeper explanations, a lecture-or-extra tag on each question, and code you can run.

Week 7 questions are about what a recurrent network is, how it is trained, and why it fails at long range. Each answer starts with a 30-second version. Questions are tagged From the notes or Extra.

7.1Interview heat-map: what is most likely

7.2Language models and sequences

bag of words  'dog bites man': [1, 1, 1]   'man bites dog': [1, 1, 1]  <- identical, order is lost
RNN final hidden state, same words in a different order differ by: 0.997

unigram: P('the dog sat on the mat') = 2.55e-07  | perplexity on the 3 test sentences  11.8 (add-one smoothing)  | without smoothing the 3 sentences get [0.0, 0.0, 0.0]
bigram : P('the dog sat on the mat') = 6.97e-05  | perplexity on the 3 test sentences   5.9 (add-one smoothing)  | without smoothing the 3 sentences get [0.01646, 0.00137, 0.0]
trigram: P('the dog sat on the mat') = 4.27e-05  | perplexity on the 3 test sentences   6.3 (add-one smoothing)  | without smoothing the 3 sentences get [0.04, 0.04, 0.0]
vocabulary 13 -> possible bigrams 169 , trigrams 2197 ; the corpus has only 68 word positions

average -log P = 2.303 -> perplexity 10.0 (as unsure as a uniform choice among 10 words)
uniform over    2 words: cross-entropy 0.6931, exp = 2.0
uniform over   10 words: cross-entropy 2.3026, exp = 10.0
uniform over 1000 words: cross-entropy 6.9078, exp = 1000.0
logit boost 0.0 on the correct word: perplexity 10.00
logit boost 1.0 on the correct word: perplexity 4.31
logit boost 3.0 on the correct word: perplexity 1.45
logit boost 6.0 on the correct word: perplexity 1.02
lower perplexity = better; perplexity 1 would mean the model is certain of every word

each sentence hides 3 independent 3-way choices = 3.30 nats over 23.3 characters, so NO model can beat perplexity 1.152
before training: perplexity 18.6  (a uniform guess over 18 characters would be 18)
step  25: perplexity 6.75
step 100: perplexity 1.35
step 400: perplexity 1.19
sample: . the cat chased the ball. the bird chased the bone. the cat ate the bone. the bird chased the fish. the bird saw th

7.3The RNN and its training

hand-written loop matches nn.RNN: True
x: (1, 5, 3)  hidden states: (1, 5, 4)  Wxh: (4, 3)  Whh: (4, 4)
parameters: 36 = h*(d+h) + 2h = 36   (independent of the sequence length T = 5 )
  T =   5: the same network reads it, output (1, 5, 4)
  T =  50: the same network reads it, output (1, 50, 4)
  T = 500: the same network reads it, output (1, 500, 4)

 [-0.1086  0.0422  0.7295]
 [-0.5028 -0.8557  0.5535]]

gradient size reaching step 0 from step T   (linear RNN: exactly rho^T times the starting size)
   rho      T=10        T=50       T=100
   0.5    5.52e-03   5.02e-15   0.00e+00
   0.9    1.97e+00   2.92e-02   1.50e-04
   1.0    5.66e+00   5.66e+00   5.66e+00
   1.1    1.47e+01   6.64e+02   7.80e+04
   1.5    3.26e+02   3.61e+09   2.30e+18
same with tanh (its derivative is at most 1, and tanh also saturates):
   0.5    4.36e-03   3.37e-15   0.00e+00
   0.9    9.16e-01   1.15e-02   5.31e-05
   1.5    1.00e+01   2.42e+01   1.10e+03
   3.0    4.99e+01   2.55e+07   5.17e+10

loss.backward()
nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)   # rescale if the total norm exceeds 1
optimizer.step()

gradient norm before clipping: 500.0
torch returned the ORIGINAL norm: 500.0  new norm: 5.0
matches the formula: True  direction unchanged: [0.5999999642372131, -0.800000011920929, 0.0]

state = None
for chunk in chunks:                       # k steps at a time
    if state is not None:
        state = tuple(s.detach() for s in state)   # carry the value forward, cut the gradient
    out, state = lstm(chunk, state)
    loss = criterion(out, target); loss.backward(); opt.step(); opt.zero_grad()

full BPTT                gradient norm reaching input step 0..11: ['1.5e-04', '2.3e-04', '3.0e-04', '5.9e-04', '1.4e-03', '4.2e-03', '4.0e-03', '5.5e-03', '7.3e-03', '7.7e-02', '1.0e-01', '1.0e+00']
truncated every 4 steps  gradient norm reaching input step 0..11: ['0', '0', '0', '0', '0', '0', '0', '0', '7.3e-03', '7.7e-02', '1.0e-01', '1.0e+00']
truncated BPTT only lets the loss reach the last 4 steps (here the inputs 8..11): earlier steps get exactly zero gradient

7.4The LSTM

hand-written LSTM matches nn.LSTM: True
gate values at the last step: forget [0.49, 0.48, 0.61, 0.33]  input [0.51, 0.52, 0.65, 0.4]  output [0.61, 0.51, 0.55, 0.51]

parameters for d=10, h=20:  RNN 640,  GRU 1920,  LSTM 2560
RNN = h(d+h)+2h = 640 ; LSTM = 4x that (4 weight blocks: input, forget, candidate, output) = 2560 ; GRU = 3x = 1920

forget gate   T=10        T=50        T=100       (and f^100 from the formula)
   0.50      9.77e-04    8.88e-16    7.89e-31     7.89e-31
   0.90      3.49e-01    5.15e-03    2.66e-05     2.66e-05
   0.99      9.04e-01    6.05e-01    3.66e-01     3.66e-01
   1.00      1.00e+00    1.00e+00    1.00e+00     1.00e+00
with f close to 1 the gradient survives 100 steps; a plain RNN multiplies by W and (1 - h^2) at every step instead
the forget gate is not magic: if it sits at 0.5 the highway is as lossy as a plain RNN (the first row)

gradient norm of the last output with respect to the FIRST input (default PyTorch initialisation)
  T      RNN         GRU         LSTM        LSTM, forget bias 5
  10    2.90e-03    2.25e-03    2.20e-03    2.84e-01
  30    8.50e-09    2.26e-07    2.00e-07    2.38e-01
  60    5.86e-17    4.13e-14    4.02e-13    1.17e-01
 100    0.00e+00    3.80e-22    1.12e-21    5.45e-02
at default initialisation NONE of them keeps the gradient alive; the LSTM only does once its forget gate starts near 1

parameters for d=10, h=20:  RNN 640,  GRU 1920,  LSTM 2560
RNN = h(d+h)+2h = 640 ; LSTM = 4x that (4 weight blocks: input, forget, candidate, output) = 2560 ; GRU = 3x = 1920

accuracy at remembering the first number across T steps (3 seeds each, chance = 0.50)
  model                       T=20                  T=40
  tanh RNN                  1.00 1.00 1.00  (3/3 learned)         0.50 0.48 0.52  (0/3 learned)
  LSTM, default init        0.51 0.50 1.00  (1/3 learned)         0.50 0.49 0.51  (0/3 learned)
  LSTM, forget bias 3       1.00 1.00 1.00  (3/3 learned)         1.00 1.00 1.00  (3/3 learned)

Week 7 on one page: chapter glance, every equation, the pairs to tell apart, a problem-and-fix table, a checklist and corrections to the source notes.

7.1Week 7 at a glance

7.2Every equation

7.3Key terms

7.4The pairs you must tell apart

7.5Last-minute checklist

7.6Corrections and additions to the source notes