Statistics 2 Week 0 Notes: Probability Foundations and Random Variables

01

Foundations · Part 1

Statistical Study and Sample Space

How data, probability and statistics fit together, and the first objects of probability: the experiment, its outcomes and the sample space.

1.1Statistical study of phenomena

Data comes from the real world.
Probability is the theory: it builds models for that data.
Statistics is the application: it draws inferences from data, using analysis built on probability theory.

Figure 1. How data, probability and statistics fit together

Dataset in the course. The sample dataset used in the course is the IPL dataset from cricsheet.org, supplied as .yaml files.

1.2Experiment, outcome, sample space

Definitions

Experiment: a process or phenomenon we wish to study statistically. Outcome: the result of the experiment. Sample space: the set containing all outcomes of an experiment, typically denoted $S$ .

Experiment	Sample space	Size
Tossing a coin	$S = \{\text{Head}, \text{Tail}\}$	2
Throwing a die	$S = \{1, 2, 3, 4, 5, 6\}$	6
Throwing two dice	$36$ ordered pairs $(i, j)$	36

Figure 2. Sample spaces of a coin and a die, and how many events each has

Example 1Write the sample space

Write the sample space for: (a) tossing a coin twice, (b) choosing a month of the year, (c) the number of heads in 3 tosses.

Solution

(a) $S = \{HH, HT, TH, TT\}$ : 4 outcomes (order matters, so HT and TH are different).
(b) $S = \{\text{Jan}, \text{Feb}, \dots, \text{Dec}\}$ : 12 outcomes.
(c) $S = \{0, 1, 2, 3\}$ : the outcome here is the count, not the sequence of tosses.

Revision overview

DataComes from the real world
Sample: IPL data (cricsheet.org, .yaml)

ProbabilityThe theory
Builds models for the data

StatisticsThe application
Draws inferences using probability

Statistical StudyWeek 0 · Part 1

ExperimentA process we study statistically

OutcomeThe result of the experiment

Sample space SSet of all outcomes
Coin: {H, T} · Die: {1, …, 6}

Revision: the setting of probability on one page

Next · Foundations · Part 2Events

02

Foundations · Part 2

Events

Events are the central objects of probability. They are just sets, so everything from set theory carries over: containment, complement, union, intersection, and De Morgan's laws.

2.1Events

Definition: event

An event is a subset of the sample space. (There is a technical restriction on which subsets can be events; for a finite sample space every subset is an event.) An event has occurred if the actual outcome belongs to that event.

Events are sets, so all set-theory notions apply to them.
Events are the central objects in probability theory.

Counting events

Toss a coin: $S = \{H, T\}$ , events are $\emptyset, \{H\}, \{T\}, \{H, T\}$ : 4 events (that is $2^2$ ).
Throw a die: $S$ has 6 outcomes, so $64 = 2^6$ events, from $\emptyset$ up to the whole $S$ .
Some are easy to describe in words: "getting an even number", "getting a multiple of 3".

Figure 3. The four events of one coin toss

Links to Week 1 of Maths. A set with

n

elements has

2^n

subsets (the power set), which is exactly why a sample space with

n

outcomes has

2^n

events.

Containment and complement

One event can be contained in another: $A \subseteq B$ . Throw a die, $A = \{2, 6\}$ , $B$ = even numbers. If $A$ occurred, $B$ has also occurred. If $B$ occurred, $A$ may or may not have occurred.
Complement of $A$ , written $A^c$ = outcomes of $S$ not in $A$ = $S \setminus A$ . Die: $A = \{2, 4, 6\}$ (even) gives $A^c = \{1, 3, 5\}$ (odd). If $A$ occurred, $A^c$ did not, and vice versa.

Figure 4. Containment (left) and complement (right) on a die

2.2Combining events

Since events are subsets, you can take complements, unions and intersections:

Operation	In English	Symbol	Occurs when
Union	"or"	$A \cup B$	at least one of A, B occurs
Intersection	"and"	$A \cap B$	both A and B occur
Complement	"not"	$A^c$	A does not occur
Difference	"A but not B"	$A \setminus B = A \cap B^c$	A occurs and B does not

Figure 5. The four basic combinations as shaded regions

2.3Disjoint events and Venn diagrams

Definition: disjoint

Two events with an empty intersection are disjoint. If $A$ and $B$ are disjoint: if $A$ occurred, $B$ did not, and if $B$ occurred, $A$ did not.

An event and its complement

$A$ and $A^c$ are always disjoint.

A \cap A^c = \emptyset \qquad A \cup A^c = S

Multiple events

$E_1, E_2, E_3, \dots$ are disjoint if for any $i \neq j$ , $E_i \cap E_j = \emptyset$ .

Large sample space? Partition it into disjoint events for easier study.

Figure 6. Reading events off a Venn diagram

De Morgan's laws

(A \cup B)^c = A^c \cap B^c \qquad (A \cap B)^c = A^c \cup B^c

In words: "not (A or B)" means "not A and not B"; "not (A and B)" means "not A or not B".

Figure 7. De Morgan's law: the outside of A ∪ B is the same region as Aᶜ ∩ Bᶜ

Example 2Describe the events

Throw a die. $A = \{1, 2, 3\}$ , $B = \{2, 4, 6\}$ . Find $A \cup B$ , $A \cap B$ , $A \setminus B$ , $B^c$ and $(A \cup B)^c$ . Are $A$ and $B$ disjoint?

Solution

Event	Set	In words
$A \cup B$	$\{1, 2, 3, 4, 6\}$	1, 2, 3 or even
$A \cap B$	$\{2\}$	small and even
$A \setminus B$	$\{1, 3\}$	in A, not even
$B^c$	$\{1, 3, 5\}$	odd
$(A \cup B)^c$	$\{5\} = A^c \cap B^c$	De Morgan ✓

$A \cap B = \{2\} \neq \emptyset$ , so $A$ and $B$ are not disjoint.

Revision overview

EventA subset of S
Occurred if the outcome is in it
n outcomes → 2ⁿ events

ContainmentA ⊆ B: A occurred ⇒ B occurred
Not the other way

ComplementAᶜ = S \ A
A and Aᶜ never both occur

EventsWeek 0 · Part 2

CombiningUnion ∪ = "or"
Intersection ∩ = "and"
A \ B = A ∩ Bᶜ

DisjointEmpty intersection
Partition a big S into disjoint events

De Morgan(A ∪ B)ᶜ = Aᶜ ∩ Bᶜ
(A ∩ B)ᶜ = Aᶜ ∪ Bᶜ

Revision: events on one page

Next · Foundations · Part 3Probability, Axioms and Properties

03

Foundations · Part 3

Probability, Axioms and Properties

Probability is a function that assigns a number between 0 and 1 to every event. Two simple axioms are enough to derive everything else, including the properties we use all the time.

3.1Probability and the axioms

Probability is a function $P$ that assigns to each event a real number between 0 and 1. The value represents the chance of that event occurring.
Higher value → higher chance. 0 means the event cannot occur; 1 means it always occurs. Probability is often quoted as a percentage.
Probability space = sample space + events + probability function, taken together.

Figure 8. Probability lies between 0 and 1

The two axioms

Axiom 1: $P(S) = 1$ . The whole sample space has probability 1.

Axiom 2: if $E_1, E_2, E_3, \dots$ are disjoint events, then $P(E_1 \cup E_2 \cup E_3 \cup \cdots) = P(E_1) + P(E_2) + P(E_3) + \cdots$

Figure 9. The two axioms

There may be infinitely many events in Axiom 2.
The axioms are intuitive and restrict which functions can be probability functions.
They lead to several useful deductions about combinations and subsets of events.

3.2Basic properties of probability

Property	Statement
1 · Empty set	$P(\emptyset) = 0$
2 · Complement	$P(E^c) = 1 - P(E)$
3 · Subset	If $E \subseteq F$ : $P(F) = P(E) + P(F \setminus E)$ , which implies $P(E) \le P(F)$
4 · Difference and intersection	$P(E) = P(E \cap F) + P(E \setminus F)$ $\;\cdot\;$ $P(F) = P(E \cap F) + P(F \setminus E)$
5 · Union and intersection	$P(E \cup F) = P(E) + P(F) - P(E \cap F)$

Figure 10. If E ⊆ F then P(F) = P(E) + P(F \ E), so P(E) ≤ P(F)

Useful difference formulas

P(A \setminus B) = P(A \cup B) - P(B) \qquad P(A \setminus B) = P(A) - P(A \cap B)

If $A$ and $B$ are disjoint, $P(A \setminus B) = P(A)$ .

Why property 5 is true. Adding

P(E) + P(F)

counts the overlap

E \cap F

twice. Subtract it once. The same idea gave

n(P \cup B) = n(P) + n(B) - n(P \cap B)

in Maths Week 1.

3.3Working with probability spaces

Example: restaurant hiring

A waiter and a cashier are hired from 4 applicants: David and Megha from Delhi, Rajesh and Veronica from Mumbai. One person is hired at random as waiter, another from the remaining as cashier.

Figure 11. The 12 outcomes (waiter, cashier) and the events A, B, C

Sample space (ordered pairs, waiter first): $S = \{(D,M), (D,R), (D,V), (M,D), (M,R), (M,V), (R,D), (R,M), (R,V), (V,D), (V,M), (V,R)\}$ : 12 outcomes.
Event A: cashier is from Delhi: $A = \{(D,M), (R,M), (V,M), (M,D), (R,D), (V,D)\}$ (6 outcomes).
Event B: exactly one position filled by a Delhiite: $B = \{(D,R), (D,V), (M,R), (M,V), (R,D), (V,D), (R,M), (V,M)\}$ (8 outcomes).
Event C: neither position filled by a Delhiite: $C = \{(V,R), (R,V)\}$ (2 outcomes).

Example: fishing town

The chance of catching more than 400 kg of fish in a day is 35%; the chance of more than 500 kg is 10%. What is the chance of catching between 400 and 500 kg?

Figure 12. B ⊆ A: anything over 500 kg is also over 400 kg

$A$ = catch $> 400$ kg, $P(A) = 0.35$ . $B$ = catch $> 500$ kg, $P(B) = 0.10$ .
Key observation: $B \subseteq A$ .
$P(>400 \text{ and } \le 500) = P(A \setminus B) = P(A) - P(B) = 0.35 - 0.10 = \mathbf{0.25}$ .

Example: weather forecast

Chance of rain = 60%. Chance of max temperature above 30° = 70%. Chance of both = 40%. What is the chance of no rain and temperature below 30°?

Figure 13. All four regions of the Venn diagram add up to 1

$A$ = rain, $P(A) = 0.6$ ; $B$ = temperature $> 30^\circ$ , $P(B) = 0.7$ ; $P(A \cap B) = 0.4$ .
We want $P(A^c \cap B^c)$ . By De Morgan: $P(A^c \cap B^c) = P((A \cup B)^c) = 1 - P(A \cup B)$ .
$= 1 - (0.6 + 0.7 - 0.4) = 1 - 0.9 = \mathbf{0.1}$ .

Example 3Using the properties

$P(A) = 0.5$ , $P(B) = 0.4$ , $P(A \cap B) = 0.2$ . Find $P(A \cup B)$ , $P(A \setminus B)$ , $P(A^c)$ and $P(A^c \cap B^c)$ .

Solution

$P(A \cup B) = 0.5 + 0.4 - 0.2 = 0.7$ .
$P(A \setminus B) = P(A) - P(A \cap B) = 0.5 - 0.2 = 0.3$ .
$P(A^c) = 1 - 0.5 = 0.5$ .
$P(A^c \cap B^c) = 1 - P(A \cup B) = 1 - 0.7 = 0.3$ .

Example 4Can this be a probability?

A student claims $P(A) = 0.7$ , $P(B) = 0.5$ and $P(A \cup B) = 0.6$ . Is that possible?

Solution

No. $A \subseteq A \cup B$ , so $P(A) \le P(A \cup B)$ by property 3. But $0.7 > 0.6$ . (Equivalently, $P(A \cap B) = 0.7 + 0.5 - 0.6 = 0.6$ would exceed $P(B) = 0.5$ , which is impossible since $A \cap B \subseteq B$ .)

3.4Inclusion–exclusion for three events, "at least one", and the birthday problem

A. Three events

For two events $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ . For three, add the singles, subtract the pairs, add back the triple:

P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(A \cap C) - P(B \cap C) + P(A \cap B \cap C)

Why. An outcome in all three events is counted $3$ times in the singles, subtracted $3$ times in the pairs, so it needs to be added back once to be counted exactly once. An outcome in exactly two events is counted twice and subtracted once, so it is counted once.

Example. $P(A) = 0.5$ , $P(B) = 0.4$ , $P(C) = 0.3$ , $P(A \cap B) = 0.2$ , $P(A \cap C) = 0.1$ , $P(B \cap C) = 0.1$ , $P(A \cap B \cap C) = 0.05$ . Then

P(A \cup B \cup C) = 1.2 - 0.4 + 0.05 = 0.85, \qquad P(\text{none of the three}) = 1 - 0.85 = 0.15

Union bound (Boole). Without any other information, $P(A_1 \cup \cdots \cup A_n) \le P(A_1) + \cdots + P(A_n)$ . In the example $1.2 \ge 0.85$ . Equality holds exactly when the events are disjoint.

B. "At least one" is a complement

Computing "at least one" directly means adding many cases. Take the complement instead: $P(\text{at least one}) = 1 - P(\text{none})$ , and "none" is usually a single product when the trials are independent.

Three fair coin tosses, at least one head: $1 - \left(\tfrac12\right)^3 = \tfrac78$ .
A fair die thrown 4 times, at least one six: $1 - \left(\tfrac56\right)^4 = 1 - \tfrac{625}{1296} = \tfrac{671}{1296} \approx 0.518$ . (This is the same idea as the geometric cumulative formula $P(G \le k) = 1 - (1 - p)^k$ .)

C. The birthday problem

In a group of $n$ people (365 equally likely birthdays, ignoring 29 February), what is the probability that at least two share a birthday? Use the complement. The first person can have any birthday, the second must avoid 1 date, the third must avoid 2 dates, and so on:

P(\text{no shared birthday}) = \frac{365}{365}\cdot\frac{364}{365}\cdot\frac{363}{365}\cdots\frac{365 - n + 1}{365}, \qquad P(\text{shared}) = 1 - P(\text{no shared birthday})

$n$	10	22	23	30	50	57
$P(\text{shared})$	0.117	0.476	0.507	0.706	0.970	0.990

With only 23 people the probability of a shared birthday already passes one half. It feels too small because we compare one person against 365 days, but the number of pairs is $\binom{23}{2} = 253$ , and any pair can match.

D. Counting equally likely outcomes

When all outcomes are equally likely, $P(E) = \dfrac{|E|}{|S|}$ , and the counts come from combinations. A committee of 3 is chosen at random from 5 boys and 4 girls. $P(\text{exactly 2 girls}) = \dfrac{\binom42\binom51}{\binom93} = \dfrac{6 \cdot 5}{84} = \dfrac{5}{14} \approx 0.357$ .

Q1The union of three events

Numerical Answer Type (NAT).

$P(A) = 0.5$ , $P(B) = 0.4$ , $P(C) = 0.3$ , $P(A \cap B) = 0.2$ , $P(A \cap C) = 0.1$ , $P(B \cap C) = 0.1$ , $P(A \cap B \cap C) = 0.05$ . Find $P(\text{none of } A, B, C \text{ occurs})$ .

Solution

Idea

"None" is the complement of the union.

$P(A \cup B \cup C) = 0.5 + 0.4 + 0.3 - 0.2 - 0.1 - 0.1 + 0.05 = 0.85$ .
$P(\text{none}) = 1 - 0.85 = 0.15$ .
Answer: 0.15

Q2At least one six

Numerical Answer Type (NAT).

A fair die is thrown 4 times. What is the probability of at least one six? (Give a decimal rounded to 3 places.)

Solution

Idea

Complement: no six in four throws.

$P(\text{no six}) = (5/6)^4 = 625/1296 \approx 0.4823$ .
$P(\text{at least one six}) = 1 - 0.4823 = 0.5177 \approx 0.518$ .
Answer: 0.518

Q3Birthdays

Multiple Select Question (MSQ): select all that apply.

Assume 365 equally likely birthdays. Choose the correct statements.

1

In a group of 23 people, the probability that at least two share a birthday is more than

\tfrac12

.

2

In a group of 23 people, the probability that at least two share a birthday is

\tfrac{23}{365}

.

3

P(\text{no shared birthday among } n) = \dfrac{365}{365}\cdot\dfrac{364}{365}\cdots\dfrac{365 - n + 1}{365}

.

4

With 366 or more people a shared birthday is certain (ignoring 29 February).

Solution

Idea

Use the complement and the pigeonhole principle.

1 ✓ for $n = 23$ the probability is $\approx 0.507$ . 2 ✗ $23/365 \approx 0.063$ counts only comparisons with one person. 3 ✓ the product formula. 4 ✓ 366 people cannot fit in 365 distinct dates.
Answer: options 1, 3 and 4.

Q4A random committee

Numerical Answer Type (NAT).

A committee of 3 is chosen at random from 5 boys and 4 girls. What is the probability that it has exactly 2 girls? (Give a decimal rounded to 3 places.)

Solution

Idea

Favourable committees: 2 of the 4 girls and 1 of the 5 boys. Total: any 3 of the 9.

$\dbinom42\dbinom51 = 6 \times 5 = 30$ and $\dbinom93 = 84$ .
$30/84 = 5/14 \approx 0.357$ .
Answer: 0.357

Revision overview

Probability PFunction: event → number in [0, 1]
0 impossible, 1 certain

AxiomsP(S) = 1
Disjoint events: probabilities add
(even infinitely many)

Probability spaceSample space + events + P

Probability and AxiomsWeek 0 · Part 3

PropertiesP(∅) = 0 · P(Eᶜ) = 1 − P(E)
E ⊆ F ⇒ P(E) ≤ P(F)
P(E ∪ F) = P(E) + P(F) − P(E ∩ F)

DifferencesP(A \ B) = P(A) − P(A ∩ B)
P(A \ B) = P(A ∪ B) − P(B)

Worked examplesHiring: 12 outcomes
Fishing: 0.35 − 0.10 = 0.25
Weather: 1 − 0.9 = 0.1

Revision: probability and the axioms on one page

Next · Foundations · Part 4Distributions, Uniform Distribution and Derangements

04

Foundations · Part 4

Distributions, Uniform Distribution and Derangements

A distribution gives probabilities to individual outcomes. The simplest one makes every outcome equally likely. A famous uniform example, the mixed-up hats, leads to derangements and the number 1/e.

4.1Distributions and the uniform distribution

Definition: distribution

A distribution assigns probabilities to individual outcomes. This is possible when the outcomes can be enumerated as first, second, third and so on (a countable sample space), or when the sample space is finite.

Uniform distribution on a finite sample space

All outcomes are equally likely: assign the same probability to each.

P(\text{one outcome}) = \frac{1}{|S|} \qquad P(\text{event}) = \frac{\text{number of outcomes in the event}}{\text{number of outcomes in } S}

Example 1: marbles in an urn

5 red and 8 blue marbles; pick one at random. $|S| = 13$ , so $P(\text{red}) = \frac{5}{13}$ and $P(\text{blue}) = \frac{8}{13}$ .

Figure 14. Uniform: count the favourable outcomes and divide by all

Example 2: sum of two dice

Throw 2 dice; $|S| = 36$ equally likely ordered pairs. Event $E$ = sum is 8 = $\{(2,6), (6,2), (3,5), (5,3), (4,4)\}$ , 5 outcomes, so $P(E) = \frac{5}{36}$ .

Figure 15. Five of the 36 equally likely pairs sum to 8

Example 3: mixed-up hats

3 people\'s identical hats get mixed up; each picks one at random. What is the probability that nobody gets their own hat? $|S| = 3! = 6$ arrangements, and only 2 of them leave nobody with their own hat, so $P(E) = \frac{2}{6} = \frac{1}{3}$ .

Figure 16. The six arrangements; two of them are derangements

What if there were 30 people? The answer approaches $\frac{1}{e} \approx 37\%$ .

4.2Derangements

Definition: derangement

A derangement is an arrangement of objects into positions such that no object goes to its own specified position. In permutation language: a permutation $\sigma$ of $n$ elements with no fixed point, i.e. $\sigma(i) \neq i$ for all $i \in \{1, \dots, n\}$ .

D_n = n! \sum_{k=0}^{n} \frac{(-1)^k}{k!} \qquad D(n) = n \cdot D(n-1) + (-1)^n

n	1	2	3	4	5	6	7	8
D(n)	0	1	2	9	44	265	1854	14833

The sum tends to $n!/e$ for larger $n$ : $D_n \approx \frac{n!}{e}$ .
Probability of a derangement: $P(n) = \frac{D(n)}{n!} \approx \frac{1}{e}$ . This is exactly why the mixed-up-hats answer settles near 37% no matter how many people there are.

Figure 17. P(derangement) for n = 1 … 9 people

Check the recursion.

D(3) = 3 \cdot D(2) + (-1)^3 = 3 \cdot 1 - 1 = 2

,

D(4) = 4 \cdot 2 + 1 = 9

,

D(5) = 5 \cdot 9 - 1 = 44

. (Here

D(1) = 0

,

D(2) = 1

.)

Example 5Uniform probabilities

A fair die is thrown once. Find the probability of (a) a number greater than 4, (b) a prime, (c) an even number or a multiple of 3.

Solution

(a) $\{5, 6\}$ : $\frac{2}{6} = \frac{1}{3}$ .
(b) $\{2, 3, 5\}$ : $\frac{3}{6} = \frac{1}{2}$ .
(c) even $\{2, 4, 6\}$ or multiple of 3 $\{3, 6\}$ gives $\{2, 3, 4, 6\}$ : $\frac{4}{6} = \frac{2}{3}$ . (Check: $\frac{3}{6} + \frac{2}{6} - \frac{1}{6} = \frac{4}{6}$ .)

Example 6Two dice

Two fair dice are thrown. Find the probability that (a) the sum is 7, (b) both show the same number, (c) the sum is at most 3.

Solution

(a) $\{(1,6), (2,5), (3,4), (4,3), (5,2), (6,1)\}$ : $\frac{6}{36} = \frac{1}{6}$ .
(b) $\{(1,1), \dots, (6,6)\}$ : $\frac{6}{36} = \frac{1}{6}$ .
(c) sum 2: $(1,1)$ ; sum 3: $(1,2), (2,1)$ : $\frac{3}{36} = \frac{1}{12}$ .

Example 7Hats for four people

Four people mix up their hats. What is the probability that nobody gets their own?

Solution

$|S| = 4! = 24$ ; $D(4) = 9$ .
$P = \frac{9}{24} = \frac{3}{8} = 0.375$ .
Compare $\frac{1}{e} \approx 0.3679$ : already very close at $n = 4$ .

Revision overview

DistributionProbabilities on single outcomes
Needs a finite or countable S

UniformAll outcomes equally likely
P(outcome) = 1/|S|
P(E) = |E| / |S|

ExamplesUrn: 5/13, 8/13
Two dice sum 8: 5/36
3 hats: 1/3

Distributions and DerangementsWeek 0 · Part 4

DerangementNobody in their own position
σ(i) ≠ i for all i

CountingD(n) = n · D(n−1) + (−1)ⁿ
0, 1, 2, 9, 44, 265, 1854, 14833

LimitP(n) = D(n)/n! → 1/e ≈ 37%
Almost independent of n

Revision: distributions and derangements on one page

Next · Probability · Part 1Conditional Probability, Total Probability and Bayes' Theorem

05

Probability · Part 1

Conditional Probability, Total Probability and Bayes' Theorem

New information changes probabilities. Conditional probability formalises this; the law of total probability combines several cases; and Bayes' theorem reverses a conditional.

5.1Conditional probability

Start with a probability space $(S, \text{events}, P)$ . Let $B$ be an event with $P(B) > 0$ . Knowing that $B$ occurred changes the space we work in:

Sample space: $B$ .
Events: $A \cap B$ , for every event $A$ in the original space.
Probability function: $P(A \cap B) \div P(B)$ .

Definition: conditional probability

P(A \mid B) = \frac{P(A \cap B)}{P(B)}

Rearranged, this is the multiplication rule: $P(A \cap B) = P(B) \cdot P(A \mid B)$ .

Example 1: throw a die, given an even number

Figure 18. Once we know an even number came up, the space shrinks to {2, 4, 6}

$S = \{1, \dots, 6\}$ , uniform, so each outcome has probability $\frac{1}{6}$ . $E = \{2, 4, 6\}$ , $P(E) = \frac{1}{2}$ .
$P(\{2\} \mid E) = \frac{1/6}{1/2} = \frac{1}{3}$ , $P(\{4\} \mid E) = \frac{1}{3}$ .
$P(\{1\} \mid E) = \frac{P(\emptyset)}{P(E)} = 0$ ; $P(\{3\} \mid E) = P(\{5\} \mid E) = 0$ .
$P(\{2, 5\} \mid E) = \frac{1/6}{1/2} = \frac{1}{3}$ ; $P(\{2, 3, 4\} \mid E) = \frac{2/6}{1/2} = \frac{2}{3}$ .

Notice. Odd numbers get probability 0 once you know an even number came up: the conditional space has shrunk to

B

.

Example 2: students from three states

15 students: 4 from State 1, 8 from State 2, 3 from State 3. Three students are chosen one after another. Probability they come from State 1, State 3, State 1 in that order? Chain the conditionals; the pool shrinks after each pick:

P(E_1 \cap E_2 \cap E_3) = P(E_1)\, P(E_2 \mid E_1)\, P(E_3 \mid E_1 \cap E_2) = \frac{4}{15} \times \frac{3}{14} \times \frac{3}{13} = \frac{6}{455}

Figure 19. The pool shrinks after every pick

Example 3: two children

A family has two children. Probability that both are girls, given that at least one is a girl? $S = \{BB, BG, GG, GB\}$ . Event $E_1$ (at least one girl) $= \{BG, GG, GB\}$ , so $P(E_1) = \frac{3}{4}$ , and $P(\{GG\} \mid E_1) = \frac{1/4}{3/4} = \frac{1}{3}$ .

Figure 20. Conditioning removes only BB, leaving three equally likely outcomes

Classic trap

The intuitive answer $\frac{1}{2}$ is wrong: conditioning removes only $BB$ , leaving three equally likely outcomes, not two.

5.2Law of total probability

P(A) = P(A \cap B) + P(A \cap B^c) = P(A \mid B)\,P(B) + P(A \mid B^c)\,P(B^c)

Proof idea: $A$ is the disjoint union of $A \cap B$ and $A \cap B^c$ . Axiom 2 splits $P(A)$ into the two pieces, then conditional probability rewrites each one.

General form

If $B_1, B_2, \dots, B_n$ form a partition of $S$ :

P(A) = P(A \mid B_1)P(B_1) + P(A \mid B_2)P(B_2) + \cdots + P(A \mid B_n)P(B_n)

Example 1: interest rates and unemployment

If interest rates rise, there is a 60% chance unemployment increases; if they do not rise, only a 30% chance. There is a 40% chance rates rise. What is the chance unemployment increases?

Figure 21. Tree diagram: add the branches that end in "unemployment up"

$A$ = rates rise, $P(A) = 0.4$ , $P(A^c) = 0.6$ . $B$ = unemployment increases, $P(B \mid A) = 0.6$ , $P(B \mid A^c) = 0.3$ .
$P(B) = (0.6 \times 0.4) + (0.3 \times 0.6) = 0.24 + 0.18 = \mathbf{0.42}$ : a 42% chance.

Example 2: five coins

A man has 5 coins: 2 double-headed, 1 double-tailed, 2 normal. He picks one at random and tosses it. Probability of a head? The three coin types partition the sample space.

Figure 22. Weighted by how many coins of each type exist

P(\text{head}) = 1 \cdot \frac{2}{5} + 0 \cdot \frac{1}{5} + \frac{1}{2} \cdot \frac{2}{5} = \frac{2}{5} + \frac{1}{5} = \frac{3}{5}

5.3Bayes' theorem

The setup: two events $A$ and $B$ ; $P(A)$ and $P(B)$ are known; $P(A \mid B)$ and $P(A \mid B^c)$ are known or easy to find. The question: find $P(B \mid A)$ , the reverse conditional.

P(A \cap B) = P(B)\,P(A \mid B) = P(A)\,P(B \mid A) \;\Rightarrow\; P(B \mid A) = \frac{P(B)\,P(A \mid B)}{P(A)}

Standard recipe

Use the law of total probability to compute $P(A)$ in the denominator, then plug it into Bayes\' formula.

Example 1: swine flu test

1% of people in a city have swine flu. 95% of people with the disease test positive; 2% of people without it also test positive. A random person tests positive. What is the chance they actually have swine flu?

$A$ = tests positive, $B$ = has swine flu. $P(B) = 0.01$ , $P(B^c) = 0.99$ , $P(A \mid B) = 0.95$ , $P(A \mid B^c) = 0.02$ .
Step 1 (total probability): $P(A) = (0.95 \times 0.01) + (0.02 \times 0.99) = 0.0293$ .
Step 2 (Bayes): $P(B \mid A) = \frac{0.01 \times 0.95}{0.0293} = 0.3242$ .

Figure 23. The same answer by counting 10,000 people

Why only 32%?

A positive test is far from a certainty when the disease is rare: there are far more healthy people, so even a 2% false-positive rate produces more false alarms (198) than true cases (95).

Example 2: MCQ guessing

A student answering a 4-choice MCQ knows the answer with probability $\frac{3}{4}$ ; otherwise she guesses at random. Given the question was answered correctly, what is the chance she knew it?

$A$ = answer correct, $B$ = knows. $P(A \mid B) = 1$ , $P(A \mid B^c) = \frac{1}{4}$ , $P(B) = \frac{3}{4}$ , $P(B^c) = \frac{1}{4}$ .
Total: $P(A) = (1 \times \frac{3}{4}) + (\frac{1}{4} \times \frac{1}{4}) = \frac{13}{16}$ .
Bayes: $P(B \mid A) = \frac{\frac{3}{4} \times 1}{13/16} = \frac{12}{13}$ .

Example 3: die then coins

Roll a fair die, then toss as many fair coins as the number shown. Given 5 heads were obtained, what is the probability the die showed 5?

$P(5 \text{ heads} \mid \text{die } i) = 0$ for $i = 1, 2, 3, 4$ : you cannot get 5 heads from fewer than 5 tosses.
$P(5 \text{ heads} \mid 5) = \frac{1}{32}$ ; $P(5 \text{ heads} \mid 6) = \frac{6}{64} = \frac{3}{32}$ .
Total: $P(5 \text{ heads}) = \frac{1}{32} \cdot \frac{1}{6} + \frac{3}{32} \cdot \frac{1}{6} = \frac{1}{48}$ .
Bayes: $P(5 \mid 5 \text{ heads}) = \frac{\frac{1}{6} \times \frac{1}{32}}{1/48} = \frac{1}{4}$ .

Example 8Conditional probability from a table

In a class of 50, 30 are boys. 12 boys and 8 girls wear glasses. A student is picked at random and wears glasses. What is the probability the student is a boy?

Solution

$P(\text{glasses}) = \frac{12 + 8}{50} = \frac{20}{50}$ .
$P(\text{boy} \cap \text{glasses}) = \frac{12}{50}$ .
$P(\text{boy} \mid \text{glasses}) = \frac{12/50}{20/50} = \frac{12}{20} = \frac{3}{5}$ .

Example 9Two factories

Factory 1 makes 60% of the bulbs, 2% of them defective. Factory 2 makes 40%, 5% defective. A bulb is defective. What is the probability it came from Factory 2?

Solution

$P(D) = 0.6 \times 0.02 + 0.4 \times 0.05 = 0.012 + 0.020 = 0.032$ .
$P(F_2 \mid D) = \frac{0.4 \times 0.05}{0.032} = \frac{0.020}{0.032} = 0.625$ .
So a defective bulb is more likely (62.5%) to be from the smaller factory, because of its higher defect rate.

5.4Bayes in practice: base rates, false positives and natural frequencies

The most common exam use of Bayes' theorem is a test for something rare: a medical screen, a spam filter, a defect detector. The surprising lesson is that even an accurate test can give a mostly wrong positive result when the thing it looks for is rare.

A. The vocabulary

Prevalence (base rate, prior): $P(D)$ , the fraction who really have the condition.
Sensitivity (true positive rate): $P(+ \mid D)$ .
Specificity (true negative rate): $P(- \mid D^c)$ . Its complement, $P(+ \mid D^c) = 1 - \text{specificity}$ , is the false positive rate.
Posterior: $P(D \mid +)$ , the probability that a person who tested positive really has it. This is what Bayes' theorem delivers.

B. A worked example

A condition has prevalence $1\%$ . A test has sensitivity $95\%$ and specificity $90\%$ (false positive rate $10\%$ ). A person tests positive. How likely is it that they have the condition?

P(+) = P(+ \mid D)P(D) + P(+ \mid D^c)P(D^c) = 0.95(0.01) + 0.10(0.99) = 0.0095 + 0.099 = 0.1085

P(D \mid +) = \frac{P(+ \mid D)P(D)}{P(+)} = \frac{0.0095}{0.1085} \approx 0.0876

Only about 8.8%, even though the test is $95\%$ sensitive.

Natural frequencies make the same point without formulas. Take 10,000 people:

	Test +	Test −	Total
Have the condition	95	5	100
Do not have it	990	8,910	9,900
Total	1,085	8,915	10,000

Of the $1{,}085$ positives only $95$ are real: $95/1085 \approx 0.0876$ . The $990$ false alarms come from the huge healthy group, and $10\%$ of $9{,}900$ is more than $95\%$ of $100$ .

C. What changes the answer

Prevalence. If the prevalence is $10\%$ instead, $P(+) = 0.95(0.1) + 0.1(0.9) = 0.185$ and $P(D \mid +) = 0.095/0.185 = 19/37 \approx 0.514$ . The same test now gives a posterior above one half.
A negative result. With the original numbers, $P(D \mid -) = \dfrac{0.05 \times 0.01}{0.0005 + 0.891} \approx 0.00056$ : a negative result is very reassuring.
Repeating the test. If two tests are independent given the person's true status, then two positives give $P(D \mid ++) = \dfrac{0.95^2(0.01)}{0.95^2(0.01) + 0.10^2(0.99)} = \dfrac{0.009025}{0.018925} \approx 0.477$ . The posterior after the first positive becomes the prior for the second.

Q1The posterior after a positive test

Numerical Answer Type (NAT).

A condition has prevalence $1\%$ ; the test has sensitivity $95\%$ and false positive rate $10\%$ . Find $P(D \mid +)$ as a percentage rounded to 2 decimals.

Solution

Idea

Total probability for $P(+)$ , then Bayes.

$P(+) = 0.95 \cdot 0.01 + 0.10 \cdot 0.99 = 0.1085$ .
$P(D \mid +) = 0.0095/0.1085 = 19/217 \approx 0.08756$ , i.e. $8.76\%$ .
Answer: 8.76

Q2Counting the positives

Numerical Answer Type (NAT).

With the same numbers, out of 10,000 people, how many are expected to test positive?

Solution

Idea

True positives plus false positives.

True positives: $0.95 \times 100 = 95$ . False positives: $0.10 \times 9{,}900 = 990$ .
Total: $95 + 990 = 1{,}085$ (equal to $10{,}000 \times P(+) = 10{,}000 \times 0.1085$ ).
Answer: 1085

Q3Why is the posterior so low?

Multiple Choice Question (MCQ): choose the one correct option.

A test with $95\%$ sensitivity gives a positive result, yet the probability of the condition is only about $9\%$ . Why?

1

The test is badly designed and its sensitivity is really below 50%.

2

The condition is rare, so the false positives from the large healthy group outnumber the true positives.

3

Bayes' theorem only works for large samples.

4

The posterior should equal the sensitivity, so the calculation must be wrong.

Solution

Idea

$P(+ \mid D)$ and $P(D \mid +)$ are different numbers.

990 false positives against 95 true positives: most positives are false alarms.
Option 4 confuses $P(+ \mid D)$ with $P(D \mid +)$ , the "transposed conditional" error.
Answer: option 2.

Q4A higher prevalence

Numerical Answer Type (NAT).

Same test (sensitivity $95\%$ , false positive rate $10\%$ ), but now the prevalence is $10\%$ . Find $P(D \mid +)$ , rounded to 3 decimals.

Solution

Idea

Same steps, new prior.

$P(+) = 0.95(0.1) + 0.10(0.9) = 0.095 + 0.09 = 0.185$ .
$P(D \mid +) = 0.095/0.185 = 19/37 \approx 0.514$ .
Answer: 0.514

Revision overview

Conditional probabilityP(A | B) = P(A ∩ B) / P(B)
Space shrinks to B
Needs P(B) > 0

Multiplication ruleP(A ∩ B) = P(B) · P(A | B)
Chain: P(E₁)P(E₂|E₁)P(E₃|E₁∩E₂)

Classic trapTwo children, at least one girl:
P(GG | …) = 1/3, not 1/2

ConditioningWeek 0 · Part 5

Total probabilityP(A) = Σ P(A | Bᵢ) P(Bᵢ)
Bᵢ partition S
Interest: 0.42 · Coins: 3/5

BayesP(B | A) = P(B) P(A | B) / P(A)
P(A) from total probability

ExamplesFlu: 32.42% · MCQ: 12/13
Die + coins: 1/4

Revision: conditioning, total probability and Bayes on one page

Next · Probability · Part 2Independence of Events

06

Probability · Part 2

Independence of Events

Two events are independent when knowing one tells you nothing about the other. This part defines it, separates it from "disjoint", and extends it to three or more events.

6.1Independence of events

Definition: independent events

A \text{ and } B \text{ are independent} \iff P(A \cap B) = P(A) \cdot P(B)

If $P(B) > 0$ and $A, B$ are independent, then $P(A \mid B) = P(A)$ : the probability of $A$ is unaffected by whether $B$ occurred.

Points to remember

Disjoint events (each with positive probability) are never independent. If $A$ and $B$ are disjoint and $B$ occurs, $A$ definitely did not occur, so $B$ \'s occurrence definitely affects $A$ \'s conditional probability.
For events (of positive probability) to be independent, they must have a non-empty intersection.

The fine print. The notes say disjoint events are never independent. That is true whenever both have positive probability. The only exception is an event of probability 0: if

P(A) = 0

then

P(A \cap B) = 0 = P(A)P(B)

for any

B

, so such an

A

is independent of everything (including events disjoint from it).

Example 1: even vs odd (not independent)

Throw a die. $A$ = even = $\{2, 4, 6\}$ , $B$ = odd = $\{1, 3, 5\}$ . $A \cap B = \emptyset$ . $P(A \cap B) = 0$ , but $P(A)P(B) = \frac{1}{4}$ , so they are not independent.

Example 2: even vs multiple of 3 (independent)

$A$ = even = $\{2, 4, 6\}$ , $B$ = multiple of 3 = $\{3, 6\}$ . $A \cap B = \{6\}$ . $P(A \cap B) = \frac{1}{6}$ and $P(A)P(B) = \frac{3}{6} \times \frac{2}{6} = \frac{1}{6}$ : independent ✓.

Figure 24. Independent although they overlap: the intersection has exactly the right size

Example 3: a card from a pack

$A$ = card is a spade, $B$ = card is a king, $A \cap B$ = king of spades. $P(A \cap B) = \frac{1}{52} = \frac{13}{52} \times \frac{4}{52} = P(A)P(B)$ : independent ✓.

Figure 25. One full row (spades) times one full column (kings) meet in exactly one card

Typical points of confusion

"A and B have an intersection, so how can they be independent?" They must, in fact, intersect (when both have positive probability).
"There is only one throw of the die / one card drawn, so how can two events be independent?" Independence is about probabilities, not about repeated trials.

6.2Mutual independence

Three events

P(A \cap B) = P(A)P(B) \quad P(A \cap C) = P(A)P(C) \quad P(B \cap C) = P(B)P(C) \quad P(A \cap B \cap C) = P(A)P(B)P(C)

Pairwise is not enough

The last condition on $A \cap B \cap C$ is an additional, important requirement. Pairwise independence alone is not mutual independence.

n events

P(A_{i_1} \cap A_{i_2} \cap \cdots \cap A_{i_k}) = P(A_{i_1})\,P(A_{i_2}) \cdots P(A_{i_k})

This is a constraint for every subset of the events: the probability of the intersection of any subset equals the product of their probabilities. Counting subsets of size at least 2, there are $2^n - n - 1$ constraints (the notes say "almost $2^n$ ").

Results that follow

If $A, B$ are independent, so are $A$ and $B^c$ : $P(A \cap B^c) = P(A) - P(A \cap B) = P(A)(1 - P(B)) = P(A)P(B^c)$ .
Also $A, B$ independent $\Rightarrow$ $A^c, B^c$ independent.
If $n$ events are mutually independent, any subset of them, with or without complementing, is independent too.

Example: roads between towns

Two roads connect A→B and two connect B→C. Each of the 4 roads is blocked with probability $p$ , independently. Find the probability there is an open route A→B, given there is no open route A→C.

Figure 26. A→B has two roads, B→C has two roads

No open route A→B means both blocked: $p^2$ . So $P(F) = 1 - p^2$ , where $F$ = open route A→B. Let $E$ = no open route A→C.
Split $E$ over $F$ and $F^c$ : $P(E) = P(E \mid F)P(F) + P(E \mid F^c)P(F^c) = p^2(1 - p^2) + 1 \cdot p^2 = 2p^2 - p^4$ .
$P(F \mid E) = \frac{P(E \mid F)P(F)}{P(E)} = \frac{p^2(1 - p^2)}{2p^2 - p^4} = \frac{1 - p^2}{2 - p^2}$ .

Check with $p = \frac{1}{3}$ .

\frac{1 - 1/9}{2 - 1/9} = \frac{8/9}{17/9} = \frac{8}{17} \approx 0.47

. Given that A→C is blocked, an open A→B is slightly less than even odds. As

p \to 0

the answer tends to

\frac{1}{2}

; as

p \to 1

it tends to 0.

Example 10Test for independence

Two fair coins are tossed. $A$ = first is a head, $B$ = second is a head, $C$ = both are the same. Are $A, B$ independent? $A, C$ ? Are $A, B, C$ mutually independent?

Solution

$P(A) = P(B) = P(C) = \frac{1}{2}$ ( $C = \{HH, TT\}$ ).
$A \cap B = \{HH\}$ : $\frac{1}{4} = \frac{1}{2} \cdot \frac{1}{2}$ ✓. $A \cap C = \{HH\}$ : $\frac{1}{4}$ ✓. $B \cap C = \{HH\}$ : $\frac{1}{4}$ ✓.
But $A \cap B \cap C = \{HH\}$ : $P = \frac{1}{4}$ , while $P(A)P(B)P(C) = \frac{1}{8}$ . ✗
So the events are pairwise independent but not mutually independent: the extra condition really matters.

Example 11Independent trials

A shooter hits a target with probability 0.8 on each of 3 independent shots. Find the probability that (a) all 3 hit, (b) none hit, (c) at least one hits.

Solution

(a) $0.8^3 = 0.512$ .
(b) $0.2^3 = 0.008$ .
(c) $1 - 0.008 = 0.992$ (complement of "none").

Revision overview

DefinitionP(A ∩ B) = P(A) · P(B)
Then P(A | B) = P(A)

Disjoint ≠ independentDisjoint (positive probability)
⇒ never independent
Independent events must overlap

ExamplesEven vs odd: no
Even vs multiple of 3: yes
Spade vs king: yes

IndependenceWeek 0 · Part 6

Mutual independence3 events: pairs AND the triple
n events: every subset multiplies

ComplementsA, B independent ⇒ A, Bᶜ independent
⇒ Aᶜ, Bᶜ independent

Roads exampleP(open A→B | no A→C) = (1 − p²)/(2 − p²)

Revision: independence on one page

Next · Probability · Part 3Bernoulli Trials, Binomial and Geometric

07

Probability · Part 3

Bernoulli Trials, Binomial and Geometric

Repeating a yes/no experiment independently gives the first two named distributions of the course: the binomial (how many successes in n trials) and the geometric (how long until the first success).

7.1Bernoulli trials

A single Bernoulli trial

Setting: the occurrence of event $A$ is "success"; non-occurrence is "failure". Let $p = P(A)$ .
Sample space: $\{\text{success}, \text{failure}\}$ , or equivalently $\{0, 1\}$ .
$P(1) = p$ , $P(0) = 1 - p$ . Denoted Bernoulli(p).

Repeated Bernoulli trials

Setting: repeat a Bernoulli trial multiple times independently: perform $n$ independent Bernoulli( $p$ ) trials.
Outcome: 0 or 1 for each of trial 1, trial 2, …, trial $n$ . Sample space: $2^n$ outcomes. E.g. $n = 3$ : $\{000, 001, 010, 011, 100, 101, 110, 111\}$ , 8 outcomes.
Probabilities are computed using independence: just multiply.

P(000) = (1-p)^3 \qquad P(101) = p(1-p)p = p^2(1-p)

The general formula

P(b_1 b_2 \dots b_n) = p^{w}(1-p)^{n-w}, \quad w = \text{number of 1s in } b_1 b_2 \dots b_n

Key insight

The actual sequence of 0s and 1s does not matter: only the number of successes matters. A 200-trial sequence with 36 ones has probability $p^{36}(1-p)^{164}$ , whatever the order.

Figure 27. The eight outcomes of three Bernoulli trials

Example: toss a fair coin 5 times

H is 0, T is 1. Sample space: $2^5 = 32$ equally likely outcomes, each with probability $\frac{1}{32}$ . The number of outcomes with $k$ tails is $\binom{5}{k}$ :

Tails	0	1	2	3	4	5
Outcomes	1	5	10	10	5	1
Probability	1/32	5/32	10/32	10/32	5/32	1/32

Figure 28. Number of tails in 5 tosses

$P(\text{at least 4 tails}) = \frac{5}{32} + \frac{1}{32} = \frac{6}{32}$ .
$P(\text{at most 3 heads})$ : at most 3 heads means at least 2 tails, so $= \frac{10 + 10 + 5 + 1}{32} = \frac{26}{32}$ (the notes write $\frac{1 + 5 + 10 + 10}{32}$ , the same number by symmetry).

Example: toss a biased coin 5 times

H is 0, T is 1, with $P(H) = \frac{1}{3}$ and $P(T) = \frac{2}{3}$ . Now the outcomes are not equally likely, but the formula still works:

Event	Probability	Value
$P(HHHHH)$	$(\frac{1}{3})^5$	$\frac{1}{243}$
$P(\text{1 tail})$	$5 \times (\frac{2}{3})(\frac{1}{3})^4$	$\frac{10}{243}$
$P(\text{2 tails})$	$10 \times (\frac{2}{3})^2(\frac{1}{3})^3$	$\frac{40}{243}$
$P(\text{3 tails})$	$10 \times (\frac{2}{3})^3(\frac{1}{3})^2$	$\frac{80}{243}$
$P(\text{4 tails})$	$5 \times (\frac{2}{3})^4(\frac{1}{3})$	$\frac{80}{243}$
$P(TTTTT)$	$(\frac{2}{3})^5$	$\frac{32}{243}$

Check.

1 + 10 + 40 + 80 + 80 + 32 = 243

, so the six probabilities add to 1. ✓

7.2Binomial distribution

Perform $n$ independent Bernoulli( $p$ ) trials. Outcome of interest: the number of successes, denoted $B(n, p)$ or just $B$ .
Sample space: $\{0, 1, 2, \dots, n\}$ .

Example: n = 3

\begin{aligned} P(B=0) &= P(000) = (1-p)^3 \\ P(B=1) &= P(001 \text{ or } 010 \text{ or } 100) = 3p(1-p)^2 \\ P(B=2) &= P(110 \text{ or } 101 \text{ or } 011) = 3p^2(1-p) \\ P(B=3) &= P(111) = p^3 \end{aligned}

Small n, all at once

Figure 29. Binomial probabilities for n = 1 … 5

The general expression

$P(B = k)$ = (number of sequences with exactly $k$ ones) $\times$ $p^k(1-p)^{n-k}$ . The number of $b_1 b_2 \dots b_n$ with exactly $k$ ones is $\binom{n}{k} = \frac{n!}{k!(n-k)!}$ .

Binomial PMF

P[B(n, p) = k] = \binom{n}{k}\, p^k (1-p)^{n-k}, \qquad k = 0, 1, \dots, n

Graph of Binomial(n, p): observations

Starts at $(1-p)^n$ , increases and reaches a peak, then falls to $p^n$ .
Where is the peak? Near $np$ : exactly at $\lfloor p(n+1) \rfloor$ , the largest integer $\le p(n+1)$ .
All the probabilities must add to 1: $(1-p)^n + \binom{n}{1}p(1-p)^{n-1} + \binom{n}{2}p^2(1-p)^{n-2} + \cdots + p^n = 1$ (the binomial theorem).

Figure 30. B(10, 0.5) peaks at 5, B(10, 0.3) at 3

A small caveat. If

p(n+1)

is itself an integer, the two values

p(n+1) - 1

and

p(n+1)

are both peaks (they have equal probability). For example

B(9, 0.5)

:

p(n+1) = 5

, and the probabilities at

k = 4

and

k = 5

are equal.

Example 1: disease testing

Each person has a disease with probability 0.1, independently. Out of 100 people tested, what is the probability that 20 test positive? $P(B(100, 0.1) = 20) = \binom{100}{20}(0.1)^{20}(0.9)^{80} \approx 0.0012$ .

Sense check. The expected number of positives is

np = 10

, so seeing 20 is unlikely: about 0.12%.

Example 2: fair coin, 10 tosses

(a) Probability the number of heads is a multiple of 3? Values $\{0, 3, 6, 9\}$ : $\frac{\binom{10}{0} + \binom{10}{3} + \binom{10}{6} + \binom{10}{9}}{2^{10}} = \frac{1 + 120 + 210 + 10}{1024} = \frac{341}{1024} \approx 0.333$ .
(b) Probability the number of heads is even? Values $\{0, 2, 4, 6, 8, 10\}$ : $\frac{1 + 45 + 210 + 210 + 45 + 1}{1024} = \frac{512}{1024} = \frac{1}{2}$ .

Why exactly one half? The binomial coefficients

\binom{10}{k}

for even

k

and for odd

k

both add up to

2^{9} = 512

, because

(1 + 1)^{10} = 1024

and

(1 - 1)^{10} = 0

.

Example 3: bits flipped in transmission

A bit sent by Alice to Bob gets flipped with $p = 0.1$ . Probability that at most 2 bits get flipped?

(a) 5 bits sent: $P(B(5, 0.1) \in \{0, 1, 2\}) = (0.9)^5 + 5(0.1)(0.9)^4 + 10(0.1)^2(0.9)^3 \approx 0.9914$ .
(b) 10 bits sent: $P(B(10, 0.1) \in \{0, 1, 2\}) = (0.9)^{10} + \binom{10}{1}(0.1)(0.9)^9 + \binom{10}{2}(0.1)^2(0.9)^8 \approx 0.9298$ .

7.3Geometric distribution

Perform independent Bernoulli( $p$ ) trials indefinitely. Outcome: the number of trials needed for the first success, denoted $G(p)$ or $G$ .
Sample space: $\{1, 2, 3, 4, \dots\}$ : it goes on and on.

P(G=1) = p, \quad P(G=2) = (1-p)p, \quad P(G=3) = (1-p)^2 p, \quad \dots, \quad P(G=k) = (1-p)^{k-1}p

First $k - 1$ trials fail, the $k$ -th succeeds.

Figure 31. Geometric(1/6): starts at p and keeps falling

Starts at $p$ and keeps falling; keeps decreasing but, if $p < 1$ , never goes all the way to zero.

The cumulative formula

P(G \le k) = p + (1-p)p + \cdots + (1-p)^{k-1}p = 1 - (1-p)^k \qquad P(G > k) = (1-p)^k

Easy way to remember.

G > k

means the first

k

trials all failed, which has probability

(1-p)^k

. The formula for

P(G \le k)

is then just

1 - P(G > k)

.

Example 1: Ludo

A player throws a die repeatedly until she gets a 1. Here $p = \frac{1}{6}$ . "Fewer than $n$ throws" means $G \le n - 1$ , so we use $P(G \le n-1) = 1 - (\frac{5}{6})^{n-1}$ :

Event	Formula	Value
fewer than 6 throws	$1 - (\frac{5}{6})^5$	0.5981
fewer than 11 throws	$1 - (\frac{5}{6})^{10}$	0.8385
fewer than 21 throws	$1 - (\frac{5}{6})^{20}$	0.9739

A correction to the source notes. The source writes the exponents 5, 11 and 21 and the values 0.5981, 0.8654, 0.9783. Only the first line fits "fewer than": for "fewer than 11" the exponent must be 10, not 11 (

1 - (5/6)^{11} = 0.8654

is at most 11 throws), and for "fewer than 21" it must be 20. The values above are the corrected ones.

Example 2: alternating free throws

Player 1 is a 40% shooter, Player 2 is a 70% shooter. They alternate, Player 1 starting, until the first basket is scored.

Figure 32. One full round multiplies the probability by 0.6 × 0.3 = 0.18

(a) Player 1 wins before the 3rd round? Favourable: P1 scores on throw 1, or P1 misses, P2 misses, P1 scores: $0.4 + (0.6 \times 0.3 \times 0.4) = 0.472$ .
(b) Player 1 wins overall? Each extra round multiplies by $0.6 \times 0.3 = 0.18$ : an infinite geometric series. $0.4 + (0.18)(0.4) + (0.18)^2(0.4) + \cdots = \frac{0.4}{1 - 0.18} = \frac{0.4}{0.82} \approx 0.487$ .

A GP formula you will keep needing

$a + a^2 + a^3 + \cdots = \frac{a}{1 - a}$ , valid when $|a| < 1$ . The version starting at 1 is $1 + a + a^2 + \cdots = \frac{1}{1 - a}$ ; the notes\' form is the same series with the first term $a$ .

Example 12Binomial or geometric?

Decide which distribution fits: (a) number of sixes in 12 throws of a die; (b) number of throws until the first six; (c) number of defective bulbs in a box of 20, each defective independently with probability 0.05.

Solution

Situation	Distribution	Why
sixes in 12 throws	$B(12, \frac{1}{6})$	fixed number of trials, count successes
throws until first six	$G(\frac{1}{6})$	count trials until the first success
defective bulbs of 20	$B(20, 0.05)$	fixed 20 independent trials

Example 13Binomial probabilities

A fair coin is tossed 6 times. Find the probability of (a) exactly 3 heads, (b) at least 5 heads, (c) at most 1 head.

Solution

(a) $\binom{6}{3}/2^6 = \frac{20}{64} = \frac{5}{16}$ .
(b) $\frac{\binom{6}{5} + \binom{6}{6}}{64} = \frac{6 + 1}{64} = \frac{7}{64}$ .
(c) $\frac{\binom{6}{0} + \binom{6}{1}}{64} = \frac{1 + 6}{64} = \frac{7}{64}$ (same as (b) by symmetry).

Example 14Geometric probabilities

A basketball player scores a basket with probability 0.3 on each independent attempt. Find (a) P(first basket on the 4th attempt), (b) P(first basket within 3 attempts), (c) P(more than 5 attempts are needed).

Solution

(a) $P(G = 4) = (0.7)^3(0.3) = 0.343 \times 0.3 = 0.1029$ .
(b) $P(G \le 3) = 1 - (0.7)^3 = 1 - 0.343 = 0.657$ .
(c) $P(G > 5) = (0.7)^5 = 0.16807$ .

Revision overview

Bernoulli(p)One trial: success 1 (prob p), failure 0
n independent trials: 2ⁿ outcomes

Sequence probabilityP = p^w (1−p)^(n−w)
Only the number of 1s matters

Coin examplesFair: 1/32 each, tails 1,5,10,10,5,1
Biased 2/3: sums to 243/243

Bernoulli, Binomial, GeometricWeek 0 · Part 7

Binomial B(n, p)Number of successes, range 0..n
P(B = k) = C(n,k) pᵏ (1−p)ⁿ⁻ᵏ
Peak at floor(p(n+1))

Geometric G(p)Trials to first success, range 1, 2, …
P(G = k) = (1−p)ᵏ⁻¹ p
P(G ≤ k) = 1 − (1−p)ᵏ

GP suma + a² + … = a/(1 − a)
Alternating throws: 0.4/0.82

Revision: Bernoulli, binomial and geometric on one page

Next · Random Variables · Part 1Random Variables, Range and PMF

08

Random Variables · Part 1

Random Variables, Range and PMF

From outcomes to numbers. A random variable turns each outcome into a real number; its PMF says how likely each value is.

8.1Random variables

Random variables are numerical values computed from the outcomes of an experiment.

Definition: random variable

A random variable is a function with domain = the sample space and range = the real numbers: a function from the sample space to the real line.

Links to Maths Week 1. This is exactly the "function" of Maths Week 1 (domain, codomain, range), with the sample space as the domain.

Example: toss a coin, S = {H, T}

$X(H) = 0,\; X(T) = 1$
$Y(H) = -10,\; Y(T) = 200$
$Z(H) = \sqrt{2},\; Z(T) = \pi$

All three are valid random variables, but usually we choose meaningful functions.

Figure 33. A random variable turns outcomes into numbers

Example: throw a die

$S = \{1, \dots, 6\}$ . Pick any 6 real numbers $x_1, \dots, x_6$ and set $X(1) = x_1, \dots, X(6) = x_6$ . If the $x_i$ are distinct, $X$ is one-to-one: essentially the same as the sample space. But the $x_i$ need not be distinct: for instance the random variable $E$ with $E(2) = E(4) = E(6) = 1$ and $E(1) = E(3) = E(5) = 0$ .

Random variables and events

If $X$ is a random variable, then $(X < x) = \{s \in S : X(s) < x\}$ is an event for every real $x$ . So $(X < x)$ , $(X = x)$ , $(X \le x)$ , $(X \ge x)$ are all events.

Figure 34. On a die with X(i) = i: events described through X

$X = 1$ : the event "die shows 1". $X < 4$ : the event $\{1, 2, 3\}$ . Any event can be expressed in terms of $X$ ; e.g. the event $\{2, 5\}$ is $(X = 2) \cup (X = 5)$ .

Why random variables?

For one IPL powerplay over, several random variables can be defined: number of runs, number of actual deliveries, number of wickets, number of boundaries, number of dot balls, and so on.
Instead of assigning probabilities to entire outcomes, we study random variables and assign probabilities to events defined through them.
This reduces the detail in the outcome to something simpler. With limited data, only such random variables can realistically be studied.

8.2Range and discrete random variables

Definitions

The range of a random variable is the set of values taken by it: a subset of the real line. Given a random variable\'s definition, the first task is always to find its range. A random variable is discrete if its range is a discrete set.

Throw a die, $X$ = number shown: range $= \{1, 2, 3, 4, 5, 6\}$ .
$E = 0$ if odd, $1$ if even: range $= \{0, 1\}$ .

8.3PMF and its properties

Definition: probability mass function

The PMF of a discrete random variable $X$ with range $T$ is the function $f_X : T \to [0, 1]$ , $f_X(t) = P(X = t)$ for $t \in T$ . Here $(X = t)$ is an event: $P(X = t) = P(\text{all outcomes that result in } X \text{ taking value } t)$ .

For a subset $B$ of $T$ , the event $(X \in B)$ has probability $P(X \in B) = \sum_{t \in B} P(X = t) = \sum_{t \in B} f_X(t)$ .

Properties of a PMF

0 \le f_X(t) \le 1 \qquad \sum_{t \in T} f_X(t) = 1

Often, random variables and their PMFs are studied without much mention of the experiment or sample space: events are defined using random variables, and probabilities are evaluated using the PMF.

Example: fair coin flipped 3 times

Let $X$ = number of heads, $Y$ = which flip first shows a head (0 stands for "none").

Outcome	P	X	Y
HHH	1/8	3	1
HHT	1/8	2	1
HTH	1/8	2	1
HTT	1/8	1	1
THH	1/8	2	2
THT	1/8	1	2
TTH	1/8	1	3
TTT	1/8	0	0

$X \in \{0, 1, 2, 3\}$ : $f_X(0) = \frac{1}{8}$ , $f_X(1) = \frac{3}{8}$ , $f_X(2) = \frac{3}{8}$ , $f_X(3) = \frac{1}{8}$ .
$Y \in \{0, 1, 2, 3\}$ : $f_Y(0) = \frac{1}{8}$ , $f_Y(1) = \frac{4}{8}$ , $f_Y(2) = \frac{2}{8}$ , $f_Y(3) = \frac{1}{8}$ .

Figure 35. Two different random variables on the same experiment

Example: working with a partial PMF

t	−1	1	2	4
f_X(t)	0.5	0.25	0.125	?

(a) $0.5 + 0.25 + 0.125 + f_X(4) = 1 \Rightarrow f_X(4) = \mathbf{0.125}$ .
(b) Range of $X$ $= \{-1, 1, 2, 4\}$ .
(c) $P(X > 3) = P(X = 4) = 0.125$ .
(d) $P(X < \tfrac{3}{2}) = P(X = 1) + P(X = -1) = 0.25 + 0.5 = 0.75$ .

Figure 36. Use "the PMF adds to 1" to fill in the missing value

Example: a PMF given by a formula

$f_X(k) = \frac{c}{3^k}$ for $k = 1, 2, 3, \dots$ Find $c$ , $P(X > 10)$ , and $P(X > 10 \mid X > 5)$ .

(a) Find $c$ : the terms form a GP with first term $\frac{1}{3}$ : $c\left[\frac{1}{3} + \frac{1}{9} + \frac{1}{27} + \cdots\right] = c \cdot \frac{1/3}{1 - 1/3} = \frac{c}{2} = 1$ , so $c = 2$ .
(b) $P(X > 10) = \frac{2}{3^{11}} + \frac{2}{3^{12}} + \cdots = 2 \cdot \frac{1/3^{11}}{1 - 1/3} = \frac{1}{3^{10}}$ .
(c) Since $(X > 10) \subseteq (X > 5)$ : $P(X > 10 \mid X > 5) = \frac{P(X > 10)}{P(X > 5)} = \frac{1/3^{10}}{1/3^5} = \frac{1}{3^5}$ .

Figure 37. f(k) = 2/3^k: the tail P(X > 10) is tiny

Memoryless.

P(X > 10 \mid X > 5) = P(X > 5)

: this is exactly a geometric distribution with

p = \frac{2}{3}

(

P(X = k) = (\tfrac{1}{3})^{k-1} \cdot \tfrac{2}{3}

), and geometric random variables "forget" how long they have already waited.

Example 15Build a PMF

Two fair coins are tossed. Let $X$ = number of heads. Find the range and the PMF of $X$ , and $P(X \ge 1)$ .

Solution

t	0	1	2
f_X(t)	1/4	2/4	1/4

Range $\{0, 1, 2\}$ ; $f_X$ adds to 1 ✓. $P(X \ge 1) = \frac{2}{4} + \frac{1}{4} = \frac{3}{4}$ (or $1 - f_X(0)$ ).

Example 16Is it a valid PMF?

Which of these can be a PMF on $\{1, 2, 3\}$ ? (a) $(0.2, 0.5, 0.3)$ (b) $(0.4, 0.4, 0.4)$ (c) $(0.6, -0.1, 0.5)$ (d) $(0.5, 0.25, 0.2)$ .

Solution

Option	Check	Valid?
(a)	$0.2 + 0.5 + 0.3 = 1$ , all in $[0, 1]$	Yes
(b)	sum $= 1.2$	No
(c)	a negative value $-0.1$	No
(d)	sum $= 0.95$	No

Revision overview

Random variableFunction: sample space → ℝ
Numbers computed from outcomes

Events from X(X < x), (X = x), (X ≤ x), (X ≥ x)
are events

Why use themSimplify the outcome
Work with limited data

Random Variables and PMFsWeek 0 · Part 8

Range, discreteRange = values taken (first task)
Discrete: range is a discrete set

PMFf_X(t) = P(X = t)
P(X ∈ B) = Σ f_X(t)

Properties0 ≤ f_X(t) ≤ 1
Σ f_X(t) = 1
Use it to find a missing value

Revision: random variables and PMFs on one page

Next · Random Variables · Part 2Common Distributions

09

Random Variables · Part 2

Common Distributions

Seven named distributions cover most situations in the course. For each one: its range, its PMF, and the story it tells. The Poisson distribution gets its own application.

The symbol $\sim$ reads as "is distributed as". Seven standard distributions are used throughout the course.

Figure 38. The standard distributions and what each one counts

9.1Uniform random variable

$X \sim \text{Uniform}(T)$ , where $T$ is some finite set.
Range: the finite set $T$ .
PMF: $f_X(t) = \frac{1}{|T|}$ for all $t \in T$ ( $|T|$ is the size of $T$ ).
Toss a fair coin: $X \sim \text{Uniform}(\{0, 1\})$ , where 0 = head, 1 = tail. Throw a fair die: $X \sim \text{Uniform}(\{1, \dots, 6\})$ .

9.2Bernoulli random variable

$X \sim \text{Bernoulli}(p)$ , $0 \le p \le 1$ .
Range: $\{0, 1\}$ .
PMF: $f_X(0) = 1 - p$ , $f_X(1) = p$ .
Example: a Bernoulli trial, where $p$ is the probability of success.

9.3Binomial random variable

$X \sim \text{Binomial}(n, p)$ , $n$ a positive integer, $0 \le p \le 1$ .
Range: $\{0, 1, 2, \dots, n\}$ .
PMF: $f_X(k) = \binom{n}{k}p^k(1-p)^{n-k}$ .
Meaning: the number of successes in $n$ independent Bernoulli( $p$ ) trials. Validity check: every term is $\ge 0$ and $\sum_k \binom{n}{k}p^k(1-p)^{n-k} = 1$ .

9.4Geometric random variable

$X \sim \text{Geometric}(p)$ , $0 < p \le 1$ .
Range: $\{1, 2, 3, \dots\}$ .
PMF: $f_X(k) = (1-p)^{k-1}p$ .
Meaning: the number of trials needed for the first success. Validity check: use the sum of a GP.

On the range of $p$ . The source writes

0 \le p \le 1

, but for

p = 0

the first success never comes (the PMF would be 0 everywhere and not add to 1), so we need

p > 0

.

9.5Negative binomial random variable

A generalisation of the geometric random variable.
$X \sim \text{Negative Binomial}(r, p)$ , $r$ a positive integer, $0 < p \le 1$ .
Range: $\{r, r+1, r+2, \dots\}$ .
PMF: $f_X(k) = \binom{k-1}{r-1}(1-p)^{k-r}p^r$ .
Meaning: the number of trials needed for the $r$ -th success in repeated independent Bernoulli( $p$ ) trials. $r = 1$ gives exactly the geometric random variable.

Why this PMF. The

r

-th success lands exactly on trial

k

: the last trial is a success, and the earlier

k - 1

trials contain exactly

r - 1

successes, which can be placed in

\binom{k-1}{r-1}

ways.

9.6Poisson random variable

$X \sim \text{Poisson}(\lambda)$ , where $\lambda > 0$ is a real parameter.
Range: $\{0, 1, 2, 3, \dots\}$ .
PMF: $f_X(k) = \frac{e^{-\lambda}\lambda^k}{k!}$ .
Validity check: $e^{\lambda} = 1 + \lambda + \frac{\lambda^2}{2!} + \frac{\lambda^3}{3!} + \cdots$ , so $\sum_k \frac{e^{-\lambda}\lambda^k}{k!} = e^{-\lambda}e^{\lambda} = 1$ .

k	0	1	2	3	4
$f_X(k)$	$e^{-\lambda}$	$e^{-\lambda}\lambda$	$e^{-\lambda}\lambda^2/2$	$e^{-\lambda}\lambda^3/6$	$e^{-\lambda}\lambda^4/24$

9.7Hypergeometric random variable

A population of $N$ persons, with $r$ of Type 1 and $N - r$ of Type 2. Select $m$ persons uniformly at random without replacement. $X$ = number of Type 1 persons selected: $X \sim \text{HyperGeo}(N, r, m)$ .
Range: $\max(0,\, m - (N - r)) \dots \min(r, m)$ .
PMF: $f_X(k) = \frac{\binom{r}{k}\binom{N-r}{m-k}}{\binom{N}{m}}$ .

Figure 39. Three range examples

$N = 100, r = 50, m = 20$ : $X \in \{0, 1, \dots, 20\}$ .
$N = 100, r = 10, m = 20$ : $X \in \{0, 1, \dots, 10\}$ .
$N = 100, r = 90, m = 20$ : $X \in \{10, 11, \dots, 20\}$ . The last case has a non-zero lower bound: with only 10 Type-2 people available, at least 10 of your 20 picks must be Type 1.

9.8Application of the Poisson random variable

Events occurring over a period of time: arrival of a person to a booking-counter queue, arrival of a visitor to a website, emission of a particle by radioactive decay, a meteorite entering the earth's atmosphere.

What do these have in common?

The arrival rate can be assumed constant.
Given one arrival, the time to the next arrival can be assumed independent of the past.
Under these assumptions, the number of arrivals in a fixed period becomes a Poisson random variable.

Example: radioactive decay

In 2608 time intervals of 7.5 seconds each, emissions of alpha particles were recorded. The source table below is cut off at 8 particles:

Particles	0	1	2	3	4	5	6	7	8
Intervals	57	203	383	525	532	408	273	139	45
Fraction	0.022	0.078	0.147	0.201	0.204	0.156	0.105	0.053	0.017

The table is incomplete. Those counts add up to only 2565, not 2608. The remaining 43 intervals had 9, 10 or 11+ particles (27, 10 and 6 intervals in the full data). Using only the printed rows would give a mean of 3.71; with all 2608 intervals the emission rate is

\lambda = \frac{\text{total particles}}{2608} \approx 3.87

(the notes quote 3.8673).

\lambda = \frac{\text{total number of particles}}{2608} \approx 3.87, \qquad P(\text{number of particles} = k) = \frac{e^{-\lambda}\lambda^k}{k!}

Figure 40. Observed fractions (bars) against the Poisson model (line)

The time of the next emission is independent of the past, and the observed fractions are close to the probabilities predicted by the Poisson model, as the figure shows.

Example 17Pick the distribution

Name the distribution: (a) number of heads in 8 tosses; (b) number of tosses to get the 3rd head; (c) number of calls arriving at a help desk in an hour; (d) 5 cards drawn without replacement, number of aces; (e) a fair die, the number shown.

Solution

Situation	Distribution
(a)	$\text{Binomial}(8, \tfrac{1}{2})$
(b)	$\text{Negative Binomial}(3, \tfrac{1}{2})$
(c)	$\text{Poisson}(\lambda)$
(d)	$\text{HyperGeo}(52, 4, 5)$
(e)	$\text{Uniform}(\{1, \dots, 6\})$

Example 18Poisson probabilities

A website gets visitors at a constant rate of 2 per minute, so the number of visitors in a minute is $\text{Poisson}(2)$ . Find the probability of exactly 0, 1 and 2 visitors, and of at least one visitor.

Solution

$P(0) = e^{-2} \approx 0.1353$ .
$P(1) = 2e^{-2} \approx 0.2707$ .
$P(2) = \frac{4}{2}e^{-2} = 2e^{-2} \approx 0.2707$ .
$P(\ge 1) = 1 - e^{-2} \approx 0.8647$ .

Example 19Hypergeometric probability

An urn has 6 red and 4 blue marbles. 3 are drawn without replacement. Find the probability that exactly 2 are red.

Solution

$N = 10, r = 6, m = 3, k = 2$ : $f_X(2) = \frac{\binom{6}{2}\binom{4}{1}}{\binom{10}{3}} = \frac{15 \times 4}{120} = \frac{1}{2}$ .
Range: $\max(0, 3 - 4) = 0$ to $\min(6, 3) = 3$ .

Revision overview

Uniform(T)Finite set T, f(t) = 1/|T|
Fair coin, fair die

Bernoulli(p)Range {0, 1}
f(0) = 1 − p, f(1) = p

Binomial(n, p)Range 0..n
f(k) = C(n,k) pᵏ (1−p)ⁿ⁻ᵏ

Common DistributionsWeek 0 · Part 9

Geometric(p)Range 1, 2, …
f(k) = (1−p)ᵏ⁻¹ p

NegBin(r, p)Range r, r+1, …
f(k) = C(k−1, r−1)(1−p)ᵏ⁻ʳ pʳ
r = 1 is geometric

Poisson(λ) · HyperGeoPoisson: e⁻λ λᵏ / k!
HyperGeo: C(r,k)C(N−r,m−k)/C(N,m)

Revision: the seven distributions on one page

Next · Random Variables · Part 3Functions of One Random Variable

10

Random Variables · Part 3

Functions of One Random Variable

Applying a function to a random variable gives a new random variable. The only task is to find its PMF: group the values that the function sends to the same place and add their probabilities.

10.1A function of a random variable

$X$ is a random variable with range $T$ and PMF $f_X(t)$ . $X$ is a function from the sample space to $T$ ; $T$ is a subset of the real line.
Let $f(x)$ be a function from the real line to the real line. Then $f(X)$ is a composition of two functions: sample space $\to T \to$ real line.
So $f(X)$ is also a random variable in the same probability space.

The general case

f_{f(X)}(a) = P(f(X) = a) = P\big(X \in \{t : f(t) = a\}\big) = \sum_{t \,:\, f(t) = a} f_X(t)

How to use it

To find the PMF of $f(X)$ at a value $a$ , collect every $t$ in the range of $X$ that $f$ maps to $a$ , and add up their probabilities.

Figure 41. X uniform on {−1, 0, 1, 2}, f(X) = X²: the values −1 and 1 both land on 1

Worked example

Let $X$ take the values $-1, 0, 1, 2$ , each with probability $\frac{1}{4}$ . Find the PMF of $Y = X^2$ .

Value of Y	t with t² equal to it	$f_Y$
0	$t = 0$	$\frac{1}{4}$
1	$t = -1, 1$	$\frac{1}{4} + \frac{1}{4} = \frac{1}{2}$
4	$t = 2$	$\frac{1}{4}$

Check.

\frac{1}{4} + \frac{1}{2} + \frac{1}{4} = 1

. Notice the range shrank from 4 values to 3: when

f

is not one-to-one, different values of

X

merge into one value of

f(X)

and their probabilities add.

Links to Maths Week 1. This is exactly "injective or not": a one-to-one

f

keeps every value separate (the PMF is just relabelled), a many-to-one

f

adds probabilities.

Example 20PMF of 2X + 1

$X$ has PMF $f_X(0) = 0.2$ , $f_X(1) = 0.5$ , $f_X(2) = 0.3$ . Find the PMF of $Y = 2X + 1$ .

Solution

Y	1	3	5
$f_Y$	0.2	0.5	0.3

$f(x) = 2x + 1$ is one-to-one, so each value of $Y$ comes from exactly one $t$ and the probabilities are simply carried over.

Example 21A many-to-one function

$X$ is the number shown on a fair die. Let $Y = (X - 3)^2$ . Find the PMF of $Y$ .

Solution

$X$	1	2	3	4	5	6
$Y = (X - 3)^2$	4	1	0	1	4	9

Y	0	1	4	9
$f_Y$	$\frac{1}{6}$	$\frac{2}{6}$	$\frac{2}{6}$	$\frac{1}{6}$

Sum $= \frac{1 + 2 + 2 + 1}{6} = 1$ ✓.

Revision overview

Ideaf(X) is a composition:
sample space → T → ℝ
So f(X) is a random variable

Formulaf_{f(X)}(a) = P(X ∈ {t : f(t) = a})
= Σ f_X(t) over f(t) = a

Functions of a Random VariableWeek 0 · Part 10

How to use itCollect every t that f sends to a
Add their probabilities

ExampleX² on {−1, 0, 1, 2}:
f(0) = ¼, f(1) = ½, f(4) = ¼

Revision: functions of a random variable on one page

Next · All topicsWeek 0 Summary Sheet

11

All topics

Week 0 Summary Sheet

Everything from Week 0 on one sheet, for a last-minute revision.

Probability toolkit

Events and sets

Event = subset of $S$ ; $n$ outcomes give $2^n$ events
$(A \cup B)^c = A^c \cap B^c$ , $(A \cap B)^c = A^c \cup B^c$
Disjoint: empty intersection

Axioms and properties

$P(S) = 1$ ; disjoint events add
$P(E^c) = 1 - P(E)$ ; $E \subseteq F \Rightarrow P(E) \le P(F)$
$P(E \cup F) = P(E) + P(F) - P(E \cap F)$
$P(A \setminus B) = P(A) - P(A \cap B)$

Counting and uniform

$P(E) = \frac{|E|}{|S|}$
D(n): 0, 1, 2, 9, 44, 265, 1854, 14833
$P(\text{derangement}) \to \frac{1}{e} \approx 37\%$

Conditioning

$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$ ; $P(A \cap B) = P(B)P(A \mid B)$
$P(A) = \sum P(A \mid B_i)P(B_i)$
$P(B \mid A) = \frac{P(B)P(A \mid B)}{P(A)}$
Independent: $P(A \cap B) = P(A)P(B)$

Standard distributions

Distribution	Counts	Range	PMF
Uniform(T)	equally likely	$T$	$\frac{1}{\|T\|}$
Bernoulli(p)	one trial	$\{0, 1\}$	$1 - p,\; p$
Binomial(n, p)	successes in n trials	$0 \dots n$	$\binom{n}{k}p^k(1-p)^{n-k}$
Geometric(p)	trials to 1st success	$1, 2, \dots$	$(1-p)^{k-1}p$
NegBin(r, p)	trials to r-th success	$r, r+1, \dots$	$\binom{k-1}{r-1}(1-p)^{k-r}p^r$
Poisson(λ)	arrivals in a period	$0, 1, \dots$	$\frac{e^{-\lambda}\lambda^k}{k!}$
HyperGeo(N, r, m)	Type 1, no replacement	$\max(0, m-N+r) \dots \min(r, m)$	$\frac{\binom{r}{k}\binom{N-r}{m-k}}{\binom{N}{m}}$

Random variables

Random variable and PMF

Function: sample space $\to \mathbb{R}$ ; first task: find the range
$f_X(t) = P(X = t)$ ; $0 \le f_X \le 1$ ; $\sum f_X = 1$
$P(X \in B) = \sum_{t \in B} f_X(t)$

Function of a random variable

$f_{f(X)}(a) = \sum_{t:\, f(t) = a} f_X(t)$
Collect every $t$ sent to $a$ , add probabilities

Back to top

Outcome	P	X	Y
HHH	1/8	3	1
HHT	1/8	2	1
HTH	1/8	2	1
HTT	1/8	1	1
THH	1/8	2	2
THT	1/8	1	2
TTH	1/8	1	3
TTT	1/8	0	0

Outcome	P	X	Y
HHH	1/8	3	1
HHT	1/8	2	1
HTH	1/8	2	1
HTT	1/8	1	1
THH	1/8	2	2
THT	1/8	1	2
TTH	1/8	1	3
TTT	1/8	0	0

Statistics forData Science 2

Chapters

Statistical Study and Sample Space

1.1Statistical study of phenomena

1.2Experiment, outcome, sample space

Events

2.1Events

Counting events

Containment and complement

2.2Combining events

2.3Disjoint events and Venn diagrams

De Morgan's laws

Probability, Axioms and Properties

3.1Probability and the axioms

3.2Basic properties of probability

Useful difference formulas

3.3Working with probability spaces

Example: restaurant hiring

Example: fishing town

Example: weather forecast

3.4Inclusion–exclusion for three events, "at least one", and the birthday problem

A. Three events

B. "At least one" is a complement

C. The birthday problem

D. Counting equally likely outcomes

Distributions, Uniform Distribution and Derangements

4.1Distributions and the uniform distribution

Uniform distribution on a finite sample space

Example 1: marbles in an urn

Example 2: sum of two dice

Example 3: mixed-up hats

4.2Derangements

Conditional Probability, Total Probability and Bayes' Theorem

5.1Conditional probability

Example 1: throw a die, given an even number

Example 2: students from three states

Example 3: two children

5.2Law of total probability

General form

Example 1: interest rates and unemployment

Example 2: five coins

5.3Bayes' theorem

Example 1: swine flu test

Example 2: MCQ guessing

Example 3: die then coins

5.4Bayes in practice: base rates, false positives and natural frequencies

A. The vocabulary

B. A worked example

C. What changes the answer

Independence of Events

6.1Independence of events

Points to remember

Example 1: even vs odd (not independent)

Example 2: even vs multiple of 3 (independent)

Example 3: a card from a pack

Typical points of confusion

6.2Mutual independence

Three events

n events

Results that follow

Example: roads between towns

Bernoulli Trials, Binomial and Geometric

7.1Bernoulli trials

A single Bernoulli trial

Repeated Bernoulli trials

The general formula

Example: toss a fair coin 5 times

Example: toss a biased coin 5 times

7.2Binomial distribution

Example: n = 3

Small n, all at once

The general expression

Graph of Binomial(n, p): observations

Example 1: disease testing

Example 2: fair coin, 10 tosses

Example 3: bits flipped in transmission

7.3Geometric distribution

The cumulative formula

Example 1: Ludo

Example 2: alternating free throws

Statistics for
Data Science 2

Outcome	P	X	Y
HHH	1/8	3	1
HHT	1/8	2	1
HTH	1/8	2	1
HTT	1/8	1	1
THH	1/8	2	2
THT	1/8	1	2
TTH	1/8	1	3
TTT	1/8	0	0