Probability & Statistics · Lecture 1 of 5

Probability theory

What can happen, and how should information change what we believe?

1 Describe2 Condition3 Learn

Developed from Ian Crawford’s lecture material. Notes, slides and PDF: fatih.ai/probabilityandstatistics

Opening question

A cyclist tests positive. How likely is it that the cyclist uses EPO?

A hypothetical cycling population and an imperfect test:

$15\%$ of cyclists use EPO

$90\%$ of users test positive

$12\%$ of non-users also test positive

Your estimateVote now and keep your answer. Part 3 calculates it.

A

\approx90\%

B

\approx57\%

C

\approx15\%

Why start here

Economists read signals backwards all the time: a test result, a credit score, a qualification. Today builds the rules that make that reading exact.

Lecture 1 · The route

Describe uncertainty, condition on information, learn from evidence

Part 1

Describe: events and their probabilities

What exactly could happen, and how do we give events probabilities that are consistent with one another?

sample spaceeventaxiomsunion rule

Notes 1.1

Part 2

Condition: how information changes a probability

When we learn that one event has happened, how should the probability of another change?

conditional probabilityproduct ruletotal probability

Notes 1.2

Part 3

Learn: independence and Bayes’ rule

When does information leave a probability unchanged, and how should evidence change what we believe?

independenceBayes’ rulepriorposteriorbase rate

Notes 1.3

By the end of this lecture you can

Write the sample space and events for an experiment, and derive the complement and union rules from the axioms.
Calculate a conditional probability as a share within the information event, keeping track of which way it runs.
Combine group rates by total probability, and check whether two events are independent.
Update a prior with Bayes’ rule, and explain why a positive EPO test means about $57\%$ , not $90\%$ .

Part 1 of 3

Describe: events and their probabilities

What exactly could happen, and how do we give events probabilities that are consistent with one another?

Key terms

sample spaceeventaxiomsunion rule

Notes 1.1 ↗

Why it matters

Every economic question about risk, from becoming unemployed to a loan default, starts by stating exactly which outcome is uncertain. The axioms keep the probabilities we attach consistent with one another, whatever the weights.

In this part

1.1.1–3Outcomes, events and set operations
1.1.4The axioms and the complement rule
1.1.5The union rule
▶Animation: where each term of the union rule comes from

1.1.1–3Outcomes and events

Events are sets of outcomes, so “not”, “and”, “or” are set operations

Definition

Sample space and events

An outcome $\omega$ is one possible result; the sample space $\Omega$ collects them all. An event $A\subseteq\Omega$ is the set of outcomes that make a statement true.

Statement	Event	One die: $B$ even, $C$ prime
not $B$	$B^c$	$\{1,3,5\}$
$B$ and $C$	$B\cap C$	$\{2\}$
$B$ or $C$	$B\cup C$	$\{2,3,4,5,6\}$

“Or” is inclusive. Events are disjoint when $A\cap B=\varnothing\text{.}$

A picture of “and”, “or” and “not”

The left oval is the even event

B

; the right oval is the prime event

C

. Only

2

lies in both. The union contains every labelled outcome except

1

. The complement of even contains

1,3,5

. Oval areas do not encode probabilities: the labelled outcomes determine membership.

1.1.4The axioms

Three axioms fix how probabilities combine, not what they are

Definition

The axioms of probability

$P(A)\ge0$ for every event: no negative mass.
$P(\Omega)=1\text{:}$ the total is fixed.
Disjoint events add: $P\bigl(\bigcup_i A_i\bigr)=\sum_i P(A_i)$ for pairwise disjoint $A_i\text{.}$

Worked example a fair and a loaded die

Fair: $P(\text{even})=3\times\tfrac16=\tfrac12\text{.}$ Loaded, $0.1$ on faces 1–5 and $0.5$ on face 6: $P(\text{even})=0.1+0.1+0.5=0.7\text{.}$ Both satisfy the axioms.

Key formula Result (1)

Complement rule

P(A^c)=1-P(A)

In words $A$ and $A^c$ are disjoint and fill $\Omega\text{,}$ so their probabilities add to $1\text{.}$

Common mistake

Probability zero is not impossibility: one exact bulb lifetime can have probability $0$ without being impossible.

1.1.5Adding probabilities

Adding two events counts their overlap twice

Key formula Result (2)

Union rule

P(A\cup B)=P(A)+P(B)-P(A\cap B)

In words add both, then remove the overlap counted twice.

Needs any two events; when they are disjoint the last term is zero.

Disjointness lets you add; overlap must be subtracted once.

Check

Fair die, $B$ even, $C$ prime. What is $P(B\cup C)$ ?

Answer

$\tfrac36+\tfrac36-\tfrac16=\tfrac56\text{.}$ The plain sum, $1\text{,}$ counts the face $2$ twice.

Final frame: in a unit square, A and B are drawn to scale and their union is outlined. Beside them, P(A ∪ B) = P(A) + P(B) − P(A ∩ B), with 0.35 = 0.25 + 0.15 − 0.05 underneath.

Part 2 of 3

Condition: how information changes a probability

When we learn that one event has happened, how should the probability of another change?

Key terms

conditional probabilityproduct ruletotal probability

Notes 1.2 ↗

So far a model $(\Omega,\mathcal F,P)$ , three axioms, and the complement and union rules.

Why it matters

Economic information usually arrives as rates within groups: defect rates per production line, unemployment among graduates, default among borrowers with a given score. Answering the question asked needs the right reference group, and recombining groups needs their sizes as weights.

In this part

1.2.1Two children: information and the reference group
1.2.2Conditional probability: restrict, then rescale
▶Animation: why conditioning divides by the probability of the information
1.2.3Reversing the condition
1.2.4Total probability: a weighted average of group rates
▶Animation: total probability as a weighted average

1.2.1Information changes the question

Learning something changes the group in which we calculate

Two children, each a boy or a girl with probability $\tfrac12\text{,}$ independently, older child first:

\Omega=\{(b,b),(b,g),(g,b),(g,g)\}

Each family has probability $\tfrac14\text{.}$ Let $A$ be “two boys” and $B$ “at least one boy”.

Predict first

We learn that $B$ is true. Are two boys now more or less likely than $\tfrac14$ ?

Answer

More: $(g,g)$ is ruled out, so three equally likely families remain and $P(A\mid B)=\tfrac13\text{.}$

No family changes. What changes is the group in which we calculate the share.

1.2.2The definition

Conditional probability: restrict to B, then rescale B to one

Key formula Definition (3)

Conditional probability

P(A\mid B)=\frac{P(A\cap B)}{P(B)}

In words the share of $B$ that also lies in $A\text{:}$ conditional probability.

Needs $P(B)>0\text{.}$

Restrict to $B\text{:}$ only $A\cap B$ survives.
Rescale by $1/P(B)\text{:}$ the new total is $1\text{.}$

Key formula Identity (4)

Product rule

P(A\cap B)=P(A\mid B)\,P(B)

In words the definition turned around: a share of a share.

Worked example

$P(B)=0.6$ and $P(A\cap B)=0.1\text{,}$ so

P(A\mid B)=\frac{0.1}{0.6}=\frac16,\qquad P(A^c\mid B)=\frac56.

Final frame: the unit square with B as the bottom band, of probability 0.6, and A as the left part of each band; the overlap has probability 0.1. Beside it, P(A | B) = P(A ∩ B)/P(B) = 0.1/0.6 = 1/6, and the product rule P(A ∩ B) = P(A | B)P(B) = 1/6 × 0.6 = 0.1.

1.2.3Direction matters

Reversing the condition changes the denominator

100 hypothetical cases
	$B$	$B^c$	Total
$A$	$10$	$10$	$20$
$A^c$	$50$	$30$	$80$
Total	$60$	$40$	$100$

Within column $B\text{:}$ $P(A\mid B)=10/60=\tfrac16$
Within row $A\text{:}$ $P(B\mid A)=10/20=\tfrac12$

Common mistake

Treating $P(A\mid B)$ and $P(B\mid A)$ as the same number. They share a numerator, $P(A\cap B)\text{,}$ but divide by different groups.

Reading claims

“Most students are Democrats” does not say that most Democrats are students. Read the condition first: it names the group you divide by.

1.2.4Recovering the whole

Total probability weights each group’s rate by the group’s size

Key formula Result (5)

Law of total probability

P(A)=\sum_i P(A\mid B_i)\,P(B_i)

In words each group’s rate times its share, added over the groups.

Needs groups $B_i$ that cover $\Omega$ without overlap (a partition), each $P(B_i)>0\text{.}$

Worked example the 100 cases

$P(A)=\tfrac16(0.6)+\tfrac14(0.4)=0.1+0.1=0.2$

Check

Line 1 makes $40\%$ of items with defect rate $0.05\text{;}$ line 2 makes $60\%$ with rate $0.10\text{.}$ What share of all items is defective?

Answer

$0.05(0.4)+0.10(0.6)=0.02+0.06=0.08\text{.}$ The plain average, $0.075\text{,}$ would treat the lines as equal in size.

Final frame: the unit square split into B (0.6) and B complement (0.4). Inside each group, A takes a width equal to its rate, 1/6 and 1/4, so each piece has area 0.1. Beside it, P(A) = 1/6 × 0.6 + 1/4 × 0.4 = 0.1 + 0.1 = 0.2; averaging the rates would give about 0.208.

Part 3 of 3

Learn: independence and Bayes’ rule

When does information leave a probability unchanged, and how should evidence change what we believe?

Key terms

independenceBayes’ rulepriorposteriorbase rate

Notes 1.3 ↗

So far conditioning restricts to the information event and rescales it; total probability weights group rates by group sizes.

Why it matters

Decisions rest on imperfect signals read backwards: a positive drug test, a credit score, rare evidence in court. Bayes’ rule makes the reversal exact, and independence says when information is irrelevant and when probabilities may be multiplied.

In this part

1.3.1Independence: an unchanged share
▶Animation: independence as an unchanged share
1.3.2Bayes’ rule: one joint probability, two directions
1.3.3The EPO test, step by step
▶Animation: every number in the posterior
1.3.4–5Base rates and the Sally Clark case

1.3.1Independence

Independent events: learning B leaves the probability of A unchanged

Key formula Definition (6)

Independence

P(A\cap B)=P(A)\,P(B)

In words equivalently $P(A\mid B)=P(A)\text{:}$ the share of $A$ inside $B$ equals its share overall.

Needs the equivalent form needs $P(B)>0\text{.}$

Independence is a fact about overlap, checked by comparing probabilities.

Worked example

$P(A)=0.30$ and $P(B)=0.40\text{.}$ Independence needs $P(A\cap B)=0.30\times0.40=0.12\text{,}$ and then

P(A\mid B)=\frac{0.12}{0.40}=0.30=P(A).

Common mistake

Disjoint is not independent. If $A\cap B=\varnothing$ and both have positive probability, learning $B$ rules $A$ out: $P(A\mid B)=0\ne P(A)$ .

Final frame: B as the bottom band (0.4) and A taking 0.3 of the width both inside and above B, so their overlap is 0.12. Beside it, P(A | B) = 0.12/0.4 = 0.3, the cases of more overlap (P(A | B) = 0.5) and none (P(A | B) = 0), and the boxed result P(A ∩ B) = P(A)P(B) = 0.3 × 0.4 = 0.12.

1.3.2Reversing a condition

Bayes’ rule rebuilds the reverse condition from the known one

Key formula Result (7)

Bayes’ rule

P(H\mid E)=\frac{P(E\mid H)\,P(H)}{P(E)}

In words write $P(H\cap E)$ both ways with (4), equate, divide by $P(E)\text{.}$

Needs $P(E)>0\text{;}$ the denominator comes from (5).

Definition

Prior, likelihood, posterior

Prior $P(H)\text{:}$ before the evidence.
Likelihood $P(E\mid H)\text{:}$ how readily $H$ produces $E\text{.}$
Posterior $P(H\mid E)\text{:}$ after the evidence.

Evidence favours $H$ only if it is more likely under $H$ than under the alternatives.

1.3.3Worked example

A positive test: count both routes to a positive result

$D\text{:}$ uses EPO, $P(D)=0.15\text{.}$ $+\text{:}$ positive test, with $P(+\mid D)=0.90$ and $P(+\mid D^c)=0.12\text{.}$

Worked example

Users who test positive: $0.90\times0.15=0.135$
Non-users who test positive: $0.12\times0.85=0.102$
All positives, by (5): $0.135+0.102=0.237$
By (7): $P(D\mid +)=0.135/0.237\approx0.57$

In 1,000 hypothetical cyclists
	$+$	$-$	Total
Users	$135$	$15$	$150$
Non-users	$102$	$748$	$850$
Total	$237$	$763$	$1{,}000$

Row of users: $135/150=0.90\text{.}$
Column of positives: $135/237\approx0.57\text{.}$

$0.90$ starts with users; $0.57$ starts with positive tests.

Final frame: the 237 positive tests from a hypothetical group of 1,000 cyclists gathered into one bar, 135 from EPO users and 102 from non-users. Beside it, P(D | +) = 135/(135 + 102) ≈ 0.57, which is not the same as P(+ | D) = 135/150 = 0.90.

1.3.4–5Evidence and assumptions

Rare evidence is not a small posterior: base rates and assumptions decide

Definition

Base rate

The prior share of a condition in the population. With only $1\%$ users, the same test gives $\dfrac{0.01(0.90)}{0.01(0.90)+0.99(0.12)}\approx7\%\text{.}$

Worked example the Sally Clark case

An argument squared a single-death frequency of sudden infant death: $(1/8{,}500)^2=1/72{,}250{,}000\text{.}$

Two separate errors

Unsupported independence. Squaring needs $P(S_2\mid S_1)=P(S_2)$ ; a shared family risk makes it larger.
The prosecutor’s fallacy. A small $P(E\mid H)$ is not a small $P(H\mid E)\text{:}$ rival explanations must be compared, with their priors.

Why economists care

The same logic governs screening, fraud detection and credit scoring: a signal’s meaning depends on how common each type is.

Review · How the ideas connect

One model, three moves: add, condition, reverse

Part 1 · Describe

Part 2 · Condition

Part 3 · Learn

assign weightsderive

Outcomes

\Omega\text{,}

events

A\subseteq\Omega

Axioms of probability

Complement and union rules (1) (2)

rescale to 1restrict to Bturn aroundadd over groups

Information:

B

occurred

Conditional probability (3)

Product rule (4)

Total probability (5)

no changeboth directionsdenominatorupdate

Independence (6):

P(A\mid B)=P(A)

Bayes’ rule (7)

Prior

\to

posterior: the EPO test

Review · Summary sheet

What to take away: the terms and the seven formulas

Key terms

1 Describe

outcome $\omega\text{:}$ one possible result.

sample space $\Omega\text{:}$ the set of all outcomes the model considers.

event $A\subseteq\Omega\text{:}$ the outcomes that make a statement true.

disjoint $A\cap B=\varnothing\text{:}$ no outcome in both.

The axioms of probability nonnegativity, $P(\Omega)=1\text{,}$ and disjoint events add.

2 Condition

conditional probability $P(A\mid B)\text{:}$ the probability of $A$ within the information event $B\text{.}$

partition disjoint groups $B_i$ that together cover $\Omega\text{.}$

3 Learn

Independence $P(A\cap B)=P(A)P(B)\text{:}$ learning one event leaves the other’s probability unchanged.

Prior $P(H)\text{:}$ belief in a hypothesis before the evidence.

Likelihood $P(E\mid H)\text{:}$ how readily $H$ produces the evidence.

Posterior $P(H\mid E)\text{:}$ belief after the evidence.

Base rate the prior share of a condition in the population tested.

Key formulas

1 Describe

(1)Complement rule Result

P(A^c)=1-P(A)

(2)Union rule Result

P(A\cup B)=P(A)+P(B)-P(A\cap B)

any two events

2 Condition

(3)Conditional probability Definition

P(A\mid B)=P(A\cap B)/P(B)

P(B)>0\text{.}

(4)Product rule Identity

P(A\cap B)=P(A\mid B)\,P(B)

(5)Law of total probability Result

P(A)=\sum_i P(A\mid B_i)\,P(B_i)

a partition

B_1,B_2,\dots

with each

P(B_i)>0

3 Learn

(6)Independence Definition

P(A\cap B)=P(A)\,P(B)

the definition; equivalently

P(A\mid B)=P(A)

if

P(B)>0

(7)Bayes’ rule Result

P(H\mid E)=P(E\mid H)\,P(H)/P(E)

P(E)>0\text{,}

with

P(E)

from (5)

Review · Back to the opening question

The cyclist: about 57%, not 90%

Result the opening question

$0.90$ is $P(+\mid D)\text{:}$ it starts with users. We wanted the reverse, which starts with positive tests:

P(D\mid +)=\frac{0.135}{0.237}=\frac{45}{79}\approx0.57

The test moved the probability from $15\%$ to about $57\%\text{:}$ strong evidence, far from certainty.

Remember

Name the outcomes and the event before calculating.
Every conditional names a group: divide by its size.
Disjointness lets you add; independence lets you multiply.

Before next time

Read notes 1.1–1.3 and the formula reference.
Exercises 1.1.6, 1.2.5 and 1.3.6, with worked solutions.
Watch the four animations not shown today.

Next · Lecture 2

Data, variables & moments. From events to measurements: random variables, distributions, expectation and variance.