Datasets and notebooks

Datasets and notebooks

Every file the course uses, listed by session. Each session has its notebook, the datasets that notebook reads, and a link to the page that documents each dataset in full.

To begin. Download the notebook and its datasets. Put the datasets in a data folder beside the notebook, open the notebook in Jupyter, and run the cells in order. Every notebook uses the relative path data/<file>, so nothing needs editing.
D1 Mathematical foundations Day 1about 90 minutesLondon listings, foundations extract

Follows the mathematics workshop in order. The eight stores from the slides are the running example, so every formula is checked against an answer the deck already prints. London data enters twice, where eight rows are not enough: to show a sampling distribution, and to read $R^2$ out of sample.

D2a What it means for a machine to learn Day 2aabout 100 minutesLondon listingsBank marketing calls

Fits a rule by hand on four properties, so nothing about fitting stays hidden inside a library call. Then asks the harder question of the bank campaign: what evidence would let you claim the rule works on clients you have never seen?

D2b Transparent global rules Day 2babout 110 minutesLondon listingsBank marketing calls

Opens both models. Least squares on London prices, then the same grammar for a probability: the sigmoid, log loss, calibration, and a decision cut-off taken from the prices of the two mistakes rather than from a software default.

D3 Flexible rules and honest comparison Day 3about 110 minutesLondon listingsBank marketing callsRecession panel

Fits models that can bend, and builds the testing discipline that stops them flattering themselves. It ends with two evaluations that reverse each other's verdict on the same data.

D4 Structure and responsible systems Day 4about 110 minutesLondon listingsUS Treasury yield curve

The answer column is taken away. Two questions replace it: which listings behave alike, and what few forces drive nine Treasury yields. Because nothing checks the result automatically, the day is mostly about designing the check.

The datasets

The same file is used on more than one day. Each has one page, giving the question it answers, every column and when it is known, where it came from, what was done to it, and what it does not establish.

London listings54,636 rows, 11 columns. Used on Day 2a, Day 2b, Day 3, Day 4.
Bank marketing calls41,188 rows, 22 columns. Used on Day 2a, Day 2b, Day 3.
Recession panel521 rows, 6 columns. Used on Day 3.
US Treasury yield curve9,143 rows, 10 columns. Used on Day 4.
London listings, foundations extract20,740 rows, 17 columns. Used on Day 1.

What the notebooks do

They teach the lecture rather than solve a project. Each idea is computed from its definition on a case small enough to check by hand, and only then applied at full scale: a line fitted on four properties before 41,188 bank contacts, a tree split scored by hand before a forest of 300, K-means run on five numbers before 32,749 listings.

Each notebook is saved with every output in place, so you can read a whole session without running anything. They need Python with numpy, pandas, matplotlib, scipy and scikit-learn. Nothing is downloaded at run time.

Oxford · United Kingdom Teaching CV
University of Oxford