Datasets
The data behind each day
Every empirical studio opens with a dataset card, and the shorter tables used by the concept studios and problem sets are printed on the question sheets themselves. This page gathers the whole register: the catalogue first, then the full card for each day, naming the unit of observation, the population and period, the outcome, the variables, the information time, the split and the source.
Row-level files are not published here. The datasets come from
the BIS, Inside Airbnb and the US Bureau of Transportation Statistics. Each publisher sets its
own redistribution terms, and those terms have not been confirmed for onward release from this
site, so the prepared files are supplied with the notebooks in class. Everything you need in
order to judge the evidence, including the exact selection rule and every column, is below.
Dataset catalogue
Use this page to find the data attached to each day. The empirical-studio files are supplied with the corresponding notebook. The shorter datasets in the concept studios and problem sets are printed on the question sheet.
| Day | Empirical-studio data | Datasets printed in the other course papers |
|---|---|---|
| 1 | BIS speech-description role extraction:
bis_role_development.csv, bis_role_audit.csv,
bis_role_deployment.csv. | Dispatch-gate audit; parcel quality-assurance audit; voluntary academic-support outreach; organisation-document classifier. |
| 2 | Inside Airbnb London listing snapshot:
london_price_development.csv,
london_price_validation.csv,
london_price_deployment_features.csv. | Fraud-review and contact-policy tables; held-out delivery errors; payment fraud review; randomised customer outreach. |
| 3 | BTS 2025 domestic flights:
bts_2025_development.csv,
bts_2025_audit_features.csv. | Tree and ranked-queue tables; the after-class questions continue to use the BTS flight study. |
| 4 | BIS speech retrieval corpus: bis_speeches.csv,
bis_chunks.csv, queries.csv. | Depot coordinates; group-wise decisions; selected company returns; selective-investigation outcomes. |
Each empirical-studio dataset card states the unit of observation, target, permitted variables, prediction or retrieval time, split, source, and the questions that use the files. Read that card before opening the notebook.
Day 1 · BIS speech-description role extraction
Used in the Day 1 empirical studio, Extracting only what a source states (60 minutes; pair work with individual votes).
Dataset
BIS speech-description role extraction
Prepared classroom extract; row-level release depends on source terms
| Item | Course definition |
|---|---|
| Files | bis_role_development.csv; bis_role_audit.csv;
bis_role_deployment.csv; day1_application.ipynb |
| Unit | one BIS speech record with one canonical URL and primary-author field |
| Population and period | eligible BIS central-bank speech records; source collection dated 10 September 1996–20 June 2025 |
| Target | one explicitly supported category:
FIRST_TIER_LEADER, SECOND_TIER_LEADER,
BOARD_MEMBER, SENIOR_STAFF, or ABSTAIN |
| Variables | record ID, URL, date, author, title, description, split; development also supplies reference label, evidence span, label method, and review status |
| Information available | the public description attached to the record; the speech body, outside biography, gender, and inferred identity are excluded |
| Prediction point | when the public description is supplied for extraction; no later biography, personnel record, or outside search may enter the result |
| Splits | development: 600 records through 2021; audit: 250 records from 2022–2023; deployment: 250 records from 2024–2025; authors do not cross splits |
| Source | BIS central-bank speeches collection, pinned snapshot through 20 June 2025 |
| Where used | development selects the plurality baseline and permits one rule revision; audit compares frozen systems; deployment remains closed today |
Day 2 · Inside Airbnb London listings
Used in the Day 2 empirical studio, Predicting a quoted London price (60 minutes; pair work with individual votes).
Dataset
Inside Airbnb London listings
Pinned snapshot; prediction exercise, not pricing advice
| Item | Course definition |
|---|---|
| Files | london_price_development.csv;
london_price_validation.csv;
london_price_deployment_features.csv;
day2_application.ipynb |
| Unit | one eligible London listing with a valid one- to seven-night quote |
| Population and period | 54,636 eligible listings from 27,278 hosts in the Inside Airbnb London snapshot of 19 June 2026 |
| Target | price_gbp, the quoted nightly price in pounds; not a
transaction price, booking, occupancy, revenue, future price, or optimal price |
| Variables | numeric: accommodates, bedrooms, beds,
minimum_nights; categorical: room_type,
neighbourhood_cleansed; host_id is split-only |
| Prediction point | same-snapshot cross-sectional prediction before the target is supplied to the fitted model; this is not a future-time forecast |
| Splits | development: 32,749 listings/16,381 hosts; validation: 11,063/5,461; final holdout: 10,824/5,436; no host crosses splits |
| Source | Inside Airbnb, London detailed listings.csv.gz, 19 June 2026 |
| Where used | development fits preprocessing and models; validation compares predeclared candidates; the final holdout evaluates the recorded specification once |
Day 3 · Reporting Carrier On-Time Performance, 2025
Used in the Day 3 empirical studio, Can pre-departure information rank late arrivals? (BTS 2025 domestic flights).
Dataset
Reporting Carrier On-Time Performance, 2025
Official BTS monthly files; monthly-balanced teaching sample
| Your files | bts_2025_development.csv;
bts_2025_audit_features.csv; day3_application.ipynb |
|---|---|
| Empirical question | Among flights that later completed, were not diverted, and have an observed outcome, can information available before scheduled departure rank arrival at least 15 minutes late? |
| One row | One reported domestic flight in the stated retrospective study population. |
| Period and sample | January–December 2025; 3,000 deterministically selected eligible flights per month. |
| Outcome | delayed_15: arrival at least 15 minutes late. |
| Prediction time | Before scheduled departure. |
| Development | January–September, using expanding earlier-to-later folds. |
| Final audit | October–December; outcome hidden until the system is frozen. |
| Source | U.S. Department of Transportation, Bureau of Transportation Statistics. |
| Used in | Day 3 concept studio Questions 8–10; empirical Questions 1–6; Day 3 problem set Questions 1–12. |
Variables
| Field | Meaning | Use |
|---|---|---|
row_id | Stable schedule identity | Alignment only |
Month, DayOfWeek | Scheduled calendar fields | Eligible |
Reporting_Airline | Reporting carrier | Eligible |
Origin, Dest | Scheduled airports | Eligible |
scheduled_departure_minutes | Planned departure time | Eligible |
scheduled_arrival_minutes | Planned arrival time | Eligible |
CRSElapsedTime | Planned elapsed minutes | Eligible |
Distance | Scheduled distance | Eligible |
DepDelayMinutes | Realised departure delay | Too late |
Cancelled, Diverted | Later status | Define exclusions; not predictors |
delayed_15 | Arrival at least 15 minutes late | Target; hidden in audit |
split | Development or audit | Partition only |
Day 4 · BIS speech retrieval corpus
Used in the Day 4 empirical studio, Can a retrieved passage support the answer? (BIS speech retrieval corpus).
Dataset
BIS speech retrieval corpus
Pinned BIS speech collection; no live web search
| Your files | bis_speeches.csv,
bis_chunks.csv, queries.csv,
day4_application.ipynb |
|---|---|
| Empirical question | Can lexical and latent-semantic retrieval place a course-authored reference passage near the top, and can the retrieved words support a supplied candidate sentence? |
| One ranked item | One 300-whitespace-token speech chunk with 50-token overlap and at least 40 tokens in a final chunk. |
| Corpus | 40 speeches dated 2018–2025, producing 474 chunks. |
| Queries | Twelve fixed questions labelled Q01–Q12. |
| Methods | BM25 over exact terms; TF–IDF plus truncated SVD and cosine similarity as an inspectable LSA-style representation. |
| Reference evidence | Known-item judgements released only after the rankings are frozen; not exhaustive relevance labels. |
| Citation | BIS:<speech_id>:<four-digit chunk number>. |
| Source | Bank for International Settlements central-bank speeches, pinned through 20 June 2025. |
| Used in | Day 4 concept studio Questions 5–10; empirical Questions 1–6; Day 4 problem set Questions 5–10. |
Files and fields
| File or field | Meaning | Use |
|---|---|---|
bis_speeches.csv | Speech-level title, author, date, URL | Provenance and inspection |
bis_chunks.csv | Citation ID, speech ID, chunk number, title, text | Candidate collection |
queries.csv | Q01–Q12 and fixed question text | Input to both methods |
student_frozen_rankings.csv | Query, method, rank, citation, score | Saved before qrels |
| Released qrels | Known reference chunks for supported questions | Post-freeze scoring |
Oxford · United Kingdom
Course home
Teaching
CV
