Day 3 · Wednesday · Student answer key
Answers and explanations
Answer first
This key is released after submission. The questions it answers are on the pre-class check, the two studios and the problem set for this day. Work through them and commit to an answer before you read the explanations here: the point of every question is the reasoning, and a letter you have not argued for teaches you nothing.
Remember
Use this sheet after you have submitted your choices. For every mistake, name which boundary you missed: information time, validation, metric, population, or audit.
Pre-class concept check
| Item | Answer | Why |
|---|---|---|
| P1 | B | The leaf reports \(17/23\), but a small, selected leaf can be unstable and uncalibrated on future cases. |
| P2 | C | The training gain accompanied by later collapse is evidence that added depth fitted development-specific detail. |
| P3 | D | Realised departure delay is observed only after the score is meant to exist. |
| P4 | B | Earlier-to-later folds imitate monthly updating; a random whole-year split mixes future and past regimes. |
| P5 | B | Precision at the highest 10% directly matches a ten-per-cent inspection capacity. |
| P6 | B | Trying many choices makes the validation folds part of selection, so the winner may contain favourable noise. |
In-class concept studio
| Item | Answer | Why |
|---|---|---|
| C1 | A | Observed first-48-hour data can support prediction of payment, not the effect or value of outreach. |
| C2 | D | Seven-day viewing is unavailable at the 48-hour score. |
| C3 | C | Depth 6 has the highest declared mean AP and does not win by one isolated fold. |
| C4 | A | Forest wins the declared full-ranking measure; logistic wins the stated highest-fifth measure. |
| C5 | B | System B places both failures inside the only three inspections the desk can make. |
| C6 | B | Equal AUC means equal correct pair ordering in aggregate, not equal membership at a cutoff. |
| C7 | A | The arithmetic describes concentration among selected units; no untreated counterfactual identifies prevention. |
| C8 | C | The 0.71 system uses information that arrives after the 48-hour clock. |
| C9 | B | The number can be studied for a later decision only after redefining the question and evaluation. |
| C10 | B | A final audit judges a frozen system once; revisions require new untouched evidence. |
Empirical studio
| Item | Answer | Why |
|---|---|---|
| E1 | B | Eligibility is known retrospectively, so the supplied rows support only the conditional retrospective ranking claim. |
| E2 | D | Realised departure delay occurs after the pre-departure information cutoff. |
| E3 | B | Rolling folds reproduce the direction from past fitting to later use. |
| E4 | A | AP 0.3325 exceeds the no-skill reference 0.2251 within the stated audit population; AP is not accuracy or an effect. |
| E5 | A | Precision among the 900 selected flights is \(350/900=0.3889\). |
| E6 | C | A live study must define eligibility for every flight at the scoring clock, before later completion and label status are known. |
After-class problem set
| Item | Answer | Why |
|---|---|---|
| S1 | C | Training improvement cannot choose between partitions; later stability does, with simplicity preferred under a tie. |
| S2 | A | The maximum of many repeatedly inspected scores selects favourable validation noise. |
| S3 | B | End-of-August outcomes are future information for earlier August predictions. |
| S4 | B | Every transformation must be learned on the training window and then applied unchanged to its later judge window. |
| S5 | B | Unrewarded complexity adds estimation and governance cost without evidence of operational gain. |
| S6 | B | AP evaluates the ranking broadly; top-decile precision evaluates one operating cutoff. |
| S7 | C | The score belongs to an after-departure information set, not the original pre-departure study. |
| S8 | B | Moving the clock changes the possible action and therefore the entire prediction contract. |
| S9 | C | Equal monthly counts support comparison in this sample, not annual prevalence inference. |
| S10 | B | The desk cannot know in advance which flights will later satisfy the study's completion and label conditions. |
| S11 | B | Once audit outcomes guide a change, they are no longer untouched audit evidence. |
| S12 | B | The redesigned unit, outcome, clock, action, and label gap all belong to the same prospective cancellation decision. |
Oxford · United Kingdom
Course home
Teaching
CV
