Day 2 · Tuesday · Problem set

Loss, regression, probabilities, and action

1. Mean and median

   Development prices are £100, £150, £200, and £550. What are the mean and median?

   1. Mean £250; median £175

   2. Mean £175; median £250

   3. Mean £250; median £200

   4. Mean £225; median £175

2. Why the squared-loss baseline is the mean

   Which statement is correct?

   1. Squaring gives larger errors extra influence, and the sample mean minimises total squared error

   2. Squaring removes the influence of £550

   3. The median always minimises total squared error

   4. The largest observation minimises every loss

3. A coefficient is not a treatment effect

   A compact model associates one additional recorded bed with £18 higher predicted price. Which interpretation is not justified?

   1. The coefficient is expressed in pounds per recorded bed

   2. The coefficient is conditional on the variables represented in the model

   3. Adding a bed to the same listing will cause its achievable price to rise by £18

   4. Omitted quality or location can contribute to the observed association

4. Host-disjoint evaluation

   What does the split improve most directly?

   1. Evidence about listings from hosts absent during fitting

   2. Evidence about next year's market conditions

   3. Identification of the causal effect of price

   4. Measurement of realised host profit

5. Validation leakage

   You learn imputation values and category definitions after combining development and validation. Why is the reported validation RMSE compromised?

   1. The preprocessing has used information from the cases meant to evaluate the fitted procedure

   2. Linear models cannot use imputed values

   3. Validation outcomes must always be predictors

   4. Category definitions are never part of model fitting

1. When MAE and RMSE disagree

   Which explanation and decision are most plausible?

   1. A is closer on ordinary errors but has more severe large misses; prefer B when large misses dominate harm

   2. B is closer on every delivery; prefer A when large misses dominate harm

   3. A must have lower error on every case because its MAE is lower

   4. The metrics cannot differ on the same holdout

1. The review threshold

   Using \(\tau=3/(3+120)\), which threshold is closest?

   1. 0.024

   2. 0.200

   3. 0.500

   4. 0.976

2. Decision at eight per cent

   What is the cost-based action when \(p=0.08\)?

   1. Pass because 0.08 is below 0.50

   2. Review because 0.08 exceeds the cost-based threshold

   3. Pass because the false-review cost is nonzero

   4. No action can be chosen from probabilities and costs

3. Calibration at one score range

   Among 200 held-out payments scored near 0.20, 70 are fraudulent. Which interpretation is best?

   1. Observed frequency 0.35 suggests underprediction near 0.20; the whole calibration curve is not established

   2. Observed frequency 0.35 proves perfect calibration

   3. The model overpredicts near 0.20

   4. The ranking must be random

1. Estimated uplift

   What are the estimated uplifts for R and S?

   1. R 0.03; S 0.13

   2. R 0.30; S 0.15

   3. R 0.27; S 0.02

   4. R 0.13; S 0.03

2. Net value of contact

   Using \(100\times\text{uplift}-5\), which pair is correct?

   1. R £25; S £10

   2. R \(-\)£2; S £8

   3. R £3; S £13

   4. R \(-\)£5; S \(-\)£5

3. Why random assignment matters

   What does random contact assignment contribute?

   1. It makes contact and no-contact groups comparable in expectation, supporting estimation of the effect of contact

   2. It guarantees that every individual effect is observed

   3. It makes subscription value irrelevant

   4. It converts response probability into profit without costs

← Day 2 lecture  ·  All four days →