Day 2 · Tuesday · Answer key

Loss, regression, probabilities, and action

Pre-class concept check

1. P1

   B. The mean responds directly to the size of an unusual value; the median depends mainly on order.

2. P2

   A. The sample mean minimises the sum of squared deviations.

3. P3

   B. This reads the slope as a conditional fitted comparison, not the effect of changing capacity.

4. P4

   A. Keeping hosts separate prevents another listing from the same host from making the test artificially easy.

5. P5

   B. The 40-to-one cost asymmetry makes review worthwhile at probabilities well below one half.

6. P6

   A. Uplift compares the outcome under contact with the outcome under no contact.

Concept studio

1. C1

   A. RMSE gives a relatively large contribution to severe errors because errors are squared before averaging.

2. C2

   C. A coefficient from observational prediction does not identify the effect of changing the feature.

3. C3

   A. Ranking suits a fixed queue; calibrated probabilities are needed for expected-cost calculations.

4. C4

   D. Costs are missing in fraud, while a no-contact comparison and values are missing in outreach.

5. C5

   B. \(0.08>0.0244\), and expected review cost £2.76 is below expected pass cost £9.60.

6. C6

   A. A 14% observed rate near an 8% score suggests local underprediction, not a conclusion about every score range.

7. C7

   B. S has £8 estimated net value; R has \(-\)£2 despite its higher response probability.

8. C8

   B. Random assignment makes action groups comparable in expectation.

9. C9

   A. Each decision uses its relevant incremental quantity: expected error cost for fraud and uplift net of contact cost for outreach.

Empirical studio

1. E1

   B. Hosts are unseen, but the evaluation remains within one listing snapshot.

2. E2

   D. Optimal pricing requires demand, revenue, cost, and the response to changing price.

3. E3

   A. Validation must evaluate the whole fitting procedure, including learned preprocessing.

4. E4

   D. The compact model has the lowest validation RMSE, 114.628.

5. E5

   B. Impossible outputs are a real support failure; any remedy inspired by validation needs fresh evaluation.

6. E6

   B. The slope is a fitted same-snapshot comparison, not a causal price effect or profit estimate.

7. E7

   A. The compact model improves unseen-host same-snapshot RMSE while still producing impossible outputs.

8. E8

   B. Pricing advice requires business outcomes and an identification design for the effect of price changes.

Problem set

1. S1

   A. Mean \(=(100+150+200+550)/4=250\); median \(=(150+200)/2=175\).

2. S2

   A. Squared loss weights large deviations heavily, and its constant minimiser is the mean.

3. S3

   C. The £18 coefficient is an association conditional on represented inputs, not the causal effect of adding a bed.

4. S4

   A. The design addresses new hosts within the same snapshot.

5. S5

   A. Learned replacements and category definitions are part of the fitted procedure and must not see validation.

6. S6

   A. Lower MAE with higher RMSE is consistent with better ordinary errors but worse extremes.

7. S7

   A. \(3/123\approx0.0244\).

8. S8

   B. Eight per cent exceeds the cost-based review threshold.

9. S9

   A. \(70/200=0.35\), above 0.20, suggesting local underprediction only.

10. S10

    A. R uplift is \(0.30-0.27=0.03\); S uplift is \(0.15-0.02=0.13\).

11. S11

    B. R gives \(100(0.03)-5=-2\); S gives \(100(0.13)-5=8\).

12. S12

    A. Random assignment supports a contact-versus-no-contact comparison in expectation, not observation of both outcomes for one person.

← Day 2 lecture  ·  All four days →