Day 1 · Monday · Concept studio

The dispatch gate

How this block runs

For each question, choose an answer on your own. Record your first vote. When I ask, compare reasons with a neighbour and vote again. A changed answer is evidence of learning, not a penalty. Do not open Sheet B until instructed.

1. A coherent prediction contract

   Which proposal joins one unit, one prediction moment, one outcome, and one decision?

   1. Score each parcel at acceptance, predict delay over 24 hours, and rank parcels for the ten-place review queue

   2. Score each completed route at day end, predict complaints, and use the score to reroute parcels at acceptance

   3. Score each merchant once a year, predict parcel delay, and automatically cancel individual deliveries

   4. Score each complaint after delivery, predict whether the parcel was late, and call this early intervention

2. A feature that crosses the time boundary

   Which input would make an acceptance-time evaluation misleading?

   1. The service level purchased before acceptance

   2. The planned route recorded at acceptance

   3. The merchant's late-delivery rate calculated from earlier completed parcels

   4. The realised delay at an intermediate hub reached after acceptance

3. What 90% accuracy does not settle

   Before audit labels are opened, the team says the model is 90% accurate. What is the most defensible response?

   1. Deploy because 90% exceeds the review capacity

   2. Reject because every useful model must exceed 95% accuracy

   3. Continue the audit because prevalence, baseline performance, and errors inside the ten-place queue are still unknown

   4. Replace accuracy with a causal effect estimate

4. Choosing an evaluation set

   The immediate use is next month's parcels from merchants already operating in the system. Which holdout is the closest first check?

   1. Random rows drawn from the whole year, including later months in model fitting

   2. The latest month, kept out of every modelling choice

   3. Five training rows from each merchant

   4. Only the ten merchants with the most observed delays

1. Read the compact audit table

   Which set of values is correct?

   1. Accuracy 90%, precision 60%, recall 50%, always-on-time baseline 88%

   2. Accuracy 94%, precision 50%, recall 60%, always-on-time baseline 90%

   3. Accuracy 90%, precision 50%, recall 60%, always-on-time baseline 12%

   4. Accuracy 88%, precision 60%, recall 90%, always-on-time baseline 94%

2. Does the ranking help at capacity ten?

   If ten parcels were selected at random, the expected number of late parcels would be \(10\times 12/100=1.2\). The model's queue contains six late parcels. Which conclusion fits this evidence?

   1. The queue prioritises late parcels in this audit, although costs and future stability still matter

   2. The model prevents exactly 4.8 late deliveries

   3. The model is useless because its recall is below 100%

   4. The model should reroute every flagged parcel automatically

3. Prediction versus rerouting

   The planned courier is associated with risk. A manager wants to move flagged parcels to the courier with the lowest historical late rate. Does the audit establish that this change will reduce delays?

   1. Yes, because planned courier was measured before acceptance

   2. Yes, because the model beat random selection

   3. No; the audit evaluates prediction, while the rerouting effect requires a credible changed-versus-unchanged comparison

   4. No; a courier variable can never be used in prediction

1. The narrowest defensible recommendation

   Which recommendation is best supported?

   1. Use the score as one input to a ten-parcel human review queue, monitor errors, and evaluate any rerouting policy separately

   2. Let the score automatically reroute parcels because accuracy is 90%

   3. Claim that flagged parcels are late because of their planned couriers

   4. Revise the rule on these 100 audit labels and report the revised score on the same rows as final evidence

← Day 1 lecture  ·  All four days →