Day 3 · Wednesday

From a spectacular score to a model that earns its place

Yesterday's models drew one line through the whole feature space. Today we meet models that cut the space into pieces, and the stricter evaluation that flexibility demands.

By the end of today

• read a decision tree as regions, and say where its predictions come from;
• explain when averaging many trees helps, and what limits the help;
• run a cross-validation that never touches the validation or test records;
• read ROC and precision-recall curves, and pick the right one for a rare event;
• audit any impressive score with two questions.

What we cover

• slides 6–14Part I: A model that cuts the feature spaceWhat a decision tree is, and what its leaves predict.
• slides 15–27Part II: Choosing flexibility fairlyThe depth dial, cross-validation, random forests, and an introduction to boosting.
• slides 28–35Part III: Reading a score like a rankingROC, precision-recall, and the contact list.
• slides 36–52Part IV: The audit, and the verdictTwo questions, one reveal, and the price of 0.94.

Slides 53–61 are optional appendix material for students who want the full algebra.

Lecture materials

Review questions

• A tree's leaf prediction is an average. Why, and how should you read a 23-client leaf?
• Averaging correlated trees stops helping after a point. Which formula says why, and what is the lever?
• Why do five folds judge a modelling choice better than one validation set?
• When does a precision-recall curve tell you more than a ROC curve, and why?
• A colleague reports a spectacular score. What two audit questions do you ask first?

← Day 2  ·  Day 4 →