Practicals
The labs and their data
Each teaching day has one question, one notebook and the data needed to answer it. All four days are available.
The four labs
One row is one clothing photograph. The classroom subset is balanced by class and drawn with a fixed seed: 12,000 training images, 3,000 validation and 3,000 final test. Validation chooses the model and the stopping epoch; the final split is opened once.
One row is one photograph of an ant or a bee: 194 training images (97 per class), 49 validation and 153 final test. The training set is deliberately small, because that is the setting in which the architecture, the augmentation and a borrowed representation decide the result. The lecture uses Oxford-IIIT Pet; the lab uses this collection because it downloads and trains inside one session.
Twenty short policy documents, twenty development questions and twenty final questions. Both labelled sets are built to the same composition: 12 answerable, 3 unanswerable, 2 ambiguous and 3 containing an injected instruction. Retrieval is TF-IDF and the answer is the first sentence of an accepted document, so every step can be inspected; no language model is called.
Sixteen policy documents and two sets of sixteen cases, one for development and one for final evaluation. The cases cover retrieval, calculation, missing evidence, an injected instruction, a high-impact action, stale state, an ambiguous timeout, invalid tool arguments and one compound task. Two controllers are compared on the same cases, and safety is read from the recorded tool calls rather than from what the controller reports.
The order every lab follows
All four notebooks run in the same sequence, because the sequence is part of what is being taught. A number that arrives in the wrong order is not evidence.
- state the question, and say what one example is;
- look at the data before modelling it;
- fix the role of each split: development, validation, final test;
- set a baseline that costs nothing to compute;
- work one mechanism by hand before calling the library version of it;
- compare models on validation evidence rather than on training fit;
- audit the errors, not only the average score;
- choose the procedure on validation, then use the final split once;
- record what the result means, and what it does not cover.
Each dataset is already prepared as one file. Its notebook loads that file directly and states the role of the training, validation or development, and final data before any model is selected. The page for each day gives the full data dictionary, source, preparation, activity question, learning outcomes and limits of the evidence.
