Practicals · Day 2
Convolution, transfer and visual shortcuts
When labelled images are scarce, what do locality, augmentation and transferred features add to an image classifier? The solution notebook investigates this question with photographs of ants and bees.
Day 2 files
day2_solution.ipynb in Jupyter. Run the cells in order. The first section includes the package-installation command if it is needed.The activity question
Can we classify ants and bees from a small labelled sample, and what changes when the model uses local image structure or features learned from a much larger image task?
You will compare a dense network with a small convolutional neural network (CNN), add training-time augmentation, and fit a new classifier to frozen ImageNet ResNet-18 features. A coloured-border experiment then tests whether a CNN can obtain a high score by following an unintended cue.
Dataset overview
| Observation | One colour photograph containing an ant or a bee. |
| Image input | A 96 by 96 image with red, green and blue channels. |
| Transferred input | A row of 512 fixed features from an ImageNet-pretrained ResNet-18. |
| Target | Ant or bee, recorded as class code 0 or 1. |
| Total | 396 photographs in the classroom dataset. |
How the data are split
The source training folder is divided into training and validation data with a fixed seed. The source validation folder becomes the held-out test set. Each split has one job.
| Split | Images | Class counts | Role in the activity |
|---|---|---|---|
| Training | 194 | 97 ant · 97 bee | Updates model parameters. Random augmentation is used only here. |
| Validation | 49 | 25 ant · 24 bee | Compares model structures, augmentation and transferred features. |
| Test | 153 | 70 ant · 83 bee | Provides one final result after the model has been selected. |
Data dictionary
The NumPy .npz file contains ten named arrays. The x arrays hold photographs, the y arrays hold their labels, and the features arrays hold the fixed ResNet representation of the same photographs.
| Array | Shape | What it contains | How it is used |
|---|---|---|---|
x_train | 194 × 96 × 96 × 3 | The training photographs. | Input to the dense model and CNNs during training. |
y_train | 194 labels | The correct class code for each training photograph. | Used to calculate training loss. |
x_validation | 49 × 96 × 96 × 3 | Photographs kept out of parameter updates. | Input for model comparison and the shortcut stress test. |
y_validation | 49 labels | The correct class code for each validation photograph. | Used to calculate validation loss, accuracy and confusion counts. |
x_test | 153 × 96 × 96 × 3 | The final held-out photographs. | Input for the final evaluation only. |
y_test | 153 labels | The correct class code for each test photograph. | Used once to calculate final accuracy. |
features_train | 194 × 512 | Frozen ResNet-18 features for the training photographs. | Trains the new two-class transfer head. |
features_validation | 49 × 512 | Frozen ResNet-18 features for the validation photographs. | Compares transfer with models trained from pixels. |
features_test | 153 × 512 | Frozen ResNet-18 features for the held-out photographs. | Evaluates the transfer head if validation evidence selects it. |
class_names | 2 names | The name attached to each class code. | Used in figure labels and result interpretation. |
Class label dictionary
| Code | Class | Meaning |
|---|---|---|
0 | Ant | The photograph is labelled as an ant image. |
1 | Bee | The photograph is labelled as a bee image. |
Source and preparation
The photographs come from the ants-and-bees collection used in the official PyTorch transfer-learning tutorial. The classroom copy reads the source JPEG images, converts them to RGB, and centre-crops and resizes them to 96 by 96 pixels.
The original training folder is divided with a fixed random seed. The original validation folder remains separate and is used as the final test set. The 512-feature arrays were calculated from the same resized photographs with the official ImageNet-pretrained ResNet-18 weights. These stored features remove the need to download or run the full ResNet during class.
What this activity asks you to do
- inspect photographs and describe variation in subject and background;
- follow the shape of an image batch through a small CNN;
- compare the parameter counts of a dense model and CNN;
- train both models on the same unaugmented photographs;
- add random flips and translations to training data only;
- fit a new two-class head to frozen pretrained features;
- plant a coloured-border shortcut and reverse it on the same validation images;
- select a model from validation evidence and evaluate it once on the test data.
Learning outcomes
- explain locality and weight sharing in a convolutional network;
- read the channel, height and width of an image tensor;
- explain why augmentation belongs in the training path only;
- distinguish training from pixels and training a head on frozen features;
- compare models using validation loss, accuracy and parameter count;
- use a paired stress test to identify reliance on a known shortcut;
- interpret a confusion matrix and report a held-out result.
What the evidence does not establish
- The validation set contains 49 photographs. One image changes its accuracy by about two percentage points.
- The source gives no photographer or collection identifier, so related photographs may appear in different splits.
- The stored ResNet features demonstrate frozen-feature transfer. They do not demonstrate fine-tuning of the full pretrained network.
- The coloured-border test examines one known shortcut. It cannot show that a model is free from other unintended cues.
- This two-class collection does not measure performance on fine-grained species recognition or photographs collected under different conditions.
