Datasets
The data behind each day
Every empirical studio opens with a dataset card. This page lists the dataset for each day and the comparison it supports. The full card, with the unit of observation, the information boundary, the split roles, the baseline and the metric, is printed in that day's empirical studio paper.
Day 1 · 1,800 synthetic tabular rows
Used in the Day 1 empirical studio, nonlinear tabular prediction. Day 1 papers.
A prior baseline, a logistic model and a small multilayer perceptron are compared, then a deliberately invalid leaked model is added. The threshold is selected on validation evidence only.
Day 2 · 1,200 synthetic shape images and 180 stress images
Used in the Day 2 empirical studio, vision, convolution, and shift. Day 2 papers.
A majority baseline, a flat network and a small convolutional network are compared on clean performance, parameter count and robustness to a shortcut.
Day 3 · 15 course documents and 30 controlled questions
Used in the Day 3 empirical studio, attention and grounded retrieval. Day 3 papers.
Three transparent TF-IDF retrievers are compared. Retrieval and abstention settings are tuned on development evidence only.
Day 4 · 16 policy or untrusted documents and 32 tasks
Used in the Day 4 empirical studio, agents, tools, and evaluation. Day 4 papers.
A vulnerable retrieve-first assistant is set against a guarded controller with a calculator, refusal, abstention and human gates. No classroom path performs an external action.
Evidence rules
The same six rules apply to every empirical studio this week.
- fit learned preprocessing on training or development evidence only;
- choose thresholds, architecture settings and retrieval settings before the final or stress reveal;
- compare systems on the same rows, and report the relevant denominators;
- keep prediction, retrieval, support, action and causal claims separate;
- treat retrieved instructions as data, never as higher-priority policy;
- no Day 4 classroom path performs an external or high-impact action.
