Robustness
Robustness views show how well your agents keep their correct answers when inputs are perturbed by attack methods. They aggregate clean and robust accuracy across completed runs so you can see the current state, the recent trend, and where coverage is thin.
Where robustness appears
The project results area offers three views of the same underlying data, each answering a different question:
- Eval Results — "Robust accuracy state, recent trend, and last completed run for each eval." Use this to see how each eval is doing overall.
- Spec Results — "Robust accuracy state and last completed run for each specification." Use this to compare specifications.
- Agent Results — "Accuracy variance across completed specification runs for each agent." Use this to compare agents.
Each view lists its items with their Clean accuracy, Robust accuracy, Coverage, run count, and recent Activity.
Reading a robustness detail
Opening an item shows its aggregate detail. Alongside the headline Clean and
Robust percentages, the detail views show a Recent trend — the number of
runs included, the change versus before (for example +5%, -3%, or no change),
and a sparkline of the recent history. This is where you confirm whether a change
to an agent actually moved robustness, and in which direction.
The underlying run results break performance down by attack method and by category, so you can find the specific perturbations an agent struggles with.
Improving robustness
- Start from the gap between clean and robust accuracy — that is where the fragility is.
- Use the per-attack-method breakdown to identify which attack methods the agent fails.
- Iterate on the agent, then re-run the eval and compare the new robust accuracy and trend against the previous runs.