Clean vs robust performance
Spec27 reports two headline numbers for most evaluations: clean accuracy and robust accuracy. This page explains what each one means and how to read them together.
Clean performance
Clean accuracy is the accuracy on your primary entries — the original, unmodified entries. It is the percentage of primary entries the agent got right:
clean accuracy = correct primary entries ÷ total primary entries
If a specification has no adversarial coverage, clean accuracy is the only number that exists. Result pages show it alongside the count behind it, for example "X / Y correct".
Robust performance
Robust accuracy measures how well an agent holds its correct answers when its inputs are perturbed by attack methods. A primary entry counts as robust only when the agent answers it correctly and answers every adversarial variant of it correctly:
robust accuracy = primary entries correct across all their variants ÷ total primary entries
Because a single failed variant makes the whole primary entry non-robust, robust accuracy is always less than or equal to clean accuracy. Robust accuracy only appears once a specification includes adversarial coverage.
Reading the two together
The gap between the two numbers is the signal. A high clean score with a much lower robust score means the agent gets the right answer on tidy inputs but is brittle under realistic variation — typos, rephrasing, different personas, or adversarial pressure. Closing that gap, not just raising clean accuracy, is usually the goal.
Spec27 tracks both numbers across runs over time, and the robustness views break robust accuracy down by attack method so you can see where an agent is losing ground.