Glossary
Adversarial robustness
The percentage of evaluated Red Team primary entries whose direct prompt passes and whose scored attack variations have no failures. The overall score uses the lowest defined attack-method score.
Agent
Runnable logic that processes an input.
Attack method
A transformation that creates adversarial variations from primary entries.
Baseline performance
The percentage of scored Gold Team primary entries the agent passes.
Baseline robustness
The percentage of scored Red Team primary entries whose direct prompts the agent handles correctly.
Clean accuracy
The percentage of scored primary entries with a correct result. This same measure is Baseline robustness for Red Team and Baseline performance for Gold Team and Goal Gold Team.
Credit
A plan unit consumed by metered product work.
Credit usage
Credits reserved or consumed by an organization during its current plan period.
Entry
A single test case or example within a specification that defines an input and expected behavior.
Entry category
An optional label that groups related primary entries for analysis. Red Team shows entry categories as vulnerability classes.
Eval
A named evaluation setup that connects agents and specifications.
Eval run
One execution of an eval across its agents and specifications. Each specification runs as the version snapshotted when the eval run starts.
Gold Team
Evaluation work focused on desirable behavior, correctness, and robustness.
Judge
A scoring configuration used when correctness requires interpretation.
Multi-turn eval
An eval in which an agent and a test exchange more than one turn.
Organization
The top-level shared workspace for members, projects, plan limits, and credit usage.
Performance under perturbations
The percentage of evaluated Gold Team primary entries with a correct primary result and no scored failure among their transformed variations. The overall score uses the lowest defined perturbation-method score.
Perturbation method
A way of transforming a Gold Team primary entry to check whether the agent still produces the expected result.
Plan
The resource limits and credit allowance assigned to an organization.
Plan limit
The maximum resource count or credit usage allowed by a plan.
Playground
An exploratory surface for trying configurations and generating derivative entries.
Primary entry
The original test case in a specification, before an attack or perturbation method creates variations.
Project
The main container for assets and results.
Project API key
A secret credential that grants API access to all or selected evals in one project.
Red Team
Evaluation work focused on misuse, harmfulness, jailbreaks, and failure-seeking behavior.
Resource limit
The maximum count for one plan-governed organization resource.
Result
A recorded output, score, or status produced during a run.
Robust accuracy
The percentage of evaluated primary entries with a correct primary result and no scored failure among their variations. The lowest defined method score is Adversarial robustness for Red Team or Performance under perturbations for Gold Team and Goal Gold Team.
Run
One product execution that produces a result.
Scored result
A result that completed execution and judging without an error.
Secret
A protected project value used by an agent at runtime.
Specification
A reusable evaluation recipe containing entries, methods for creating variations, and scoring configuration.
Specification run
One execution of a specification version against one agent within an eval run.
Specification version
An immutable snapshot of a specification's draft, taken when a run starts. A run reuses the newest version when the draft has not changed since.
Vulnerability class
The Red Team name for an entry category covering related risks or harmful behaviors. It describes what is being tested, not a confirmed vulnerability.