Generating dataset entries
When you need more coverage than you can write by hand, Spec27 can generate synthetic entries for you, based on the entries the specification already has and the instructions you provide. Generation is available for Gold Team and Goal Gold Team specifications.
Generated entries are added as primary entries of the specification, alongside any entries you wrote manually or imported, so you stay in control of what ends up in the final set. See Specifications for an overview of a specification's structure.
Generate entries
-
Open the specification.
-
In the entries toolbar, choose Add entries → Generate entries.
-
Set Number of queries to generate (between 1 and 50; the default is 10).
-
Optionally steer the output:
- Which aspect of the agent would you like to test in this dataset?
- Any custom instructions
These answers apply only to this generation run.
-
Start the generation. It runs in the background — the editor shows progress while it works, and the new entries appear when it finishes.
Review generated entries
Generation is a starting point, not a finished dataset. After the run completes, review the new entries the same way you would review any entry in your specification:
- Check that inputs are representative and expected outputs are correct.
- Edit or remove entries that do not fit.
- Keep the set focused — a smaller, well-curated dataset is easier to reason about than a large, noisy one.