Golden dataset
A fixed set of real inputs with expected answers or checks, used to test an AI feature after every prompt or model change.
The cases come from real traffic and from bugs already found, not only from the happy path, and each one says what a good answer must contain or avoid. Because the set stays the same between runs, a drop in the score points at the change you just made. It grows over time: every bug a user reports becomes a new case.
Related terms
- Evals: A repeatable test set that scores an AI feature, so changes can be measured instead of guessed.
- Prompt regression: An answer that used to be right goes wrong after a change to the prompt or the model.
- LLM as judge: Using a model with a rubric to grade another model’s output.
