Best-of-N sampling
Generating several answers to the same prompt and keeping the best one, chosen by a test, a score or another model acting as judge.
Model output varies between runs, so asking N times and choosing raises quality without changing the model. The choice can be code (does it compile, does the JSON validate), a scoring function or an LLM judge. It costs N times the tokens and adds the judge’s cost, so it fits work where quality matters more than speed, such as published text or generated code.
Related terms
- LLM as judge: Using a model with a rubric to grade another model’s output.
- Evals: A repeatable test set that scores an AI feature, so changes can be measured instead of guessed.
- Tokens: The pieces of text models read and write, and the unit you pay for.
