Evaluator-optimizer (maker-checker)
One model drafts, another checks the draft against stated criteria and returns feedback, in a loop until it passes or a round limit stops it.
Evaluator-optimizer is a loop of two roles. A generator writes a draft. An evaluator checks it against stated criteria and returns specific feedback. The generator revises, and the loop repeats until the draft passes or a round limit stops it. Banks call the same idea maker-checker. It is one of the agentic workflow patterns.
Anthropic’s Building effective agents says it works best when there are clear evaluation criteria and when feedback measurably improves the result: a translation, a contract summary, code that has to pass tests. The evaluator can be a model, a test suite, a linter or a person. Deterministic checks are cheaper and harder to fool, so run them first and let a model judge only what code cannot. Each round costs another draft and another review, so cap the rounds and track cost per task. A loop with no exit condition is the most common bug.
The neighbors differ in shape. LLM-as-a-judge scores outputs, often offline in evals, with no revision loop. Best-of-n generates several candidates and picks one, with no feedback. Adversarial review uses a reviewer whose job is to break the work. Evaluator-optimizer is the pattern that feeds criticism back into the next draft.
Sources
- Anthropic: Building effective AI agents anthropic.com
Related terms
- Agentic workflow (workflow vs agent): A process where your code fixes the steps and a model does some of them, as opposed to an agent that decides its own steps.
- LLM as judge: Using a model with a rubric to grade another model’s output.
- Best-of-N sampling: Generating several answers to the same prompt and keeping the best one, chosen by a test, a score or another model acting as judge.
- Adversarial review (cross-model review): Asking a second AI model, or a fresh agent, to find what is wrong with a change before a person reviews it, instead of asking the author to check itself.
- Evals: A repeatable test set that scores an AI feature, so changes can be measured instead of guessed.
- Cost per task (agent unit economics): What one finished agent task really costs: tokens, retries, tool calls and the human time to review it, compared with doing it another way.
- Human in the loop: A person approves, edits or rejects the agent’s action before it takes effect.
- Prompt chaining: A fixed sequence of model calls where each step works on the previous step’s output, with checks in code between the steps.
