ciunkos.com/Glossary

Agents and tools

Evaluator-optimizer (maker-checker)

One model drafts, another checks the draft against stated criteria and returns feedback, in a loop until it passes or a round limit stops it.

Evaluator-optimizer is a loop of two roles. A generator writes a draft. An evaluator checks it against stated criteria and returns specific feedback. The generator revises, and the loop repeats until the draft passes or a round limit stops it. Banks call the same idea maker-checker. It is one of the agentic workflow patterns.

Anthropic’s Building effective agents says it works best when there are clear evaluation criteria and when feedback measurably improves the result: a translation, a contract summary, code that has to pass tests. The evaluator can be a model, a test suite, a linter or a person. Deterministic checks are cheaper and harder to fool, so run them first and let a model judge only what code cannot. Each round costs another draft and another review, so cap the rounds and track cost per task. A loop with no exit condition is the most common bug.

The neighbors differ in shape. LLM-as-a-judge scores outputs, often offline in evals, with no revision loop. Best-of-n generates several candidates and picks one, with no feedback. Adversarial review uses a reviewer whose job is to break the work. Evaluator-optimizer is the pattern that feeds criticism back into the next draft.

Sources

Share: LinkedIn · X ·

Related terms

  • Agentic workflow (workflow vs agent): A process where your code fixes the steps and a model does some of them, as opposed to an agent that decides its own steps.
  • LLM as judge: Using a model with a rubric to grade another model’s output.
  • Best-of-N sampling: Generating several answers to the same prompt and keeping the best one, chosen by a test, a score or another model acting as judge.
  • Adversarial review (cross-model review): Asking a second AI model, or a fresh agent, to find what is wrong with a change before a person reviews it, instead of asking the author to check itself.
  • Evals: A repeatable test set that scores an AI feature, so changes can be measured instead of guessed.
  • Cost per task (agent unit economics): What one finished agent task really costs: tokens, retries, tool calls and the human time to review it, compared with doing it another way.
  • Human in the loop: A person approves, edits or rejects the agent’s action before it takes effect.
  • Prompt chaining: A fixed sequence of model calls where each step works on the previous step’s output, with checks in code between the steps.

Linked from

← All terms

Free tools

Business idea validatorTen rubrics, a fixed-rule verdict and three cheap experiments. Free.Software cost estimatorYour project among 125 real products, with honest ranges.Engagement configuratorPick the work and its size, see hours, price and start date.SPF, DKIM and DMARC checkerGrade any domain’s email DNS, with records to paste.BIMI checkerRecord, DMARC, SVG Tiny PS logo and VMC or CMC, with an inbox preview.App store readiness checkWhat your web app lacks for the stores, TWA or Capacitor, files to copy.Build or buy calculatorSaaS against your own app over 1, 3 or 5 years, with the payback month.Runbook generatorA RUNBOOK.md for a system only you know, in two minutes, in your browser.SLA calculatorMonthly fee, response times, a downtime budget and contract clauses.Local model or API?Does the model fit your hardware, how fast, what it costs against an API.OpenAPI to MCP toolsPaste an OpenAPI spec, get narrow MCP tools and a server skeleton.KSeF invoice validatorCheck an FA(3) invoice against the official schema in your browser.Quoting time calculatorHours spent turning inquiries into quotes, and when an agent pays off.Multi-tenant SaaS calculatorWhich multi-tenant model, the work per release and the weeks to get there.Quote DeskDrop a customer inquiry, get lines matched to your price list. Free.AI Visibility DeskDoes Claude recommend your brand? Weekly runs of your buyers’ questions, free.List DeskClean an email list in your browser: dead domains, disposable, typos, duplicates.Eval DeskTest cases for your AI prompt, run after every change: what regressed, what got fixed.Dashboard DeskA growth dashboard created in your PostHog, plus what your tracking is missing.Signature DeskOne email signature template for the whole team, as HTML for Gmail, Outlook and Apple Mail.Phishing email checkerForward a suspicious email or upload the .eml file; three AI models and technical checks give a verdict.Email TestSend one email, check one-click unsubscribe, SPF, DKIM, DMARC and spam signals.Vitals DeskA daily lab check of your pages on a throttled phone: LCP, CLS and TBT, and an email when a page gets slower. Free.Legacy rewrite estimateLines of code to function points, hours and cost, with and without agents.

Want the fun version? The Corporate-speak Translator has a Polish dictionary of office phrases and what they really mean. Corporate-speak dictionary (Polish) →

Need this built, fixed or reviewed?

I build and rescue products with AI coding agents under rules: AI features in production, honest estimates, email that gets delivered.

Przemysław Zalewski

How I work

Przemysław Zalewski · 15+ years shipping software

“a true bias for action” client CTO, public LinkedIn recommendation

My pledge

All principles, the AI policy and working hours →

Pricing · Security · Privacy and analytics opt-out · Trust center