Guardrails
Checks around a model that block, fix or flag inputs and outputs that break the rules.
Input filters, output validation, allow-lists for tools, limits on spend and rate, and approval steps. The strongest guardrails are in code and permissions, not in the prompt, because a prompt can be argued with.
Related terms
- Prompt injection: Instructions hidden in data (a web page, an email, a file) that the model follows as if they came from you.
- Human in the loop: A person approves, edits or rejects the agent’s action before it takes effect.
- Cost cap: A hard spending limit enforced by your code, per user, per day or per month.
