Promptfoo
An open-source command-line tool for testing prompts and models against test cases defined in a config file, including security red teaming.
Tests live in a YAML file: inputs, the prompts and models to compare, and assertions on the output, from exact matches to a judge model. It runs locally or in CI, so a prompt change that breaks a case fails the build, see prompt regression. Its red teaming mode generates attacks such as prompt injection and jailbreaks against your application. In March 2026 OpenAI announced an agreement to acquire Promptfoo and said the project would stay open source.
Related terms
- Prompt regression: An answer that used to be right goes wrong after a change to the prompt or the model.
- AI red teaming: Deliberately attacking your own AI feature with harmful, tricky and injected inputs to find failures before users or attackers do.
- Evals: A repeatable test set that scores an AI feature, so changes can be measured instead of guessed.
- Ragas: An open-source Python library for evaluating RAG systems with metrics such as faithfulness, answer relevancy, context precision and context recall.
- DeepEval: An open-source Python framework that tests LLM applications the way pytest tests code, with ready-made metrics for RAG and agents.
