Answer relevancy
A RAG metric that scores how directly an answer addresses the question asked, regardless of whether it is factually correct.
An answer can be true and still miss the point: incomplete, padded, or about a neighboring topic. Ragas, which now calls this metric response relevancy, has a model generate questions the answer would fit and compares them with the original question by embedding similarity. Other tools ask a judge model directly. It does not measure truth, so read it together with faithfulness.
Related terms
- Faithfulness (groundedness): A RAG metric: the share of claims in an answer that are supported by the retrieved sources, not by the model’s memory or invention.
- Ragas: An open-source Python library for evaluating RAG systems with metrics such as faithfulness, answer relevancy, context precision and context recall.
- LLM as judge: Using a model with a rubric to grade another model’s output.
- Evals: A repeatable test set that scores an AI feature, so changes can be measured instead of guessed.
