Reranker
A model that rescores the top search results against the query, so the most relevant ones go to the language model first.
Fast retrieval returns, say, 50 candidates by rough similarity. A reranker (a cross-encoder or a small LLM) reads the query and each candidate together, which is slower but much more precise, and keeps the best few. Fewer, better passages mean a shorter prompt, lower cost and fewer answers built on the wrong document.
Related terms
- Hybrid search: Combining keyword search (such as BM25) with vector search, so results match both exact terms and meaning.
- RAG (retrieval-augmented generation): Finding relevant documents first and putting them into the prompt, so the model answers from them.
- Chunking (RAG): Splitting documents into smaller passages before indexing them for search, so retrieval returns the relevant part instead of a whole file.
