Chunking (RAG)
Splitting documents into smaller passages before indexing them for search, so retrieval returns the relevant part instead of a whole file.
Chunks that are too large dilute the meaning and waste context; too small, and they lose the sentence that explains them. Splitting along the document’s own structure (headings, sections, table rows) works better than fixed lengths, and a small overlap or the section title added to each chunk keeps context. Tables, scanned PDFs and spreadsheets usually need their own handling.
Related terms
- RAG (retrieval-augmented generation): Finding relevant documents first and putting them into the prompt, so the model answers from them.
- Embeddings: Vectors of numbers that place texts with similar meaning close together.
- Reranker: A model that rescores the top search results against the query, so the most relevant ones go to the language model first.
