What is chunking?
Chunking is a RAG preparation step that splits long documents into smaller, self-contained passages for embedding and retrieval. An assistant can only answer well if the right passage is found first. Chunks that are too small lose context, and chunks that are too large carry unrelated text. That makes chunking one of the decisions with the greatest effect on the quality of a RAG system.
Definition
Chunking is a RAG preparation step that splits long documents into smaller, self-contained passages for embedding and retrieval.
How is chunking done?
The simplest method splits text into fixed-length pieces. It is easy to build, but it can cut sentences and ideas in half. Structure-aware methods follow headings, paragraphs and clause boundaries. An overlap between neighbouring chunks keeps information at the border from being lost.
Each chunk carries metadata such as document name, section heading, date and access rights. The chunk is then converted into a vector by an embedding model and written to the index.
Why does chunk size matter?
In a RAG system the model does not receive the whole document. It receives the retrieved chunks. Small chunks are more precise in search but may not make sense on their own. Large chunks keep context but carry unrelated text and fill the context window faster.
The right balance depends on the document. FAQ content works well with short chunks. Procedures and contracts need their sections kept intact.
What should you watch for?
Split regulations, contracts and specifications at clause boundaries. If the paragraphs of one clause land in different chunks, the answer will be incomplete. Store the clause number and heading in each chunk’s metadata.
Check the text extracted from PDFs before you index it. Non-English characters are a common casualty. In Turkish documents, letters such as ı, İ, ş and ğ can be corrupted, which weakens both semantic search and keyword search. For documents with personal data, decide early who may see which chunk and record it as access metadata. This supports data minimisation under GDPR.
Further reading
How to evaluate a RAG systemTest sets, metric choice, LLM-as-judge and live monitoring for measuring retrieval and generation quality in RAG systems.
RAG for WordPressHow retrieval-augmented generation works on WordPress sites for search, support and content discovery, and which build decisions matter.
What drives the cost of a RAG project?One-off and running cost drivers of a RAG knowledge assistant, the choices that raise or lower them, and what to prepare before a quote.Related terms
- RAG (Retrieval-Augmented Generation)RAG (Retrieval-Augmented Generation) is an AI architecture that lets a language model retrieve relevant passages from a chosen document collection, such as a company's own documents, and use them as context before it writes an answer.
- EmbeddingAn embedding is a fixed-length array of numbers (a vector) that represents the meaning of a word, sentence, document or image, so that similar content ends up close together.
- Semantic SearchSemantic search is a search method that finds the most relevant content by comparing the meaning of the query and the text rather than matching exact words.
- RerankingReranking is a RAG step that uses a second model to reorder the candidate results of an initial search by their relevance to the query.
- Context WindowA context window is the limit, measured in tokens, on how much text a language model can read and take into account in a single request.
Frequently asked questions
What is the ideal chunk size?
There is no single correct value. It depends on the document type, the embedding model and the kind of questions users ask. The sound approach is to try several sizes on your own documents and real questions, then compare retrieval results.
Why do chunks overlap?
A sentence or an idea can be cut at the border between two chunks. Letting neighbouring chunks share a few sentences keeps that information whole in at least one of them. More overlap means a larger index and more repeated results.
How are tables and scanned PDFs chunked?
Scanned documents need OCR first. Tables should not be cut mid-row and are best kept as one chunk together with the header row. If the extracted text is poor, no chunking strategy will rescue the result.
Let’s talk about your transformation
Tell us which work repeats every week. We will tell you honestly whether it is worth automating.
How we start
- A free first callWe pin down the problem and the goal together.
- A 2-week discovery sprintWe look at the data, the process and the tools, and decide where to start.
- A written proposalScope, architecture and timeline, in writing.