What is chunking?

Chunking is a RAG preparation step that splits long documents into smaller, self-contained passages for embedding and retrieval. An assistant can only answer well if the right passage is found first. Chunks that are too small lose context, and chunks that are too large carry unrelated text. That makes chunking one of the decisions with the greatest effect on the quality of a RAG system.

Definition

Chunking is a RAG preparation step that splits long documents into smaller, self-contained passages for embedding and retrieval.

How is chunking done?

The simplest method splits text into fixed-length pieces. It is easy to build, but it can cut sentences and ideas in half. Structure-aware methods follow headings, paragraphs and clause boundaries. An overlap between neighbouring chunks keeps information at the border from being lost.

Each chunk carries metadata such as document name, section heading, date and access rights. The chunk is then converted into a vector by an embedding model and written to the index.

Why does chunk size matter?

In a RAG system the model does not receive the whole document. It receives the retrieved chunks. Small chunks are more precise in search but may not make sense on their own. Large chunks keep context but carry unrelated text and fill the context window faster.

The right balance depends on the document. FAQ content works well with short chunks. Procedures and contracts need their sections kept intact.

What should you watch for?

Split regulations, contracts and specifications at clause boundaries. If the paragraphs of one clause land in different chunks, the answer will be incomplete. Store the clause number and heading in each chunk’s metadata.

Check the text extracted from PDFs before you index it. Non-English characters are a common casualty. In Turkish documents, letters such as ı, İ, ş and ğ can be corrupted, which weakens both semantic search and keyword search. For documents with personal data, decide early who may see which chunk and record it as access metadata. This supports data minimisation under GDPR.

Further reading

Related terms

Frequently asked questions

What is the ideal chunk size?

There is no single correct value. It depends on the document type, the embedding model and the kind of questions users ask. The sound approach is to try several sizes on your own documents and real questions, then compare retrieval results.

Why do chunks overlap?

A sentence or an idea can be cut at the border between two chunks. Letting neighbouring chunks share a few sentences keeps that information whole in at least one of them. More overlap means a larger index and more repeated results.

How are tables and scanned PDFs chunked?

Scanned documents need OCR first. Tables should not be cut mid-row and are best kept as one chunk together with the header row. If the extracted text is poor, no chunking strategy will rescue the result.

Let’s talk about your transformation

Tell us which work repeats every week. We will tell you honestly whether it is worth automating.

How we start

  1. A free first callWe pin down the problem and the goal together.
  2. A 2-week discovery sprintWe look at the data, the process and the tools, and decide where to start.
  3. A written proposalScope, architecture and timeline, in writing.

Example questions

Go straight to

CorrAI answers with AI from the content of this site and can be wrong. Please do not enter personal details. Privacy

Book a free call