Chunking is the process of dividing long documents into smaller pieces so embedding models and retrieval systems can process them accurately and return focused results.
Chunking splits text into smaller segments using strategies such as fixed token counts, sentence boundaries, paragraph breaks, or recursive separators. Each chunk is embedded and indexed separately so retrieval returns the most relevant segment rather than an entire document.
That division has real consequences for retrieval quality. Too small and context is lost; a chunk containing only half a sentence may miss the point. Too large and the embedding becomes diluted; a five-thousand-token chunk mixes the relevant sentence with unrelated paragraphs. Chunking strategy determines how well semantic search actually works in practice.
Think of it like this. Think of tearing a book into pages before indexing. If pages are too short, they make no sense alone. If they are too long, the indexer cannot tell which part matters. The right page size makes the book searchable.
Text is split by character count, token count, sentence, paragraph, or semantic similarity. Overlap between chunks preserves context at boundaries. Recursive splitters try progressively smaller separators to avoid breaking mid-thought. Some systems use content-aware chunking based on document structure.
"Larger chunks preserve more context." They also dilute the embedding and return irrelevant passages. "Chunking does not matter." It is one of the biggest levers on retrieval quality. "One size fits all." Legal documents, code, and prose need different strategies.
Small chunks return precise matches but lose context; large chunks preserve context but introduce noise. The right size depends on query type, document structure, and the embedding model's effective receptive field.