Technique
What is Chunking?
Also known as: text splitting
Chunking is splitting long documents into smaller passages before embedding them, so retrieval can return the specific paragraph that answers a question rather than a whole 200-page manual.
You can't embed a book usefully — one vector for 200 pages averages everything into mush, and you can't fit it in a prompt anyway. Chunking cuts the document into retrievable units, typically a few hundred tokens each, with a small overlap so a sentence spanning a boundary isn't lost. Where you cut matters more than how big: splitting on headings, paragraphs or code blocks keeps each chunk about one thing, while splitting every 500 characters happily severs a table from its header.
Chunking is the least glamorous part of RAG and the most common reason it disappoints: if an answer is split across two chunks and neither is retrieved whole, no model can rescue it. The trade-off is context — small chunks match precisely but lose their surroundings; large chunks carry context but dilute the match and spend tokens. It's the same discipline behind a capability mesh: Velaris makes each tool its own retrievable unit with its own description, so the agent pulls in one well-formed capability rather than a connector's entire schema.