A product page ranks third on Google for its target term. The same page gets zero citations in ChatGPT for the identical question, while a competitor two positions lower on Google gets quoted directly. The gap is not relevance. It is retrieval-augmented generation, and the way it breaks a page into pieces before any citation happens.
Key Takeaways
- Models do not retrieve whole pages. They retrieve chunks of roughly 100 to 300 tokens, and only the chunk that best matches the query embedding gets pulled into the answer.
- A paragraph that runs long or covers two ideas often gets split mid-thought, which weakens the embedding and lowers its odds of retrieval.
- Headings function as retrieval boundaries. A heading placed directly above the answer, not several paragraphs before it, anchors the chunk that follows to the query it should match.
What Actually Happens When a Model Retrieves a Page
When an AI model answers a question by pulling from the web, it is not reading your page top to bottom the way a person does. A retrieval system has already broken that page into overlapping segments, typically 100 to 300 tokens each, roughly 75 to 225 words. Each segment gets converted into a vector embedding, a numeric representation of what that specific piece of text means.
When a user asks a question, the system embeds the question the same way and searches for the chunks whose vectors sit closest to it. The model then generates its answer from whichever chunks score highest, and if it cites a source, it cites the page that chunk came from. Your page is not competing as a whole document. One paragraph, sometimes one sentence, is doing all the competing.
Why Paragraph Length Decides Which Chunk Survives
Chunking tools generally split text at paragraph or sentence boundaries, not mid-sentence, to keep each segment coherent. That means paragraph length directly shapes chunk quality. A paragraph that stays under roughly 150 words and covers one idea tends to become one clean chunk with a tight, specific embedding.
A paragraph that runs 400 words and drifts across three subtopics gets split by the tool wherever it hits the token ceiling, often in the middle of a thought. The resulting chunk carries a blurred embedding: part definition, part example, part caveat, none of it dominant. Blurred embeddings match fewer queries with confidence, so the chunk gets retrieved less often even when the underlying information is correct and useful.
A page is not competing as a whole document. One paragraph is doing all the competing.
Headings Are Retrieval Boundaries, Not Just Navigation
Headings do more than break up a page visually. Many chunking pipelines use heading tags as natural split points and attach the heading text to the chunk that follows, which means the heading effectively becomes part of that chunk’s embedding. A heading that states the exact question a reader would ask gives the chunk beneath it a sharper, more specific vector.
Placement matters as much as wording. If the heading sits three paragraphs above the actual answer, with scene-setting or throat-clearing in between, the chunk boundary lands in the wrong place. The paragraph that answers the question gets grouped with unrelated context above it, or split away from its own heading entirely. The fix is structural: put the direct answer in the paragraph immediately following the heading that asks the question, not after a run-up.
average position when Moburst’s own brand was cited across AI answers, out of 42,435 total citations tracked.See the case study
The Self-Contained Sentence Test
A useful check before publishing: take a single paragraph, remove everything else on the page, and read it alone. Does it still make sense? Does it answer a specific question without needing the paragraph before or after it? If the answer depends on a pronoun referring back three sentences, or on context set up two headings earlier, that paragraph will read as incomplete once it is lifted out as an isolated chunk, which is exactly what happens at retrieval time.
This is why dense, self-referential writing performs worse in AI answers even when it reads well as a full article. The chunk that gets pulled has to carry its own meaning. Name the subject in the sentence rather than pointing back to it. State the number, the date, or the term directly rather than saying “as mentioned above.”
What This Means for Content Architecture
Treat each section of a page as a candidate answer, not a subsection of a narrative. That shifts a few habits. Keep paragraphs under roughly 150 words when the paragraph is meant to answer something specific. Put one idea per paragraph rather than layering a definition, an example, and a caveat into the same block.
Write headings as the actual question a reader or a model would pose, and put the direct answer in the sentence right after it. This is retrieval presence, one of the two layers of AEO alongside training-data presence, and it is earned technically rather than through volume of coverage. A page can be authoritative and still lose citations if its structure fights the way retrieval systems slice it apart.
What to Do Next
Pull up your three highest-traffic pages and count the words in each paragraph under your main headings. Flag anything over 200 words, and split it at the natural idea boundary rather than an arbitrary line break. Then check whether the heading directly above each key paragraph states the question that paragraph answers, or whether it takes two or three sentences of setup to get there.
Fix the setup problem first. It is the cheapest change with the clearest effect on which chunk a retrieval system pulls, and it does not require touching your CMS template or your existing SEO structure. This works alongside the technical and structured-data work, it does not replace it.
FAQs
What Is a Token in the Context of RAG Chunking?
A token is a small unit of text, often a word or part of a word, that language models use to measure length. A 100 to 300 token chunk is roughly 75 to 225 words, depending on the model’s tokenizer.
Does Chunk Size Vary Between AI Models?
Yes. Different retrieval systems set different thresholds, and some use overlapping chunks to avoid losing context at the edges. The 100 to 300 token range is a common working band across major systems, not a fixed universal rule.
Can One Page Have Multiple Chunks Cited Across Different Answers?
Yes. A well-structured page with several self-contained sections can supply different chunks for different queries. This is one reason modular, single-idea paragraphs under specific headings outperform long narrative sections that only yield one usable chunk.
Does Improving Chunk Structure Help With Google Search as Well?
Generally yes. Clear headings, tight paragraphs, and direct answers tend to help traditional featured snippets and AI Overviews too, since both rely on extracting a specific, self-contained passage rather than the full page.
