Blog · Updated 2026-08-17 · 5 min
Why docs chatbots hallucinate — and how to stop them
A documentation chatbot that sounds confident while naming an API that never shipped is worse than no bot. Support teams lose trust in one wrong answer. This guide explains why RAG bots invent product facts, how to catch it in ten questions, and what a docs-first stack should do instead of guessing. The failure is not silence. It is a fluent paragraph that looks like your docs and is not.
What hallucination looks like on docs
On a marketing site, a made-up adjective is annoying. On product docs it is a support incident. The visitor copies the invented snippet, pastes it into a terminal or a webhook handler, and opens a ticket when it fails. They blame your product, not the model.
Typical failures:
- An endpoint, SDK method, or webhook event that does not exist.
- A price, seat limit, or plan name from last year.
- Setup steps that skip auth, mix v1 and v2, or invent a dashboard path.
- A “yes” to a feature that is still on the roadmap.
Users treat chat as the official voice of the product. A wrong rate limit or a fake header name is the same class of error as shipping it on the billing page. That is why docs chat is a trust problem first and a retrieval problem second.
Why the model invents when your docs exist
Most “docs chatbots” are a general model plus a vector search over crawled pages. Hallucination is not a random glitch. It is what the model does when the retrieved context is thin, conflicting, or off-topic. The prompt still says be helpful. Helpful without a source becomes fiction.
Common causes:
- The crawl never indexed the right page (login walls, JS-only shells, skipped /docs paths).
- The chunk is the wrong section — the bot retrieved “Overview” when the question needed “Rate limits.”
- Two versions live in the same index, so v1 and v2 blend.
- The question uses customer language (“SSO from Okta”) that does not match the heading (“Enterprise authentication”).
- The prompt tells the model to always answer, even when retrieval is empty.
If you only measure “the bot answered,” you will ship a fluent liar. Measure whether the cited URL would satisfy a human support agent. A page titled Overview is not a source for a numeric limit.
A ten-question test you can run this afternoon
Pick ten questions from real tickets, Discord, or sales calls. Include at least two the docs do not cover. Run them in the current widget or in a dashboard test chat. For each answer, score three things:
- Citation: does it point at a public page you would send a customer?
- Faithfulness: would a writer accept the sentence as a quote from that page?
- Refusal: on the uncovered questions, does it decline, or does it improvise?
Write the results in a sheet. A bot that scores well on covered questions and refuses the rest is safer than a bot that always replies. Run the same ten questions after every major docs change or model swap.
If you want a first pass on coverage before you embed anything, run a gap audit on the docs origin: /tools/gap-audit
What to require from the product, not the prompt
Prompt instructions (“only use the context”) are not enough. The product has to enforce the loop:
- Retrieve with titles, URLs, and version metadata — not a bag of anonymous chunks.
- Show citations in the widget so a human can open the source.
- Decline when confidence is low, then log the question as a content gap.
- Prefer published help articles when they exist, because they are the canonical customer answer.
- Keep crawl scope tight (a section or site, not the whole marketing domain plus /login).
usedocs is built around that loop: cited streaming answers, an honesty gate, and a gap list you can draft into FAQs. The widget is chat with a link out to the help center — visitors read the source page instead of trusting a tab that only redirected. Gaps become drafts. A human still publishes. That is the only safe path from a miss to a public sentence.
Fix the docs, then the bot
A refusal is useful only if someone writes the missing page. After a week of production questions, cluster the misses. Typical themes:
- SSO and SCIM
- Webhooks and retries
- Billing, seats, and plan limits
- Rate limits and quotas
- Version or migration notes
Draft those articles, publish them, recrawl, and rerun the original questions. That is the operating cadence: chat proves what the docs lack; the help center closes the hole; the next visitor gets a citation instead of a shrug. Tools that only chat, or only publish articles, leave half of that loop on a spreadsheet.
If you already use Chatbase, SiteGPT, or a DX assistant, keep them if the citation and refusal test passes. Switch or add usedocs when the failure mode is invented product facts and you need the gap → draft path. Comparisons: /compare/chatbase, /compare/sitegpt.
What not to do
- Do not hide citations behind a “sources” dump nobody opens.
- Do not train on the entire marketing site if the bot is supposed to answer implementation questions.
- Do not auto-publish model drafts to the live help center.
- Do not claim “zero hallucination.” You can refuse, cite, and measure. You cannot promise a language model will never be wrong.
- Do not skip the two questions the docs cannot answer. That is the only honest test of fallback.
usedocs stays on the measurable side of that list. Free is 100 messages per month. Pro is $29 for 5,000. Growth is $79. Business is $299.
FAQ
Is retrieval-augmented generation enough to stop hallucinations?
No. RAG reduces invention when the right chunk is retrieved and the product refuses weak matches. It does not eliminate errors if crawl, chunking, or prompts push the model to always answer.
Should a docs bot always answer?
No. For product facts, a clear “this is not in the docs” plus a human handoff is safer than a guessed endpoint or price.
How do I know if my current widget is inventing?
Ask two questions your docs do not cover. If you still get a specific how-to, the bot is filling gaps from model memory.
Do citations guarantee a correct answer?
No. They let a reader check. Pair citations with a refusal when retrieval is weak, then fix the underlying page.
Where should the widget live — docs site or in-app?
Both can work. Docs-site chat catches evaluation and integration traffic. In-app help catches signed-in users. Restrict each embed to allowed origins.
How does usedocs handle low-confidence questions?
It declines instead of inventing, can capture a lead or escalate, and files the miss as a content gap you can draft into a help article. A human still publishes.