RAG chatbot knowledge base: how to build one that answers correctly

A RAG chatbot is only as good as the passages its search turns up. The model writes from whatever comes back, so the knowledge base is the part you control. If the right passage exists, states the fact plainly and uses words close to the customer's, the answer is usually right. If it doesn't, no model upgrade will save you.

The examples below come from an online store; the same rules hold for a clinic, a salon or a software help desk.

What RAG means, in plain words

RAG stands for retrieval-augmented generation. The term comes from a 2020 research paper that paired a language model with a searchable index of documents, so the model could look facts up while answering instead of relying only on what it absorbed in training.

In a business chatbot it runs in two steps:

1. Retrieval. When you upload a file or add a web page, the platform extracts the text, cuts it into short passages and indexes them for search. When a customer writes, the bot searches that index with the question and picks the closest passages.
2. Generation. A language model reads those passages together with your instructions and writes the reply, in the customer's language.

Picture an open-book exam where a librarian hands the student three pages. Give the student the right pages and you get a good answer. Give them the wrong pages, or none, and they either say they don't know or bluff.

So an answer can go wrong in two separate ways, and they need opposite fixes:

• The right passage wasn't found. The fact is missing, buried in a long mixed document, or written in words nothing like the customer's. Fix the knowledge.
• The right passage was found, and the reply still missed. The passage was vague, contradicted another one, or the instructions pulled the bot somewhere else. Fix the wording or the instructions.

ChatPal follows the same two steps: extract the text, split it into searchable chunks, index them, and compose each answer from the chunks that best match the question.

How a RAG chatbot differs from ChatGPT

A general assistant like ChatGPT answers from what its model learned in training, plus web searches and whatever you paste in. Unless a web search turns up your returns page, it may answer from what other shops usually do. A RAG chatbot answers every customer from your own pages, and you can see and change what it reads.

The model underneath can be the same kind. What changes is where the facts come from.

What to put in the knowledge base

Start from the questions, not from the documents you happen to have. Read your last 50 to 100 customer messages from email, DMs and chat, and sort them by topic. For an online store the list usually looks like this:

• shipping: cost, delivery times, regions, late parcels;
• returns and exchanges: the window, item condition, sale items, refund timing;
• sizing, materials and care, per product line;
• payment methods, discounts and gift cards;
• changing or cancelling an order, warranty and damaged items;
• contact details and support hours.

Every topic needs a source that answers it in plain words. Anything that answers no customer question can stay out: company history, job ads, blog posts, internal procedures. More pages don't make the bot smarter; they give search more near-misses.

Two things don't need a document at all:

• Product catalogs. If you sell on Shopify or Horoshop, connect the store under Integrations instead of uploading a catalog PDF. ChatPal then answers from its own copy of your catalog and keeps it updated: a Shopify copy follows your store within minutes, a Horoshop copy is refreshed from the store's daily export. On other platforms, product answers come only from pages ChatPal crawls and files you upload, so those pages need to stay current.
• Short facts. A phone number, opening hours or a single policy line don't need a document of their own. ChatPal lets you add short FAQ, policy, promotion and contact items directly in Configuration.

How to write documents that search can find

Search matches the customer's words against your passages. A passage wins when it's about one thing, uses the customer's words and makes sense on its own. These six habits get you most of the way.

1. One topic per document or section. A 40-page customer handbook mixes shipping, returns and care into passages that half-match everything. Split it into a shipping document, a returns document and a sizing document. If you keep one file, give each topic its own heading.
2. Headings that sound like questions. "Can I return sale items?" beats "Section 4.2: Exceptions". The heading stays attached to its passage and gives search the customer's own words.
3. Every section stands alone. The bot reads a passage without the page around it, so "as mentioned above" and "these items" mean nothing. Repeat the subject: "Sale items can be returned within 14 days."
4. Exact numbers, dates and names. "Usually a few days" can't answer "will it arrive by Friday?". "Standard shipping in the US takes 3 to 5 business days and costs $6.95, free over $75" can. Name products the way your catalog does and the way customers do: "Trail Runner 2, our green trail shoe".
5. One version of each fact. If the return window is 30 days on your site and 14 days in an old PDF, the bot may quote either. Delete the old file rather than adding a correction next to it.
6. Plain text over layout. Text pulled from tables, multi-column PDFs and slides often comes out jumbled, and a scanned PDF has no text at all. Paste the text into a Word document and upload that.

Before and after: a returns section

Here is a returns section the way many stores write it:

Returns

We want you to love your purchase! If something isn't right, you can send it back within the usual period as long as it's in its original condition. Some exclusions apply (see above). Refunds are processed promptly once we receive the item. For exchanges, please contact us.

A customer asks "can I return shoes I wore once on a run?". Search will probably find this passage, but there is no answer in it: no number of days, no definition of original condition, and the exclusions are "above", in a passage the bot may never see. The bot either guesses or says it doesn't know.

The same content, rewritten for retrieval:

How many days do I have to return an order?

You can return any order within 30 days of delivery for a full refund to your original payment method. The refund reaches your account 5 to 10 business days after we receive the parcel.

Can I return shoes I have worn?

Shoes must be unworn, with clean soles and the original box. Shoes worn outdoors can't be returned, but our 6-month warranty covers defects.

Can I return sale items?

Items bought at 30% off or more are final sale and can't be returned or exchanged. Items with a smaller discount follow the normal 30-day policy.

How do I exchange for a different size?

Start a return for the original pair and place a new order for the new size. Exchanges for the same model ship free.

Each block answers one question, names its subject and carries its own numbers. The figures are invented for this example; use your own.

Files or links: which to use

Prefer links for anything your website already says, so the page stays the one place you edit. Four details about links that save a confused afternoon:

• Single page only, when ticked, indexes just the address you entered, such as your returns page. Leave it unticked and ChatPal also crawls the pages linked from it, up to your plan's link limit. For a whole site, add the homepage; ChatPal finds the pages itself.
• Machine-readable addresses are refused: sitemaps, robots.txt, RSS feeds, llms.txt, and addresses ending in .xml, .json, .csv, .txt or .md. Add the page a customer would read instead.
• Pages that are mostly navigation, cookie banners or a login screen are dropped, so a link can show Error even though the page loads fine in your browser.
• One language version of your site is crawled: the one you point at. The bot still answers in whatever language the customer writes in.

Adding sources in ChatPal

1. Click Configuration and open your assistant.
2. For web pages, open the Links tab and paste the address. Tick Single page only for just that page, then click Add Link.
3. For documents, open the Files tab, click Upload files and choose PDF, Word (.docx) or plain text files. You can pick several at once.
4. Wait until each source shows Ready. Processing means the bot can't use it yet. Error means it failed or held no usable text; for a link, check that the page opens without a login.
5. Open a source to read the text ChatPal extracted. That text is exactly what the bot has, so it is the first place to look when an answer seems off, and you can edit it there.

How many files and links you can add depends on your plan; Free has the smallest limits.

Keeping it current

An outdated knowledge base is worse than a small one: a bot quoting last month's shipping price does more harm than one that says it doesn't know.

• Links: after you change a page, click Reindex on it the same day. Don't count on the bot noticing the change by itself.
• Files: delete the old version and upload the new one. Don't upload returns-v2.pdf next to returns.pdf.
• Edits you make in ChatPal to extracted text last until the page is crawled again or the file is uploaded again, so make lasting changes at the source.
• Time-limited facts, such as a sale, holiday hours or a pre-Christmas shipping cutoff, need their end date written into the text and a reminder in your calendar to remove them.

Once a month, scan the list of sources for anything you no longer sell or do.

Checking which source an answer used

The quickest way to watch retrieval at work is ChatPal's Playground, in the left navigation under Chats. Ask a question the way a customer would, typos included. Under each answer, a short line shows how many sources it used and how confident it was. Click it to see:

• each source it used, with a score; click one to open it with the passage it read highlighted;
• what the assistant actually searched for, which often explains a miss better than the answer itself.

Read what you find like this:

Click Why this answer? under a reply and Copilot, the assistant built into your dashboard, explains what the bot read. Tell it what the answer should have been, and it suggests up to three fixes; Apply shows the before and after first, and the change goes live on your website too. Ask again then re-asks the question against your current setup, and Show changes marks what moved.

Every Playground reply and every Copilot reply counts toward your monthly messages, the same as a reply to a customer: Free has 100 a month, Starter 500 for $20 a month (pricing, checked 13 October 2026). Applying a fix is free.

Once an answer is right, click Save as test so it can't quietly break again. How to test an AI chatbot turns that into a full test set, and chatbot instructions that don't hallucinate covers the other half: what the bot should do when the knowledge base has no answer. If your FAQ already lives on a Shopify store, turning a Shopify FAQ page into an AI assistant walks through that case.

The cheapest way to start is the free plan: add your shipping and returns pages as links, ask the Playground ten real customer questions, and open the sources under each answer. Whatever comes back wrong is your to-do list for the knowledge base.