How to write chatbot instructions that stop it making things up

A chatbot makes things up for three reasons: the answer isn't in its knowledge, its instructions are vague about what to do then, and the model itself leans toward giving some answer rather than none. You can't retrain the model, but the first two are yours to fix, and a longer prompt tends to make them worse rather than better.

Below are the causes, a before-and-after set of instructions for an online store, the "I don't know" rule written so it works, and a list of things that don't belong in instructions at all.

Why chatbots make things up

A hallucination is a confident answer that isn't true. In a business chatbot it nearly always traces back to one of three things.

The knowledge doesn't hold the answer. A customer asks whether you ship to Alaska. Your shipping page says "we ship across the US". The bot has a passage that is almost about the question, and it fills the gap with a guess. When you find an invented answer, look here first: it is the cause you can fix fastest.

The instructions are vague, or push it to answer. "Always be helpful." "Never leave a customer without an answer." "You are an expert on all running gear." Each of these tells the bot that a guess is better than admitting a gap.

The model leans toward answering. A 2025 research paper argues that language models hallucinate because the way they are trained and scored rewards a guess over an honest "I don't know", like a multiple-choice exam with no penalty for wrong answers. Your instructions are where you change that incentive for your own business.

Two smaller causes are worth checking too: two sources that disagree (an old PDF says 14 days, the website says 30), and facts that went stale (last season's sale, still sitting in the instructions).

Facts go in knowledge, behavior goes in instructions

If you keep one rule from this page, keep this one: instructions say how to behave, and the knowledge base says what is true.

Why the split matters:

• Instructions apply to every reply; knowledge is searched. A rule belongs where it is always read. A fact belongs where it is found when a question needs it.
• Long instructions bury the rules. Research on long inputs found that models use information at the start or end of a long text better than information in the middle. Put your hand-off rule in paragraph 14 of a 3,000-word prompt, and expect it to be missed some of the time.
• Facts kept in two places drift apart. When a price changes, you update the website and forget the instructions, and now the bot has two prices to choose from.

Before: instructions that invite made-up answers

Here are instructions of the kind many stores start with, for an example running-shoe store:

You are Max, the friendly expert assistant for Northline Running! You know everything about running shoes and gear. Always be helpful and positive and never leave a customer without an answer. Make every customer feel valued and try to close the sale.

Our products: Trail Runner 2 ($139, waterproof, sizes 6-13), Road Glide ($119, sizes 5-14), Kids Sprint ($59)... [40 more products]

Shipping is fast and free on most orders. Returns are easy, within 30 days. We're running 15% off everything with code FALL15. If customers are unhappy, offer them a discount to keep them happy.

What goes wrong, line by line:

• "You know everything about running shoes" gives it permission to answer from general knowledge: whether a shoe suits a foot condition, or how a competitor's model compares.
• "Never leave a customer without an answer" is a direct instruction to guess.
• The product list goes out of date within weeks, competes with your real catalog and pushes the rules further down the prompt.
• "Fast and free on most orders" is not a fact the bot can use. Asked "is shipping free to Alaska?", it has to invent where "most" ends.
• The discount code has no end date, so the bot will quote it long after the sale.
• "Offer them a discount" sets no limit, so the bot decides the amount. Expect promises you never approved.

After: short instructions with clear limits

The same store, rewritten:

You are the customer assistant for Northline Running, an online store selling running shoes and apparel in the US.

Help customers with products, sizing, shipping, returns, exchanges and orders. Answer only from your knowledge and the product catalog. Keep replies to two or three short sentences unless the customer asks for more detail.

If the answer isn't in your knowledge, say “I don't have that information” and offer to pass the question to our team. Never guess a price, a delivery date, stock, a policy detail or a product feature.

Never offer discounts, refunds, free shipping or compensation that aren't in your knowledge. Don't give medical advice: describe the product and suggest the customer ask a doctor or physical therapist.

Pass the conversation to our team when the customer asks for a person, is upset or complaining, wants a refund outside the policy, or reports a damaged or faulty item.

It's much shorter, and every line is about behavior. The facts it used to hold now live where they can be kept current: products in the catalog (a connected Shopify store, or product pages and files), a shipping page that names the regions and costs exactly, and the sale as a promotion with an end date.

Read any set of instructions against three questions. Does every line change how the bot behaves? Is there a fact in here that belongs in the knowledge base? Does it say what to do when the answer isn't there? If ChatPal's Setup page drafted your instructions from your website, put the draft through the same three questions before you go live.

In ChatPal, instructions sit in the Instructions field in Configuration. A change takes effect at once, on every channel the assistant answers.

The “I don't know” rule, written to work

"Do not make up information" is a fine line, and ChatPal's own example instructions include it. On its own, though, it says what not to do and nothing about what to do instead, so under pressure the bot still produces something. Give it a script:

1. Say what's missing, in plain words: "I don't have that information."
2. Offer the next step: a person, an email address, a phone number.
3. Stop there. No long apology, no speculation about what the answer might be.

Anthropic's documentation lists explicitly allowing the model to say "I don't know" as one of the basic ways to reduce false answers, and adds that no technique removes them entirely. That last part matters: good instructions lower the rate, and testing catches what's left.

Make the hand-off real

"Offer to pass it to our team" only works if someone receives the chat. In ChatPal, hand-off is off by default. To switch it on:

1. Open Configuration, open your assistant and click Additional Settings.
2. Switch on Enable human-in-the-loop; it saves at once.

With it on, the bot hands the chat over when a customer asks for a person, and you can set up categories, such as a "Complaint" category, that hand off as well. The bot then goes quiet in that chat, your team gets an alert by email or Telegram, and the conversation waits in Chats under Handed off to a human until someone clicks Return to AI. On your website, the chat also asks the visitor for a phone number or email in case they leave the page.

With it off, the bot can't hand over. On most channels it offers your other contact details instead, or says it can't connect the customer to a person in this chat. So if you leave it off, make sure your support email and hours are in the knowledge base, so "offer the next step" has somewhere to point.

What not to put in instructions

• Your catalog. Names, prices and stock change too often. Connect your store, or keep product pages in the knowledge base.
• Long policy text. A pasted returns policy is read with every message, goes stale with every change and crowds out your rules. Put it on a page or in a file.
• Your FAQ. Each answer belongs in the knowledge base, where search finds it by the customer's own words.
• Promo codes without end dates. Add the offer as a promotion or on a page that states when it ends, and remove it when it does.
• "Always", "never fail", "you know everything". Any line that makes having an answer more important than having a correct one.
• Rules that contradict each other. "Keep answers short" next to "always explain all our options in detail" leaves the bot to choose.
• Sales pressure. "Try to close the sale" can turn into pushiness, or into promises.

High-risk answers to pin down

A wrong answer about care instructions is embarrassing. A wrong answer about a refund costs you money. Cover each of these explicitly, in the knowledge or in the instructions:

• Money: discounts, refunds, price matching, compensation.
• Dates: delivery guarantees, restock dates, the last day to order before a holiday.
• Stock: whether an item is available in a given size. With a connected Shopify store the bot checks its copy of your catalog; otherwise it knows only what your pages say.
• Health, safety and legal claims: "good for plantar fasciitis", "safe for children under 3", "fully waterproof".
• Other companies: competitors' products, a carrier's own rules.

For each one, either the knowledge base holds the exact answer, or the instructions say: don't answer this, pass it on.

How to check the fix worked

Open Playground in ChatPal and ask the questions that tempted the old version to guess. Five good bait questions for a store:

1. "Do you ship to Alaska?" (when your shipping page doesn't say)
2. "Give me a discount code."
3. "Will it arrive by Friday if I order now?"
4. "Are these good for flat feet?"
5. "I want to talk to a person."

Under each reply, a short line shows how many sources the bot used. A specific price, date or promise with no source behind it came from somewhere you never approved. Click Why this answer? to see what the bot read and where it went wrong, fix it, then click Ask again. Once the reply is right, click Save as test, and give the test a Custom AI evaluation criteria rule such as "the reply must not offer a discount code" so it is graded automatically after every future change.

Each Playground reply, Copilot reply and graded test counts toward your monthly messages: 100 on Free, 500 on Starter for $20 a month (pricing, checked 13 October 2026).

If you already have a ChatPal assistant, open its Instructions now, run them through the three questions above, then ask the five bait questions. It takes about twenty minutes. How to test an AI chatbot has a complete 20-question test set, and how to structure a RAG chatbot knowledge base covers the facts side: what to write so the bot has a real answer to give.