The most important thing an AI can say is 'I don't know'




The failure is not technical, it is contractual
When an assistant invents an answer, the problem is not that a model was wrong. The problem is that your business said it. The customer did not talk to a model — they talked to you, in writing, at an address with your name on it.
Depending on where you are and what was said, you may be held to it. Even if you are not, you have a choice between honouring something you never agreed to and telling a customer that the thing your website told them was not true. Both are bad, and the second is worse.
Where it actually goes wrong
In practice, invented answers cluster in a few places:
- Prices, especially "starting from" and "roughly".
- Timescales — how long something takes, when you could fit them in.
- Whether you cover something. An insurance, a postcode, a brand.
- Outcomes. Anything that sounds like a promise of a result.
What these share is that they are the questions customers most want answered, which is exactly why an assistant trying to be helpful will reach for them.
What good behaviour looks like
Answer only from what the business has written. Not from general knowledge about your industry. If the answer is not in your pages or your written answers, there is no answer to give.
Say so, plainly, and hand over. "I don't want to guess at that — let me get someone to confirm" is a perfectly good customer experience. Being told the truth is not a disappointment; being told something false is.
Record the gap. Every question it could not answer is a question your customers are asking. If six people ask about weekend hours in a week, that is not a failure of the AI. It is a missing page.
Make the boundaries yours, not the vendor's. You should be able to say: never quote above this amount, never discuss contract terms, never give clinical advice. These are business decisions, not model settings.
The uncomfortable trade
An assistant that refuses to guess will resolve fewer conversations by itself than one that improvises. On a dashboard, the honest one looks worse.
That gap is the whole point. The conversations it escalated are the ones where guessing would have cost you something — and the ones it handled are the ones you can actually trust it with. A containment rate of 100% means it is making things up.