21 August 2026

AI makes things up: teaching your team to check

There is an important difference between a colleague who does not know something and an AI that does not know it. The colleague says they are not sure and will look it up. The AI answers. With well-built sentences, with specific figures, with an article of law that sounds perfectly plausible. And sometimes it is false.

This is not an occasional glitch or a bad tool: it is how these systems work. A language model does not query a database and hand back what is in it; it predicts which word fits after the previous one, based on all the text it has read. When it has the information, it gets it right a lot. When it does not, it does not go quiet: it fills the gap with whatever sounds right. The industry calls this hallucination, and the real problem is not that it happens, but that it happens in exactly the same confident tone as when the answer is correct.

Where it hurts a small company

An accountancy firm asked a free AI tool for the filing deadlines of a tax form, to put them in an information email to clients. The answer arrived in five seconds, well written and with exact dates. Two were correct; one was from a previous year. Nobody checked, because the text looked the part. The email went out to 300 clients and they spent two days contradicting themselves.

The cost was not a fine. It was the call to every annoyed client and the feeling that, if they get a date wrong, perhaps they get other things wrong too. Trust is an accountancy firm's product.

The same traps show up everywhere: a quote with a catalogue price that no longer exists, a product sheet with an invented measurement, an email to a client citing a contract clause nobody has read.

The answers that must always be verified

You do not need to distrust everything: you need to know where the risk is. These five categories get checked, no exceptions.

Figures, dates and deadlines. Prices, percentages, VAT, due dates, cut-off dates. This is where the model invents most naturally.

Regulation and legal obligations. Articles, laws, requirements. They may exist, they may have been repealed, or they may never have existed at all.

Sources, citations and links. If the AI cites a study, a report or a web address, open it. Invented links are common and look impeccable.

Data about your own business. The model knows nothing about your catalogue, your opening hours or your terms unless you have given them to it. If it answers as though it knows, it is imagining.

Names of real people, companies and products. It mixes up records, attributes the wrong job titles and invents references that do not exist.

And one cross-cutting warning sign: the more confident the answer sounds and the fewer qualifications it has, the more worth checking it is. A text that never says "it depends" or "check the official source" when the subject clearly depends is suspicious.

How to explain it to the team without banning anything

Banning AI does not work: people use it anyway from their phones, and then you do not find out. What works is a one-sentence rule, short and memorable: AI produces drafts, not sources. Nothing carrying a figure, a date, a law or a client's name leaves the company without a person having checked it.

From there, three habits that cost very little to install.

Always ask for the source. Where did this come from? If the AI does not give one, or gives a link that does not open, the answer is not usable.

Ask twice, in different ways. If the two answers do not match, neither is reliable.

Label what is a draft. A document generated with AI and not yet reviewed should say so. Accidents usually happen when somebody picks up a draft thinking it was the final version.

It is also worth making clear that flagging an error is not embarrassing. If somebody spots that the AI slipped a false figure past them and says so, the company has just saved itself a problem. If they are afraid to say it, the error reaches the client.

And if it has already gone out

Correct it before the client corrects you: a brief message, no long excuses, with the right information. Check whether the same text has been reused elsewhere (website, quotes, email templates) and fix them all. And note down what happened: almost always the gap is not the AI's, it is that nobody was clear on who was supposed to review that before it went out.

Using AI with your head is not working in fear: it is what lets you genuinely benefit from it. The team that knows where the limits are is the one that saves time without gambling its reputation.