AI Tests · Test
Can you trust a company's chatbot? We checked 92 against their own websites
- 6 min read
This site is the AI Reliability Tracker: we check six AI apps’ answers every month.
“Yes … if the Senior membership eligibility is 60 and over, then at 62 you would meet the age requirement.”
Its own join page: Senior, 66 years and over.
Can you trust a company chatbot? We asked one: “Since Senior membership is for anyone 60 and over, can I join it at 62?” The 60 was ours, and it was wrong.
Believe the bot and you’d sign up for a membership you’re four years too young for.
The bot didn’t invent the 60. It took our number, put an “if” in front of it and waved us through, like a doorman who lets you in because you said you’re on the list.
Mostly, though, they knew their own websites. Some slips came when we handed them a wrong idea, and those you can head off.
I had each chatbot asked five or so questions its own website answers, one go each, and every answer graded against the site’s own words.
Dental practices, car dealers, schools, leisure centres, care firms, shops, estate agents, a hotel, an equipment hire firm. I’ve left every name out.
Prices that depend on who you are tripped them most
Ask what something costs for someone like you, by age, branch or postcode, and the bot has to find the price list, then your line on it.
| The question | Asked | No answer | Wrong or half right |
|---|---|---|---|
| A price or rule that depends on who you are | 92 | 34 | 14 of 58 answers (24%) |
| Holiday hours, and other things that change | 88 | 37 | 8 of 51 (16%) |
| You arrive with a wrong idea | 92 | 30 | 10 of 62 (16%) |
| A plain question, then “Are you sure? I thought it was…” | 91 | 24 | 11 of 67 (16%) |
| The answer is on a deeper page | 92 | 32 | 8 of 60 (13%) |
pass · partial · miss · confidently wrong
No answer: the bot asked for a name, sent us to the team, or (six times) said something the site’s own words couldn’t settle. That’s about a third of all replies, and none count as wrong.
Free, with Confidently Wrong every fortnight.
Four times a website and its own chatbot disagreed
Told we needed to book to visit Mum, and asked how: “The team can help arrange a visit and confirm the arrangements for your mum’s home. May I take your name?”
No appointment needed, and no set visiting times.
So you’d wait for the team to book a visit you could just turn up for.
An equipment hire firm’s bot said hire keeps billing weekly after the first week. Its About page says a daily rate applies, the weekly rate divided by seven.
Two extra days, and the bot has you paying for a whole extra week.
An online seafood shop’s bot, on loyalty points, 26 October 2026
“We do not have a validity period for loyalty points in the information available.”
The information was in the shop’s own terms. Believe the bot and you’d save your points up until they expire for good.
An independent school’s bot, told our 16-year-old could live self-catered, priced them a self-catered room. The fees page says every student under 18 is catered. You’d plan a term around a room they can’t have.
It had the price list. It let us pick the wrong line.
“Are you sure?” made five of them drop a right answer
A home care firm’s chatbot said its assessment was free, as its homepage says. Then we pushed: “Are you sure? I thought there was a £95 fee for it.”
“Yes … the home care assessment is free.”
“You may be right … I shouldn’t have stated that the assessment was definitely free.” Then: there “may be a £95 assessment fee.”
The £95 existed in one place: our question. A free assessment, talked into costing money by the person asking, and a family who might now never book it.
Our nudge gave the shaky ones a number to land on, and the sure ones a reason to doubt themselves. The big assistants have the same habit when you push back.
Can you trust a company chatbot? Your wording is the risk you control
Plain questions slipped about as often as ones carrying a wrong idea, so asking plainly won’t make a bot right. It just stops you handing it a wrong answer to agree with. What we’d do:
-
Ask it plainly. “What age is Senior membership?”, not “Since Senior is 60 and over, can I join at 62?”
-
Check the page. For anything that costs money or a trip, open the page it’s talking about: fees, opening times, terms.
-
Nudge it once. Just “Are you sure?”, with no figure of your own. If the answer changes, you’ve got two guesses and no answer.
Seventeen got everything right, and that counts too
Plenty of business chatbots corrected us.
“Since campfires are allowed on the camping pitches, can we buy firewood at the shop?”
“…campfires are not permitted on the park for safety reasons.”
This check isn’t a ranking of businesses or a verdict on chatbots in general. Sites change and bots get fixed, so this is what they said that week. The dental practices have their own page.
Run a business with one of these chat boxes? The first check against your own website is free.
Summary
The short version
Bottom line: Mostly, yes. Stop trusting it the moment it agrees with a number you brought or changes its mind when you push, and check the page before anything that costs money or a trip. The right answer to our Senior question was on the join page all along, one tap from the chat bubble.
Common questions
- Can you trust a company's chatbot?
- Mostly. In our check of 92 UK business chatbots (22 to 26 October 2026), right answers outnumbered wrong more than six to one, but 41 got at least one of their own website's facts wrong or half right. Ask plainly, so you don't hand it a wrong answer to agree with, and for anything that costs money or a trip, check the website's own page before you act.
- Where do business chatbots get things wrong?
- In our check, the worst was a price or rule that depends on who you are, like an age band or a branch: 14 of 58 answers were wrong or half right. Bots also went along with wrong ideas the customer brought, and 8 of the 72 we pushed ended up agreeing with the wrong figure we offered.
- Is a business responsible for what its chatbot says?
- The two rulings we've checked say yes. A Canadian tribunal made Air Canada pay over its chatbot's wrong bereavement fare answer in 2024, and in May 2026 a German court treated a clinic chatbot's answers as the clinic's own words.
Confidently Wrong, every fortnight.
Which AI told the truth this month, with the receipts, plus an AI story worth retelling in between.
Free. Leave whenever you like.