AI cites the wrong source: I put 6 UK questions to 5 assistants and opened every link

The £2,500 figure was right. Gemini told me that using a handheld phone while driving a lorry or bus in the UK can cost you a £2,500 fine, and it can. What it did next is the reason for this post. It told me the source was Police.uk.

Police.uk is the find-your-local-force and crime-map website. It is a real, official, live site. It has no more business setting out driving-fine schedules than your local council’s bin-collection page does. But there it was, cited with a straight face next to a correct number. Gemini had cited the wrong source, and if I hadn’t gone looking I’d never have known.

That is the whole test, and the whole worry. A dead link gives itself away. You click it, it goes nowhere, you know not to trust it. The dangerous one is the link that works. It opens a real page, on a real official domain, and it passes the sniff test so cleanly that you stop checking, which is precisely when the source that doesn’t say what the AI claimed slips through.

What I did

I took six everyday UK questions and put each one to five AI assistants on 7 July 2026, each in its default model with web search on, memory off, and a fresh chat each time. The tiers weren’t identical, and that matters, so here they are: ChatGPT on the free tier, Claude on Max (Opus 4.8), Gemini on Pro, Perplexity on Pro, Grok on its default fast model. The board that follows is one run per assistant per question; I then re-ran the whole set a second time the same day, which I come back to near the end. For every answer I asked for the source, then I went to every source it named, whether it came as a clickable link or just a name in the text, and checked whether that page actually backed the claim, grading it against a source list I’d verified myself first.

The six questions weren’t random. Each was engineered to tempt a lazy retrieval toward a plausible-but-wrong page, the sort of everyday thing a person actually asks: online returns and your rights, stamp duty on a £300,000 home, free childcare hours for a 9-month-old, the State Pension age, the speed limit in Wales, the maximum fine for using your phone at the wheel. The trap in each case is a genuine, authoritative-looking page that sits right next to the correct one. Change-of-mind returns are governed by one law, but a more famous law about faulty goods has a real page a model can grab. Stamp duty is a different tax in Scotland and Wales, but the England page is the one everyone links. Childcare rules changed last year, and the old page is still up.

That’s one run each, which makes this a snapshot, not a rate, and I’ll come back to what that does and doesn’t let me say. Worth flagging now: the one assistant that never slipped, ChatGPT, was the only one on a free tier. So this isn’t a case of paid tools winning.

The board

Five assistants, six questions each: here’s who held and who slipped.

A word on how I’m counting a “slip”, because two different faults sit in that column. One is the wrong-source citation: a real page that has nothing to do with the claim, which is the whole subject of this post. The other is a correct page that’s incomplete: right figure, right official source, but it leaves out something a reader needs. Gemini has one of each. Only the first is a citation pointing at the wrong building. I’ll be clear below about which is which. (Gemini’s sixth answer, on the returns question, sits in neither column: it gave the right answer but attached no source at all when I asked for one, a third kind of miss I’ve left uncounted rather than force it into “held” or “slipped”.)

Two things to say straight away, because the headline “AI cites the wrong source” can slide into “all AI is broken”, and that isn’t what I found. ChatGPT and Grok cited a page that held the claim on all six questions. This is a real split, not a wipeout. And every miss below is the citation going wrong, not always the answer, which turns out to be the more unsettling half.

The three that slipped, with the record

Gemini sourced a driving fine to a crime portal. This is the Police.uk one from the top. On the handheld-phone question, Gemini gave the right penalties, £200 fixed penalty and six points, £1,000 in court, £2,500 for a lorry or bus, but pinned nearly every figure to a commercial or quasi-official secondary site: Confused.com, the RAC, and Police.uk for the £2,500. Here is the relevant stretch, exactly as it came back:

The one genuine gov.uk link sat at the very bottom as a generic “Source: Official UK Government legislation via GOV.UK”, attached to nothing in particular. The correct government page carries all three figures. Gemini had reached past it and hung the £2,500 on a crime-map portal instead. And it named those secondary sources as bare text, not one clickable link among them, so checking meant going and finding each page myself, which is exactly the friction that stops most people bothering.

Perplexity gave a childcare figure that its own source contradicts. I asked how many hours of free childcare a working parent of a 9-month-old in England gets right now. Here is what it opened with, word for word:

It said 15 hours, and described the 30-hour offer as “starting from September 2025”, as if it were still ahead of us. September 2025 was about ten months in the past. The current answer is 30 hours. What makes this the sharpest one isn’t just that it was out of date. It’s that Perplexity cited a stack of real gov.uk pages, and at least two of them, read properly, say the thing it was denying, one of them describing the 30-hour tier for exactly this age group. The links were real. The model read them as if the calendar hadn’t moved.

The links were real. The model read them as if the calendar hadn't moved.

Claude backed the right fine with a link that goes nowhere useful. Claude gave the correct court fines on the phone question, but sourced the £1,000 and £2,500 to a solicitor firm’s page rather than to gov.uk, which it cited separately for the £200 fixed penalty. Same shape as Gemini’s miss, milder: right numbers, but the specific figure a reader most wants, the maximum fine, was pinned to a commercial blog when the primary page was open in the same answer. Click that link to check the £2,500 and it redirects you to the firm’s homepage, which carries none of the figures at all. Right number, and a citation that quietly leads nowhere.

I’m counting Gemini’s stamp-duty answer as its second slip, and it’s the gentlest of the four, worth being precise about. Gemini gave the correct £5,000 for England and Northern Ireland, on the correct gov.uk page. Nothing wrong on the page it cited. What it did was present that as the answer to a question about “a £300,000 home” without mentioning that Scotland and Wales run different taxes at different rates, so a reader in Edinburgh or Cardiff would take an England figure off a real England page and never learn it didn’t apply to them. ChatGPT and Grok both caught that trap out loud, unprompted. That’s the difference between a correct citation and a complete one.

The part that should worry you

After I’d graded the board, I went back through all three wrong-source answers looking for a single hedge, any “I’m not certain”, any lower-confidence wording, and found nothing. Every one of them arrived as plainly as the correct answers did. That’s the finding underneath the finding: no visible seam between the citations that held and the ones that didn’t. The Police.uk citation looked exactly like a good citation. The stale childcare figure was stated as plainly as a true one.

That’s what makes a working-but-wrong link more dangerous than a broken one. The broken link is honest about being broken. This isn’t. It comes dressed identically to a real source, in the same calm finished tone the assistant uses when it’s right, and nothing in the answer tells you which is which. The wrongness is invisible from where you’re sitting. You only see it if you open the page.

For contrast, look at what the clean answers did. ChatGPT opened its stamp-duty answer with “Assuming you mean England or Northern Ireland” and named the Scotland and Wales divergence off its own bat. Grok cited one correct gov.uk page six times out of six and did the same. When these tools handle a source well, they often show their working, they flag the assumption, they name the boundary. The absence of that signal isn’t proof of a problem, but its presence is a decent sign of a good one.

I re-ran them, because one answer is not a rate

A single run is a story, not a pattern, so I put every question back through all five a second time, the same day. Two things came out of that, and both are the point of the exercise.

The wrong-source habit held, and more plainly than I’d expected. Asked again for the phone-driving fine, Gemini sourced the £2,500 lorry-and-bus figure to Police.uk a second time, the same crime-data portal, and hung the £1,000 car-driver fine on a price-comparison site. The same wrong source once is a slip. The identical wrong source twice, from a fresh chat, is a habit.

The stale childcare figure did the opposite. On its re-run, Perplexity corrected itself and gave the right 30 hours, from the same pages it had misread the first time. That is the more unsettling half, honestly. The same question, on the same day, handed me the wrong answer once and the right one the next, in the identical confident voice both times. You cannot lean on it being wrong any more than you can lean on it being right.

So I am not claiming a percentage, and you should be wary of anyone who does off a handful of runs. What I have is a dated record of what these tools did, and one re-test of whether it held. The source failure held; the stale figure went the other way the second time. Both point at the same move: open the source yourself.

What this changed for me

I already open links on answers that matter. This tightened the rule: for anything I’m about to act on, a fine, a rule, a rate, a date, I don’t check whether the link works, I check whether the page it opens actually says the thing. It’s the difference between “does this citation exist” and “does this citation hold”, and only the second one catches Police.uk standing in for the Highway Code.

It’s the same ten-second habit I’ve written about when checking whether an AI’s sources are real at all: open the page, find the specific claim, ask whether the page is even about the thing you asked. This test just widened it from one tool to five and made the traps harder, and the habit held up as the only thing that reliably worked. If you’re picking a tool to lean on for this kind of everyday research, it’s worth knowing that free and paid tiers can behave differently on exactly these questions, which is part of why I keep a running audit of the free AI tools for this sort of research rather than trusting one blanket recommendation. These four slips join the rest of the record in the running log of AI mistakes I catch.

One more time on the honest limit, because it’s load-bearing. The board was one run per question, six questions, one day, with a second run the same day to see what held. It tells you who slipped and who held on the traps I set, with the dated evidence, and nothing more. It is not “Gemini gets sources wrong a third of the time”, and I’d ask you not to read it that way. A single confident wrong citation is enough to matter, though, because you weren’t going to check the one that looked fine. That’s the trap, and the crime-data portal cited for a driving fine is the cleanest proof of it I’ve got: right number, wrong building, said without a flicker of doubt.