How to check if a ChatGPT citation is real or fake

The £2,500 figure was right. On 7 July 2026, Gemini told me the maximum fine for using a handheld phone while driving a lorry or bus in the UK is £2,500, and it is. Then it told me where it got that from: Police.uk. Police.uk is the find-your-local-force and crime-map website. It is real, official and live, and it has no more to do with driving-fine schedules than your council’s bin-collection page does. The number was right. The source underneath it was nonsense.

If I hadn’t opened the link, I’d never have known. And that is the whole problem with checking AI citations, in one line: a dead link gives itself away, so you go and check it, but a live link to a real official page passes your sniff test, so you stop. That is exactly where the bad citation survives. The working link is the one to worry about.

This is a how-to, not a hit piece on any one tool. The check below took me about ten seconds each time, it works on ChatGPT, Gemini, Claude or Perplexity, and it is the only move I’ve found that catches the citation that looks fine and isn’t. Everything else in this post is here to show you what “isn’t” looks like in the wild, using real answers I’ve already opened and graded.

The two ways a citation goes wrong

There are two separate failures here, and most advice on the subject squashes them into one.

The first is fabrication: the model invents a source that doesn’t exist. A citation to a study nobody wrote, a link that goes nowhere. This is the well-known one, and it is the easy one. The link is dead, or the paper isn’t real, and you catch it the moment you click. It happens most when the model is working from memory rather than searching, and the numbers can be startling: one 2023 study found an older ChatGPT inventing 18% of the references it cited, rising to 55% on the weaker model tested.

The second is misattribution: the model cites a source that does exist. The page is real, it loads, it’s often from an authoritative site, and it still doesn’t contain the claim. Wrong figure, wrong page on the right site, or a rule stitched together from several places and pinned on one. This is the Police.uk case: the right answer handed over with a straight face and a footnote pointing at the wrong building. And it is the one that slips past you, because the page passes every test your eye runs on it.

Here is the part that surprises people. Turning on web search doesn’t fix this. It fixes the first failure and quietly makes the second one more common. Search kills most fabrication, because the model is now pulling from pages that actually exist, and it hands you a real link every time. But “real page” and “page that backs the claim” turn out to be two different things, and the tidy citation pills under a searched answer make it feel checked, so almost nobody opens them. That gap is where a misattributed source lives. I’ve written separately about how web search makes AI differently unreliable rather than more reliable, and this is the cleanest example of it.

The check: ten seconds, four steps

The whole method is that you look at the page instead of the link. Here is how I run it on any answer I’m about to act on.

1. Open the cited page. Not “does the link work”. Actually open it. If the link is dead or goes somewhere unrelated, you’ve caught a fabrication and you’re done.
2. Find the specific claim on the page. If the claim is a number, a fee or a figure, search the page for it: Ctrl-F “0.19”. If it’s a rule, read the heading and the first paragraph. You are looking for the exact figure or the exact rule the AI attached to this source, in plain sight on the page.
3. Ask whether the page is even about the thing you asked. This is the Police.uk test. A crime-map portal is not a fine schedule, and a page about moving abroad is not a page about transfers. If the page is about a different topic, the citation is wrong however official the domain looks.
4. If the claim isn’t on the page, treat the answer as unsourced. It might still be true. But you no longer have a source for it, so go and find one before you rely on it. A citation you haven’t checked is an unsourced answer with better presentation.

Steps 2 and 3 are the ones that do the work, and they are the ones an existence-checker can’t do for you. A tool can tell you a page loads. Only you can read the page and see that it’s about the wrong thing.

Three tells worth knowing by sight

Once you’ve done this a few times, the bad citations start to rhyme. These three came out of two dated tests I ran on everyday UK questions, and each one is a shape you’ll see again.

A secondary site standing in for the official one. On the phone-driving question, Gemini hung the correct £2,500 figure on Police.uk and the £1,000 figure on a motoring association’s page, when the one gov.uk page carries all three fines. Right numbers, secondary sources, and the genuine government link buried at the bottom attached to nothing. When you see a motoring body, a comparison site or a law firm’s blog cited for a figure that a government page would obviously hold, open the government page yourself. The full board is in the test where I put six UK questions to five assistants.

A real official page that’s the wrong page on the right site. Asked for the UK ISA transfer rule, ChatGPT cited a real, live gov.uk page, one about what happens to your ISA if you move abroad or die. The rule it was backing lives on a different gov.uk page entirely. Same trusted domain, wrong page, and if you’d clicked to reassure yourself you’d have landed on an official government site and skimmed past the mismatch. That’s the ChatGPT sources test in full.

A page that’s still up but out of date. Asked how many free childcare hours a working parent of a nine-month-old gets now, Perplexity said 15 and described a rollout that finished ten months ago as still to come, citing real gov.uk pages that, read properly, say the opposite. The page was genuine. The reading was stale. When a rule changed recently, check the date on the page, not just the domain.

It isn’t every tool, every time

It would be easy to read all this as “AI can’t cite anything”, and that isn’t what I found. On the six-question test, two of the five assistants cited a page that held the claim on every single question, and two of them flagged a jurisdiction trap I hadn’t asked about, unprompted. On the ISA question, one tool cited the correct page and quoted the exact line that contains the rule, on the same day another tool got it wrong.

That’s what makes the check worth ten seconds rather than a reason to distrust everything. The tools split. On the same question, on the same day, one attaches the answer to the right page and another attaches it to the wrong one, and both arrive with the same even confidence. The good citation and the bad one are dressed in the same clothes. Nothing in the wording tells you which is which, and there’s no warning colour on the wrong one. You only find out by opening the page, which is the entire point.

Worth knowing too: the tiers behave differently, and not in the direction you’d guess. On that six-question test two assistants never slipped, and the only one of them on a free tier held on all six. If you’re choosing which tool to lean on for this kind of everyday research, I keep a running audit of the free AI tools worth using rather than trusting one blanket recommendation, precisely because “paid” and “reliable on sources” aren’t the same thing.

One honest limit before you go. Everything above comes from dated snapshots, two tests, a handful of questions each, re-run once. It tells you these failures are real and what they look like. It is not a reliability rate, and I’d be wary of anyone who hands you one off a few runs. A single confident wrong citation is enough to matter anyway, because you weren’t going to check the one that looked fine. That’s the trap the whole check exists to close. If you want the running record of the ones I’ve caught, they’re in the log of AI mistakes, and the rest of the paste-in checks live on the guardrails hub. The one in this post is the cheapest of the lot: open the page it sent you to, and read it.