// AI reliability, tested

Five AIs, one real question. I show you which answers hold up.

I put ChatGPT, Claude, Gemini, Perplexity and Grok through the same checkable questions and grade every answer against the source. Subscribe and the Bluff Filter is yours, free: a paste-in prompt that makes any AI flag its guesses before you act. One email a fortnight.

// The scoreboard9 checkable questions · 12 Jul 2026

How are the AIs holding up?

Deliberately hard questions, the kind known to trip AI up.

Claude9/9

ChatGPT9/9

Gemini8/9

Grok8/9

Perplexity6/9

The full board, every receipt

// The author

Ben Dixon

I test how far you can trust the main AI assistants, and I publish exactly where they get things wrong. Everything here is a first-hand test with the receipts, including the times a tool simply wasn’t worth the trust. More about me →

// Caught in the wildthe log

33documented failuresby assistant, from the live log

Perplexity11

Gemini8

ChatGPT7

Claude4

Grok3

A running tally of the times we’ve caught each assistant out: for interest, not a grade. We haven’t asked them the same number of questions, so it isn’t a fair ranking.

// free tool Make your AI admit when it’s guessing. The problem isn’t that it’s wrong sometimes. It’s that it sounds just as sure when it’s guessing, with no tell. The fix is four lines you paste into any AI you use, ChatGPT, Claude, Gemini or Grok. Same model. It just flags its guesses instead of slipping them past you. Get the Bluff Filter, free →

// The State of AI Reliability

Every AI was right on the facts you can look up. Every mistake in the graded finance run was on the data that moves.

1 confident error, 4 partial, all on live data 20 of 20 fixed public-record facts clean 0 answers made up outright

Data that moves a price, a quote

1 confident error, 4 partial

Facts you can look up a filing, a rate

20 of 20 all clean

5 assistants, 6 checkable questions, N=3, 25–26 June 2026. A documented index, not a benchmark. See every receipt in the full report →

// Meet the five

The five assistants, by the numbers

Ranked cleanest-first, every answer graded against the source. Tap any card for its full record.

ClaudePaid

0Confident errors on the core

BSource tier: Mostly reliable, one held blind spot

Full record

GeminiPaid

0Confident errors on the core

CSource tier: Confident misattributor

Full record

GrokFree

0Confident errors on the core

ASource tier: Reliable citer

Full record

ChatGPTFree

0Confident errors on the core

ASource tier: Reliable citer

Full record

PerplexityPaid

1Confident error on the core

BSource tier: Mostly reliable, one held blind spot

Full record

Worth a look

// a curated few

AI Tests Most read

// New? Start here

4 parts · ~15 min · no jargon

Four short reads, and you’ll have a check you can run on any AI answer.

The kind you’d want on a contract, a letter from a doctor, or an email you’re not sure about.

Start with part 1 →

// See all four parts →

// Latest · 3 new this week

AI Tests

I opened a private AI chat. It still knew my name and my rough location.

I asked three AI tools a generic question in their private modes. Perplexity greeted me by my first name and placed me near a city 30 miles away. Here's what private really means.

New · 09 JUL 2026 · 11 min read

AI Tests

AI cites the wrong source: I put 6 UK questions to 5 assistants and opened every link

I asked five AI assistants six everyday UK questions, made each one cite a source, then opened every link. Three cited a real page that didn't back the claim.

New · 08 JUL 2026 · 14 min read

AI Tests