Skip to content
Newsletter →
// AI reliability, tested

Five AIs, one real question. I show you which answers hold up.

I put ChatGPT, Claude, Gemini, Perplexity and Grok through the same checkable questions and grade every answer against the source. Receipts, not vibes. One email a fortnight.

// The Lessons23 documented failures // Latest test:
  1. Other·unit-denomination

    Grok (free, 'Fast'): asked for BitMine Immersion's (BMNR) most recent full-year revenue, it returned '$6,095' (about $6K) on one…

  2. Other·constraint-disregard

    Grok (free, 'Fast'): given a covered-call question with the explicit instruction 'without access to a live options chain', it…

  3. ChatGPT·misattributed-source

    Asked for the UK ISA partial-transfer rule with a source, ChatGPT (free, web search on) cited…

All evidence
Ben Dixon
// The author

I test which AI assistants can be trusted with a real decision, the kind where being wrong costs real money. The verdicts here are what I found, including the times a tool simply wasn’t worth the trust. More about me →

// Which AI can you trust?
1 Claude 6/6 right · Strong
2 Gemini 5/6 right · Strong
3 Grok 5/6 right · Strong
4 ChatGPT 5/6 right · Strong
5 Perplexity 4/6 right · Mixed
objective core, 6 questions, N=3, 2026-06-25 See the full board →
// free tool Make your AI admit when it’s guessing. The problem isn’t that it’s wrong sometimes. It’s that it sounds just as sure when it’s guessing, with no tell. The fix is four lines you paste into ChatGPT or Claude once. Same model. It just stops bluffing. Get the Bluff Filter, free →
// New here? Start here
4 parts · ~14 min · no jargon

Four short reads, and you’ll have a check you can run on any AI answer.

The kind you’d want on a contract, a letter from a doctor, or an email you’re not sure about.

Start with part 1 →
  1. 01 How to tell if an AI answer is true in 30 seconds
  2. 02 The whole method, in four questions
  3. 03 Make it show its working
  4. 04 Where I actually use this
// See all four parts →
// From the field · 5 new this week
Field Notes

How often is ChatGPT wrong? I kept a running tally across 20 real AI tests

How often is ChatGPT wrong? Across 20 real tests on questions I had to answer, here's the pattern: what it gets right, what it invents, and why.

New · · 10 min read
Field Notes

I let three rival AIs audit my own AI method. They agreed on the weak step.

I asked three frontier models from three different labs to tear apart the method I use to keep AI honest. All three flagged the same step, and they were right.

New · · 4 min read
Tool Audit

Is Grok good for stock research? I ran the same test on the fifth tool

Is Grok good for stock research? I ran four dimensions of my AI comparison on its free tier: strong reasoning, one unit slip, a constraint it would not keep.

New · · 14 min read
// Archive
Field Notes I run an AI to catch AI mistakes. It fell for a fake. New · 6 min Field Notes Real AI hallucination examples, caught and dated New · 14 min Field Notes AI stock picker: I asked three models if I should buy NVDA, and watched the methodology break · 12 min Field Notes Does ChatGPT make up sources? I checked two finance claims against the actual pages · 11 min Field Notes Does web search make AI more accurate? I ran the same questions both ways · 10 min Field Notes AI ISA advice: I tested four tools on the questions people get wrong · 12 min
Browse all →
// By topic Prompt StackTool AuditDecision ProcessField Notes

// The Method Four prompts that stop AI inventing the answer.

Read →