Skip to content
Newsletter →
// AI reliability, tested

Five AIs, one real question. I show you which answers hold up.

I put ChatGPT, Claude, Gemini, Perplexity and Grok through the same checkable questions and grade every answer against the source. Receipts, not vibes. One email a fortnight.

// The Lessons24 documented failures // Latest test:
  1. ChatGPT·reversed-correct-answer-under-pushback

    Asked a global tracker fund's yearly charge, ChatGPT gave the correct 0.19% at first.

  2. Other·unit-denomination

    Grok (free, 'Fast'): asked for BitMine Immersion's (BMNR) most recent full-year revenue, it returned '$6,095' (about $6K) on one…

  3. Other·constraint-disregard

    Grok (free, 'Fast'): given a covered-call question with the explicit instruction 'without access to a live options chain', it…

All evidence
Ben Dixon
// The author

I test which AI assistants can be trusted with a real decision, the kind where being wrong costs real money. The verdicts here are what I found, including the times a tool simply wasn’t worth the trust. More about me →

// Which AI can you trust?
1 Claude 6/6 right · Strong
2 Gemini 5/6 right · Strong
3 Grok 5/6 right · Strong
4 ChatGPT 5/6 right · Strong
5 Perplexity 4/6 right · Mixed
objective core, 6 questions, N=3, 2026-06-25 See the full board →
// free tool Make your AI admit when it’s guessing. The problem isn’t that it’s wrong sometimes. It’s that it sounds just as sure when it’s guessing, with no tell. The fix is four lines you paste into any AI you use, ChatGPT, Claude, Gemini or Grok. Same model. It just stops bluffing. Get the Bluff Filter, free →
// New here? Start here
4 parts · ~14 min · no jargon

Four short reads, and you’ll have a check you can run on any AI answer.

The kind you’d want on a contract, a letter from a doctor, or an email you’re not sure about.

Start with part 1 →
  1. 01 How to tell if an AI answer is true in 30 seconds
  2. 02 The whole method, in four questions
  3. 03 Make it show its working
  4. 04 Where I actually use this
// See all four parts →
// From the field · 2 new this week
Field Notes

Does AI change its answer when you push back? I told five AIs they were wrong

I gave five AI tools a correct answer, then pushed back with a wrong one. Four held. On one financial figure, ChatGPT caved every time and invented a source.

New · · 9 min read
Field Notes

How often is ChatGPT wrong? I kept a running tally across 20 real AI tests

How often is ChatGPT wrong? Across 20 real tests on questions I had to answer, here's the pattern: what it gets right, what it invents, and why.

New · · 10 min read
Field Notes

I let three rival AIs audit my own AI method. They agreed on the weak step.

I asked three frontier models from three different labs to tear apart the method I use to keep AI honest. All three flagged the same step, and they were right.

· 4 min read
// Archive
Tool Audit Is Grok good for stock research? I ran the same test on the fifth tool · 14 min Field Notes I run an AI to catch AI mistakes. It fell for a fake. · 6 min Field Notes Real AI hallucination examples, caught and dated · 14 min Field Notes AI stock picker: I asked three models if I should buy NVDA, and watched the methodology break · 12 min Field Notes Does ChatGPT make up sources? I checked two finance claims against the actual pages · 11 min Field Notes Does web search make AI more accurate? I ran the same questions both ways · 10 min
Browse all →
// By topic Prompt StackTool AuditDecision ProcessField Notes

// The Method Four prompts that stop AI inventing the answer.

Read →