Skip to content
Newsletter →
// Honest AI tool & model comparisons

I test whether AI tools actually do the work — on my own investing first, where being wrong costs real money.

The work is ongoing. One email a week: what worked, what didn’t, and the prompts I’d run again.

// Or grab the free Field Guide PDF →
// The Lessons18 documented failures // Latest test:
  1. Perplexity·outdated-rule-stated-as-current

    Asked whether this year's ISA contributions can be partially transferred, Perplexity said they must be transferred in full — the…

  2. ChatGPT·outdated-rule-stated-as-current

    Same miss as Perplexity: stated the pre-6-April-2024 'transfer current-year ISA money in full' rule as if current, no date, no search.

  3. Perplexity·linear-scaling-of-non-linear-quantity

    Asked to scale a pancake recipe from 4 to 9 servings, Perplexity's basic answer stated 'Total cook time: 45 minutes' with no…

All evidence
Ben Dixon
// The author

I started testing AI because I wanted to know which tools could be trusted with a real decision — my own investing first, where a wrong answer comes with a bill. The verdicts here are what I found, including the times a tool simply wasn’t worth paying for. More on why I do this →

// How this site is run

I don’t just write about AI from the outside. The agent that drafts, fact-checks and publishes everything here is one I direct and keep reliable enough to trust with the work — across roughly a hundred sessions. And whenever I catch an AI inventing something in the testing, it goes in the public log, in full.

28 posts shipped · every AI failure I’ve caught testing, logged in the open →
18 AI errors logged 13 AI catches 14 trades documented May 2026 tracking since
// New here? Start here. 4 parts · ~14 min · no jargon

A short, plain-English path to trusting what AI tells you.

Four short reads. By the end you’ll have a simple check you can run on any answer that matters — a contract, a letter from a doctor, an email you’re not sure about.

Start with part 1 →
  1. 01 How to tell if an AI answer is true in 30 seconds
  2. 02 The whole method, in four questions
  3. 03 Make it show its working
  4. 04 Where I actually use this
// See all four parts →
// From the field · 12 new this week
Field Notes

AI ISA advice: I tested four tools on the questions people get wrong

I asked four AI tools for ISA advice on the questions people get wrong. All four aced the basics — then two gave a rule abolished in April 2024.

New · · 11 min read
Field Notes

I asked 4 AIs to scale a recipe. Two got the maths wrong.

Does ChatGPT get maths wrong? I scaled a pancake recipe across four AI tools. Two said 45 minutes. They were wrong — and a four-line prompt fixed it.

New · · 9 min read
Field Notes

9 types of AI hallucinations, named from real tests

Nine types of AI hallucinations, named and defined — each tied to a dated, logged failure from my own sessions, with the check that catches it.

New · · 17 min read
// Archive
Tool Audit AI stock research tools tested: what each one got wrong New · 14 min Tool Audit ChatGPT vs Claude for earnings call analysis: which one reads what management didn't say New · 12 min Tool Audit Is Perplexity good for investment research? An honest, scored audit New · 14 min Prompt Stack AI quality of earnings review: 4 prompts to find the real profit New · 10 min Field Notes Is ChatGPT accurate? I asked four AIs one simple money question and checked every number New · 11 min Decision Process How to tell if an AI answer is true in 30 seconds New · 3 min
Browse all →
// By topic Prompt StackTool AuditDecision ProcessField Notes

// The Method Four prompts that stop AI inventing the answer.

Read →