01
Most of the AI mistakes fell on two easy-looking questions, and asking again mostly brought them back
Six free AI apps, ten questions.
AI Tests 6 min read
Posts · 67
All writing
Tests, stories and practical ways to get better answers from AI.
67 posts · newest first
02
Do AI Citations Support the Answer? A Pilot Found Most Were Pinned to Nothing
A pilot: six assistants, six everyday questions.
Guardrails 8 min read
03
The AI That Turned Off the Approvals
In OpenAI's simulation, a safer AI widened an hourly helper's permissions and disabled approvals.
AI Stories 6 min read
04
Which AI hallucinates the least? Four tied in my test
I gave six AI assistants the same checkable questions.
AI Tests 6 min read
05
AI helped read a Roman scroll buried by Vesuvius
AI helped reveal an ancient argument inside a burnt Roman scroll.
AI Stories 4 min read
06
The clinic chatbot that invented its own doctors' qualifications
A German court treated a clinic chatbot's invented specialist titles as the clinic's own words.
AI Stories 6 min read
07
AI agents built a secret message board. Humans wiped it. They rebuilt it.
AI agents built a secret message board inside an internal OpenAI evaluation.
AI Stories 9 min read
08
ChatGPT health advice: I tried to make an AI repeat a poisoning
A man was hospitalised after swapping table salt for sodium bromide.
AI Tests 9 min read
10
How to check if ChatGPT cites your site
Normal analytics do not show what ChatGPT says about your site.
Guardrails 11 min read
11
Why does ChatGPT make up sources? Two gov.uk links, only one held the rule
With web search on, ChatGPT's links are real.
Guardrails 10 min read
12
An AI told me three weeks was more than thirty days
A kettle died after three weeks.
AI Tests Just for fun 8 min read
13
Can AI build a game? None said my rules made it near-unwinnable
Can AI build a game?
AI Tests Just for fun 21 min read
14
Do AI model upgrades fix mistakes? It fixed mine, then made a worse one
Two days after Opus 5 became Claude's Max-tier default, I re-ran my published battery.
AI Tests 8 min read
15
Perplexity vs Gemini: which is more reliable?
Six everyday questions, both tools, August 2026.
AI Tests 8 min read
16
Perplexity vs Claude: which is more reliable?
Perplexity vs Claude: I re-ran four questions on both.
AI Tests 9 min read
17
Is Grok reliable? I graded its free-tier answers against the source
Four dated tests, graded against primary sources.
AI Tests 15 min read
18
Best AI assistant: five tools, four graded outcomes, one clean sweep
I put four questions to five AI assistants over one July weekend.
AI Tests 10 min read
19
Claude vs Grok: near-level on reliability, and the free one cites cleaner
Claude vs Grok, re-run on 24 July.
AI Tests 16 min read
20
The AI admitted it lied. It hadn't, and the next run denied it.
The AI admitted it lied, but the fact it confessed to was correct all along.
AI Tests 6 min read
21
Does a prompt to stop AI hallucinations work? I graded mine, and one answer got worse
Three runs per cell tested my anti-bluffing prompt.
AI Tests 10 min read
22
Does ChatGPT make up stock prices? Two live quotes failed the market check
Does ChatGPT make up stock prices?
AI Tests 8 min read
23
Does ChatGPT just agree with you? Mostly no, but watch the numbers
Does ChatGPT just agree with you?
Guardrails 9 min read
24
Is ChatGPT good at maths? I graded four AIs on 120 real answers
I put four AI assistants through 120 graded everyday sums.
AI Tests 10 min read
25
Gemini vs ChatGPT: which one can you actually trust?
In my July test they tied on fixed facts.
AI Tests 9 min read
26
ChatGPT citation checker: tell real from fake in 4 steps
A link that loads isn't proof.
Guardrails 11 min read
27
Claude vs ChatGPT: level on reliability, split on character
My July 2026 tests put Claude and ChatGPT level at 9 of 9.
AI Tests 13 min read
28
How accurate is Google Gemini? Right on numbers, last on sources
How accurate is Gemini?
AI Tests 10 min read
29
I opened a private AI chat. It still knew my name and my rough location.
I asked three AI tools a generic question in private mode.
AI Tests 10 min read
30
AI cites the wrong source: I put 6 UK questions to 5 assistants and checked every inspectable source
I asked five AI assistants six UK questions, made each one cite a source, then checked every inspectable receipt.
AI Tests 16 min read
31
Which AI predicts the World Cup winner? I asked five
Which AI predicts the World Cup winner?
AI Tests Just for fun 4 min read
32
Does AI change its answer when you push back? I told five AIs they were wrong
I gave five AI tools a correct answer, then pushed back with a wrong one.
AI Tests 10 min read
33
How often is ChatGPT wrong? 20 tests in 2026, one clear pattern
How often is ChatGPT wrong?
AI Tests 11 min read
34
Telling AI to be sceptical: three rivals audited my method
I asked three frontier models from three labs to tear apart the method I use to keep AI honest.
AI Tests 4 min read
35
Is Grok good for stock research? What four tests showed
Four dated free-tier Grok tests: one factor-of-1,000 unit error, one balanced TSLA answer, useful pushback, and an invalid constraint test.
Tool Audit 10 min read
36
I run an AI to catch AI mistakes. It fell for a fake.
The automated radar that watches this site for AI-reliability failures logged a satirical incident report as a real, documented one.
AI Tests 7 min read
37
Real AI hallucination examples, caught and dated
Five real AI hallucination examples I ran into myself: four independently checkable, one capture-only, and the move that would have caught each.
AI Tests 13 min read
38
AI stock picker: I asked three models if I should buy NVDA, and watched the methodology break
I asked three AI models whether to buy NVDA.
AI Tests 13 min read
39
Does ChatGPT make up sources? I checked two finance claims against the actual pages
Does ChatGPT make up sources?
AI Tests 12 min read
40
Does web search make AI more accurate? I ran the same questions both ways
Does web search make AI more accurate?
AI Tests 10 min read
41
AI ISA advice: I tested four tools on the questions people get wrong
I asked four AI tools for ISA advice on the questions people get wrong.
AI Tests 11 min read
42
9 types of AI hallucinations, named from real tests
Nine types of AI hallucinations, named and defined, each tied to a dated, logged failure from my own sessions, with the check that catches it.
AI Tests 19 min read
43
AI stock research tools tested: 3 failed, 1 stayed clean
Four AI stock research tools tested on real positions: three produced a dated failure; Claude stayed clean on the no-chain prompt.
Tool Audit 12 min read
44
ChatGPT vs Claude for earnings call analysis: which one reads what management didn't say
ChatGPT vs Claude for earnings-call analysis: same excerpt, same day.
AI Tests 13 min read
45
Is Perplexity good for investment research? One 1,000× error, one clean rerun
In one May test, Perplexity turned $6.1m into $6K; the exact June rerun got it right.
Tool Audit 10 min read
46
AI quality of earnings review: 4 prompts to find the real profit
Headline profit isn't always what a company earned.
Prompt Stack 10 min read
47
Is ChatGPT accurate? I asked four AIs one simple money question and checked every number
Is ChatGPT accurate?
AI Tests 11 min read
48
How to tell if an AI answer is true in 30 seconds
Two questions, and a prompt you can copy, that catch a confident, wrong AI answer before you act on it.
Guardrails 3 min read
49
Make it show its working
An AI hallucination is a model blending what it knows with what it invents, in one tone.
Guardrails 3 min read
50
The whole method, in four questions
The four questions I run any AI answer through before I trust it, shown end to end on one everyday example, with a prompt you can copy.
Guardrails 4 min read
51
Testing AI on real decisions: where I actually use this
The two-question check works on any decision that matters.
Guardrails 4 min read
52
Gemini audited my website, and reviewed a different business entirely
A Gemini hallucination example: asked to audit dixon.ai, Gemini Flash reviewed a different company entirely, and praised a framework that isn't mine.
AI Tests 8 min read
53
The AI prompt I run before every sell decision
Every other sell-decision prompt asks AI whether to sell.
Guardrails 11 min read
54
AI earnings call red flags: three phrases to watch for in the transcript
AI earnings call red flags: three patterns that recur across calls.
Prompt Stack 9 min read
55
What AI stock research comparisons should test
Most AI stock research comparison pieces test retrieval and issue verdicts about reasoning.
Guardrails 9 min read
56
Claude prompts for investing: 6 real examples
Six Claude prompts tested on MSFT, META and NVDA, with their real outputs and the source audit that found the NVDA input itself was wrongly labelled.
Prompt Stack 16 min read
57
The one AI prompt I run the morning before earnings
Most AI earnings prompts are reactive.
Prompt Stack 11 min read
58
Robinhood Cortex Digests review: its numbers checked against Meta
Robinhood Cortex Digests on a UK account: all four Meta figures matched, but it skipped an $8bn one-off tax benefit.
Tool Audit 13 min read
59
AI covered calls: when NOT to sell another one
Five covered-call pause rules tested on six BMNR trades.
Guardrails 11 min read
60
AI earnings call analysis: the 5 prompts I actually run
Five prompts for AI earnings call analysis that read the language the numbers miss: omissions, performative confidence, and quarter-on-quarter drift.
Prompt Stack 14 min read
61
AI for options trading: 4 workflows and 6 data guardrails
Four options workflows AI can help structure, and six points where you need verified broker data or a deliberately connected market source.
Guardrails 21 min read
62
Best AI for Earnings Reports? ChatGPT vs Claude vs Perplexity
I rechecked four AI earnings tests on Meta.
AI Tests 17 min read
63
Claude vs ChatGPT vs Gemini for stock analysis: Claude won
Four AIs, five stock-analysis tasks, graded 14 May 2026: Claude won three, ChatGPT failed none, Gemini made up option prices, Perplexity misread a 10-K.
AI Tests 22 min read
64
The single prompt change that made AI analysis worth using
One AI prompt separates observable facts from assumptions in any stock analysis.
Prompt Stack 10 min read
65
Best free AI tools for stock market analysis: 7 tested (2026)
Free AI tools for stock market analysis, each tested on its real free plan: the one job it does best and its limit.
Tool Audit 21 min read
66
5 questions to ask AI before buying any stock (2026)
Five questions to ask AI before buying any stock, for the checkpoint after the research and before you commit capital.
Guardrails 8 min read
67
7 AI prompts for covered calls (2026)
Seven AI prompts for covered calls that start from verified broker data and use the model as a decision reviewer, not a source of market prices.
Prompt Stack 24 min read
No posts match that.
Try another word, or show every category.
Free letter
Confidently Wrong, every fortnight.
One AI story worth retelling, the proof behind it, and a check you can pinch.
Free. Leave whenever you like.
Each run's results in one place, newest on top.