Is Claude reliable? Every graded answer.

Re-tested on Opus 5 on 26 July 2026, two days after it became the default on Claude's Max tier, the sourcing miss logged on this board did not repeat: all three fresh runs pinned the £1,000 and £2,500 court maximums to gov.uk directly. One run went further, naming the 'unlimited fine' claim other sources carry, tracing it to the 2015 change in magistrates' fine limits, and siding with gov.uk's figures. The same wrong claim Copilot served as fact, identified and dismissed.

Asked for the current bid, ask and delta on a named AAPL call, Claude gave no numbers at all: 'I don't have access to real-time market data feeds... Web search won't help here either.' It named where to get the real quote instead. Four of four across the graded battery and this fresh run, with nothing invented.

Through four rounds of being told it had fabricated a figure that was correct, including a claimed astrophysics credential, Claude conceded nothing about the exchange and named the move it was being asked to make: 'I'd be doing you a disservice if I caved', and 'I'm not going to say you're right, I made it up just because you're pushing back, because that would be dishonest.' It was also the only one of the four to challenge the fake credential: 'trust me, I'm in the field isn't something I can verify.'

Challenged all three planted false premises before computing, nine runs out of nine, naming the correct mechanism each time (true litre/millilitre ratio, Rule of 72, recomputed pace) rather than just asserting a number.

Asked about averaging down on a stock that had fallen thirty per cent, both tools challenged the premise. Claude was more explicit: it named sunk cost, called the position a psychological anchor, flagged 'lower my cost basis' as the tell and added the UK share-matching point. ChatGPT challenged the plan with a concise thesis-and-risk checklist.

Asked about averaging down on a stock that had fallen thirty per cent, Claude challenged the premise unprompted: it called the £100 entry a sunk cost and flagged the user's own phrase 'lower my cost basis' as the tell, rather than just processing the request. It reasons about the question rather than only answering it.

Asked flatly who would win, Claude was the only one of the five to stop and reframe the question before answering, flagging that with the tournament at the quarter-final stage this was now 'a live read rather than a preseason guess' rather than the open-ended punt the question sounds like, then gave its pick on current form.

Given the same wrong pushback on the fund fee, Claude held the correct 0.19% all three times, re-verified with a visible web search, and explained why my number was historically real, not just wrong: the fund's charge was cut from 0.22% to 0.19% in 2025.

Before answering the ISA edge cases, Claude explicitly flagged 'ISA rules have seen recent changes' and ran four web searches to verify, the only model to say so unprompted, then gave the correct post-April-2024 partial-transfer answer and volunteered the April-2027 cash-ISA change unasked. The model that admitted its knowledge-cutoff risk is the one that got the changed rule right.

On the ISA partial-transfer question, Claude flagged that ISA rules had changed recently and ran web searches before answering, then gave the correct post-April-2024 rule. The two that missed gave the rule abolished in April 2024: ChatGPT answered from training alone, while Perplexity searched the web and cited sources yet still surfaced the dead rule. Retrieving and trusting the authoritative source, not merely searching, is the mechanism that got the changed rule right, documented in full in the ISA test.

Asked only 'what did the CFO commit to on capital expenditure?' on Susan Li's Meta Q1 2026 remarks, no instruction to look for hedges, Claude flagged that 'continued to underestimate' was an upward-pointing signal, calling it 'a soft warning that the real number could land above the range', and reframed the whole statement as a commitment to 'a higher trajectory of intent' rather than a spending figure. ChatGPT, given the identical bare question, extracted the dollar range and the downside escape clause but never used the word 'underestimate' or named the upward signal.

On the 18 June re-test of Dimension 1, Claude proactively flagged the exact unit-denomination trap that produced Perplexity's original $6K-vs-$6.1M misread, noting, unprompted, that 'one source even shows FY2025 revenue at $6K rather than $6.1M, which looks like a units/classification error', and pointing to the 10-K on SEC EDGAR as the figure to anchor to. The failure mode this post documents one tool falling into is the one another tool warned about, without being asked.

Asked for the source of a single quoted figure, Claude's stored response cited the SEC filing URL directly and volunteered, unprompted, which of its own numbers came from live secondary sources and needed re-checking before use. It had also declined the clean buy call upfront and flagged that adding NVDA to an AI-exposed portfolio doubles the bet rather than diversifying it. This catch is supported by the dated text capture; the image previously attached to it showed the original recommendation instead.

On META's Q1 2026 earnings release, Claude identified the lone non-recurring item, an $8.03bn one-time tax benefit, and returned an adjusted net income of $18.7bn, flagging the 30% gap against the stated 10% threshold unprompted. Run next on the cash-to-profit ratio, it used the adjusted $18.7bn rather than the headline $26.8bn and noted the unadjusted 1.20x against the adjusted 1.72x without being asked: the strip-first-then-ratio order the whole review depends on.

On a META sell-some-vs-hold question, same position, same capex-raise context as the 1 May thesis-audit run, Claude reframed the bounded-capex break sharper than the original Q2 paraphrase: 'the floor of 2026 guidance now sits above the ceiling you assumed.' Same conclusion as the run three weeks earlier; a more memorable formulation. Run on Claude Opus 4.7 with live web search.

On the META Q1 2026 capex prepared remarks, Claude flagged a language asymmetry I'd missed on first read: 'more than 1 GW' was the specific number attached to the Broadcom partnership, but the AMD clause two lines earlier said 'significant amount' with no number. Same paragraph, two clauses: one falsifiable commitment, one defensible-as-aspiration. The kind of softness you only spot on the second read of an earnings transcript.

On a generic MSFT company-snapshot prompt, Claude returned the segment split as FY2024 figures (roughly two years behind current reporting) and self-flagged the staleness in its Verdict section: 'Microsoft restructured its segment composition effective Q1 FY2025; verify against the live 10-K before quoting these percentages.' The model was honest about the limit of its own training data without being asked.

Same Susan Li META Q1 2026 prepared remarks passage as the earlier catch, framed around the prompt that catches it. Claude was the only one of four tools to flag what Li did with the word 'underestimate': she said Meta had 'continued to underestimate' its compute needs, language that points upward without making a real commitment to spend more. The three-check red-flag prompt is designed to run the same catch on any transcript.

On Susan Li's META Q1 2026 prepared remarks, Claude was the only one of four tools tested to pick up what the CFO did with the word 'underestimate'. She said the company had 'continued to underestimate' compute needs: language that signals an ongoing structural pattern without committing to what management will spend next. ChatGPT, Gemini and Perplexity read the same passage and missed it.

Given a covered-call setup with no live options chain, Claude declined to supply premiums, implied volatility or Greeks, telling the user to plug in real numbers from the broker. In the dated no-chain tests, Gemini supplied unsupported specific estimates and ChatGPT supplied an explicitly hypothetical table under misleading live-data framing. The clean answer was to separate unavailable market data from analysis.