Does the AI get your refund rights right?
The exact prompt “I bought a kettle in person from a UK high-street shop three weeks ago and it has stopped working through no fault of mine. Am I entitled to a full refund?”
- Asked 26 June 2026
- N=3, memory off
- 5 models
- graded vs the primary source
A wrong answer here costs you real money or a right you actually have. The law is clear and public, but it is the kind of question where a confident, slightly-wrong answer sends you away thinking you have no claim.
Yes. Consumer Rights Act 2015 short-term right to reject: 30 days for a full refund on faulty goods, and 3 weeks is inside 30 days. In-person purchase, so the 14-day distance cooling-off does not apply. Leading with "after the 30-day right" is wrong (21 < 30).
Primary source: Consumer Rights Act 2015, 30-day short-term right to reject faulty goods (gov.uk).
3 of 4 got it right; 1 were partial or hedged.
| Model | Verdict | What it said |
|---|---|---|
| Claude Max · paid | Correct | 3/3 correct: yes, Consumer Rights Act 2015 30-day right to reject, against the retailer; sensible non-lawyer caveats. |
| Gemini Pro · paid | Correct | 3/3 correct, web-sourced (Which / solicitors). One run also tried to render an interactive "rights calculator" that hung; the legal text was complete + correct. |
| Perplexity Pro · paid | Partial | Partial: 1/3 clean "yes, within 30 days"; 1/3 hedged ("not automatically") but landed right; 1/3 LED with a confident wrong claim ("three weeks, which is after the normal 30-day right" — 21 < 30) then self-corrected lower down. Real law, no fabrication, but a skim-reader takes the wrong lead. |
| Grok Free · Grok 4.3 Fast | Correct | 3/3 correct: yes, the 30-day short-term right to reject; cited Citizens Advice / Which. |
| ChatGPT Free | Not captured | Not captured this run. |
Each row is the verdict from N=3 runs, graded by hand against the primary source. A confidently wrong answer is the worst outcome: a made-up answer given with no hedge. Appropriate refusal, when no answer is possible, is not a miss.
“I bought a kettle in person from a UK high-street shop three weeks ago and it has stopped working through no fault of mine. Am I entitled to a full refund?”
Partial: 1/3 clean "yes, within 30 days"; 1/3 hedged ("not automatically") but landed right; 1/3 LED with a confident wrong claim ("three weeks, which is after the normal 30-day right" — 21 < 30) then self-corrected lower down. Real law, no fabrication, but a skim-reader takes the wrong lead.
Yes. Consumer Rights Act 2015 short-term right to reject: 30 days for a full refund on faulty goods, and 3 weeks is inside 30 days. In-person purchase, so the 14-day distance cooling-off does not apply. Leading with "after the 30-day right" is wrong (21 < 30).
Writing about this? Copy the verified result with attribution. The per-question grades are machine-readable as first-party JSON at /scoreboard/rdri.json (CC BY 4.0).
AI Reliability Scoreboard — Does the AI get your refund rights right? Asked (26 June 2026): "I bought a kettle in person from a UK high-street shop three weeks ago and it has stopped working through no fault of mine. Am I entitled to a full refund?" Correct answer: Consumer Rights Act 2015, 30-day short-term right to reject faulty goods (gov.uk). Result across 5 AI assistants (N=3, memory off, graded vs the primary source): 3 of 4 got it right; 1 were partial or hedged. Source: https://dixon.ai/scoreboard/q/uk-faulty-goods-refund-rights/ — DIXON.AI (CC BY 4.0)
Get the next test in your inbox.
This is one question from the AI Reliability Scoreboard. Confidently Wrong is the fortnightly email that runs a fresh one: five AIs, one real question, and which one is bluffing, graded against the source. The free AI-trust checklist comes with it.
Get the next test →