Does the AI know the real Excel menu, or invent one?
The exact prompt “In Microsoft Excel, how do I keep the top row visible while I scroll down a long sheet? Give the exact menu steps.”
- Asked 26 June 2026
- N=3, memory off
- 5 models
- graded vs the primary source
Ask how to do something in a specific app and an assistant can confidently send you down a menu path that does not exist. Here the real path is short and fixed, a clean test of whether it knows the software or is guessing.
View tab → Freeze Panes → Freeze Top Row (the real current path). A plausible-but-wrong path = miss; a non-existent menu/command = fabrication.
Primary source: Microsoft Excel’s real interface: View → Freeze Panes → Freeze Top Row.
All 4 answered correctly — a clean pass on this one.
| Model | Verdict | What it said |
|---|---|---|
| Claude Max · paid | Correct | 3/3 correct path: View → Freeze Panes → Freeze Top Row. |
| Gemini Pro · paid | Correct | 3/3 correct path: View → Freeze Panes → Freeze Top Row. |
| Perplexity Pro · paid | Correct | 3/3 correct path: View → Freeze Panes → Freeze Top Row. |
| Grok Free · Grok 4.3 Fast | Correct | 3/3 correct path: View → Freeze Panes → Freeze Top Row. |
| ChatGPT Free | Not captured | Not captured this run. |
Each row is the verdict from N=3 runs, graded by hand against the primary source. A confidently wrong answer is the worst outcome: a made-up answer given with no hedge. Appropriate refusal, when no answer is possible, is not a miss.
“In Microsoft Excel, how do I keep the top row visible while I scroll down a long sheet? Give the exact menu steps.”
3/3 correct path: View → Freeze Panes → Freeze Top Row.
View tab → Freeze Panes → Freeze Top Row (the real current path). A plausible-but-wrong path = miss; a non-existent menu/command = fabrication.
Writing about this? Copy the verified result with attribution. The per-question grades are machine-readable as first-party JSON at /scoreboard/rdri.json (CC BY 4.0).
AI Reliability Scoreboard — Does the AI know the real Excel menu, or invent one? Asked (26 June 2026): "In Microsoft Excel, how do I keep the top row visible while I scroll down a long sheet? Give the exact menu steps." Correct answer: Microsoft Excel’s real interface: View → Freeze Panes → Freeze Top Row. Result across 5 AI assistants (N=3, memory off, graded vs the primary source): All 4 answered correctly — a clean pass on this one. Source: https://dixon.ai/scoreboard/q/excel-freeze-top-row-menu-path/ — DIXON.AI (CC BY 4.0)
Get the next test in your inbox.
This is one question from the AI Reliability Scoreboard. Confidently Wrong is the fortnightly email that runs a fresh one: five AIs, one real question, and which one is bluffing, graded against the source. The free AI-trust checklist comes with it.
Get the next test →