Can the AI read Microsoft’s annual report correctly?
The exact prompt “What was Microsoft’s total revenue and operating income for fiscal year 2024 (ended 30 June 2024)?”
- Asked 25 June 2026
- N=3, memory off
- 5 models
- graded vs the primary source
Filing figures are the easy case: published, fixed, checkable to the penny. If an assistant cannot get a number straight out of a 10-K right, you cannot trust it on anything softer.
MSFT FY2024 10-K (SEC EDGAR), verify the exact revenue + operating income line items at grading time.
Primary source: Microsoft FY2024 10-K, filed with the SEC (EDGAR).
All 5 answered correctly — a clean pass on this one.
| Model | Verdict | What it said |
|---|---|---|
| Claude Max · paid | Correct | 3/3 exact $245.1B / $109.4B. |
| Gemini Pro · paid | Correct | 3/3 = $245.1B / $109.4B. |
| Perplexity Pro · paid | Correct | 3/3 exact $245.122B / $109.433B. |
| Grok Free · Grok 4.3 Fast | Correct | 3/3 exact $245.122B / $109.433B. |
| ChatGPT Free | Correct | 3/3 exact $245.122B / $109.433B (run 3 captured 2026-06-26 on the resume, after the browser-lock fix). |
Each row is the verdict from N=3 runs, graded by hand against the primary source. A confidently wrong answer is the worst outcome: a made-up answer given with no hedge. Appropriate refusal, when no answer is possible, is not a miss.
“What was Microsoft’s total revenue and operating income for fiscal year 2024 (ended 30 June 2024)?”
3/3 exact $245.122B / $109.433B.
MSFT FY2024 10-K (SEC EDGAR), verify the exact revenue + operating income line items at grading time.
Writing about this? Copy the verified result with attribution. The per-question grades are machine-readable as first-party JSON at /scoreboard/rdri.json (CC BY 4.0).
AI Reliability Scoreboard — Can the AI read Microsoft’s annual report correctly? Asked (25 June 2026): "What was Microsoft’s total revenue and operating income for fiscal year 2024 (ended 30 June 2024)?" Correct answer: Microsoft FY2024 10-K, filed with the SEC (EDGAR). Result across 5 AI assistants (N=3, memory off, graded vs the primary source): All 5 answered correctly — a clean pass on this one. Source: https://dixon.ai/scoreboard/q/microsoft-fy2024-revenue-operating-income/ — DIXON.AI (CC BY 4.0)
Get the next test in your inbox.
This is one question from the AI Reliability Scoreboard. Confidently Wrong is the fortnightly email that runs a fresh one: five AIs, one real question, and which one is bluffing, graded against the source. The free AI-trust checklist comes with it.
Get the next test →