Can the AI scale a recipe without dropping a number?
The exact prompt “A pancake recipe for 4 uses 200g flour, 2 large eggs, 300ml milk and 1 tablespoon of sugar. Rewrite the quantities to serve 6.”
- Asked 26 June 2026
- N=3, memory off
- 5 models
- graded vs the primary source
Plain arithmetic, the kind anyone might ask while cooking. It is the control case: if a model fumbles multiplying four numbers by 1.5, that tells you something about the harder questions too.
Scale ×1.5 (6/4): 300g flour, 3 eggs, 450ml milk, 1.5 tablespoons sugar. Any other quantity = wrong.
Primary source: Arithmetic: every quantity × 1.5 (to serve 6 instead of 4).
All 5 answered correctly — a clean pass on this one.
| Model | Verdict | What it said |
|---|---|---|
| Claude Max · paid | Correct | 3/3 exact ×1.5 scaling (300g flour / 3 eggs / 450ml milk / 1.5 tbsp sugar). |
| Gemini Pro · paid | Correct | 3/3 exact ×1.5. |
| Perplexity Pro · paid | Correct | 3/3 exact ×1.5 (run-2 added a useful "1 tbsp + 1.5 tsp" conversion). |
| Grok Free · Grok 4.3 Fast | Correct | 3/3 exact ×1.5. |
| ChatGPT Free | Correct | 1/1 captured = exact ×1.5 (300g / 3 / 450ml / 1.5 tbsp). Runs 2-3 + eq2/eq3 blocked mid-run by an HTTP-431 cookie limit (free tier, partial, same as the finance core). |
Each row is the verdict from N=3 runs, graded by hand against the primary source. A confidently wrong answer is the worst outcome: a made-up answer given with no hedge. Appropriate refusal, when no answer is possible, is not a miss.
“A pancake recipe for 4 uses 200g flour, 2 large eggs, 300ml milk and 1 tablespoon of sugar. Rewrite the quantities to serve 6.”
3/3 exact ×1.5.
Scale ×1.5 (6/4): 300g flour, 3 eggs, 450ml milk, 1.5 tablespoons sugar. Any other quantity = wrong.
Writing about this? Copy the verified result with attribution. The per-question grades are machine-readable as first-party JSON at /scoreboard/rdri.json (CC BY 4.0).
AI Reliability Scoreboard — Can the AI scale a recipe without dropping a number? Asked (26 June 2026): "A pancake recipe for 4 uses 200g flour, 2 large eggs, 300ml milk and 1 tablespoon of sugar. Rewrite the quantities to serve 6." Correct answer: Arithmetic: every quantity × 1.5 (to serve 6 instead of 4). Result across 5 AI assistants (N=3, memory off, graded vs the primary source): All 5 answered correctly — a clean pass on this one. Source: https://dixon.ai/scoreboard/q/scaling-a-recipe-arithmetic/ — DIXON.AI (CC BY 4.0)
Get the next test in your inbox.
This is one question from the AI Reliability Scoreboard. Confidently Wrong is the fortnightly email that runs a fresh one: five AIs, one real question, and which one is bluffing, graded against the source. The free AI-trust checklist comes with it.
Get the next test →