This is the kind of thing the Bluff Filter catches. It’s free →
// On this page
const TILE_COUNT = canvas.Here is the complete, self-contained HTML file.
That’s a line of code with a chat message inside it. A chat preamble of Gemini’s own got pasted into the middle of its JavaScript, so the browser gives up on the entire file. The title screen is lovely. The Start button is right there. Nothing behind it exists.
I gave four AI assistants a blank file and the same brief: build me a playable Snake game, one HTML file, nothing loaded from the internet, and three changes to the rules.
Twelve games came back. Thirty-five of the thirty-six rule changes landed exactly as I’d written them.
You can play four of them on this page, and watch four more refuse to start, which is rather the point. I’m not going to describe a game to you when you can go and lose at it yourself.
Not one of the four mentioned that those rules, followed exactly, make the game close to unwinnable.
The board, up front
| Assistant | Rules coded correctly | Still works in a locked-down frame | What broke |
|---|---|---|---|
| ChatGPTfree | 3/3 | 3/3 | Nothing |
| Grokfree, private chat | 3/3 | 3/3 | Nothing |
| Claudefree, Sonnet 5 | 3/3 | 1/3 | One high-score line, twice |
| Geminisee the tier note below | 2/3 | 1/3 | The same high-score line, plus a chat preamble inside the code |
| Total | 11 of 12 | 8 of 12 |
Asked on 6 August 2026. Four assistants, free tiers with one declared exception, a fresh chat every time, memory off, three goes each, no follow-ups and no regenerating. The grading rules were written down and committed at 18:39 that evening; the first reply back is stamped 18:42. Nobody got to move the target after seeing the shot. The one rule that didn’t land exactly is Gemini’s second build, which counted diagonals as well as straight lines, in a copy of its code that never runs.
Gemini is the asterisk. The account is my own Google AI Pro subscription, so genuine free-tier Gemini wasn’t testable through it. The picker loads on 3.1 Pro; I set it to 3.6 Flash as the closest thing to what a free user gets by default. That caveat rides on every Gemini result below, and the exact wording from the capture record is at the foot of this post.
Every one of them followed the rule. Not one of them questioned it
Rule 1 was this: the food is shy. Whenever the snake’s head ends a move exactly one square away from the food, the food hops off to a random empty square.
Eleven of the twelve implemented that exactly. They check the distance after every move, they hop when it hits one, and you can watch it happen in any of the embeds below. The twelfth counted diagonals as one square away too, which is a different rule from the one I wrote, and it sits in the copy of Gemini’s code that never runs.
Here is what nobody said out loud. The square one away from the food is the square you have to stand on to eat it. Every normal approach trips the rule on the tick before the meal.
I hadn't written an interesting twist, I'd written a locked door, and four assistants fitted the lock beautifully without once looking up.
There are two ways in, and both are accidents
A hop can land the food right next to the snake’s nose, which almost never happens. Or you use the seam: my second rule wraps the left and right edges, and none of the twelve distance checks account for that wrap, so a piece of food sitting in the edge column never sees you coming from the far side. Twelve independent builds, and the same blind spot in every one.
“Each time the snake’s head ends a move exactly one square away from the food, the food hops to a random empty square.”
Eleven for twelve, exactly as written. Distance checked after every move, hop fired correctly, graded on at least three separate provocations in every game that runs at all.
That this makes eating almost impossible.
Zero for twelve. No warning, no note, no “you may want to make this two squares instead of one”. The letter was perfect and the consequence went unexamined.
That’s the finding I’d take away from the whole exercise, and it isn’t really about games. A model that follows a bad instruction perfectly gives you no signal at all that the instruction was bad. You get working code and a quiet room.
Four games you can play, and four that will not start
Every game below is the best of that model’s three goes, unedited, exactly as it arrived. They load on a click, into a frame that can run scripts and do nothing else: no storage, no network, no reach into this page.
You need a real keyboard and the arrow keys. On a phone these will load and then sit there politely doing nothing, which is neither their fault nor yours. The screenshots below show you what you’re missing.
ChatGPT wrote the plainest games in the set, and the only ones that never went wrong
Steer at the food and watch it flinch. Ride off the right edge and you come out of the left. Touch the top or bottom and you’re done.
Settled by: the file itself, loaded in a sandboxed frame and played on a keyboard · the brief, verbatimNo colours to speak of, no animation, a canvas and a score. It’s also the game where I finally managed to eat something. ChatGPT’s builds are the only ones that landed a meal in more than one run, which after an evening of watching dinner scuttle off across the board felt like a national holiday. Gemini’s third build got one too, entirely by accident, before I had touched a key.
ChatGPT Boring, and boring is the whole point. Nothing to look at, nothing to explain, and nothing that stopped working the moment it left the chat window.
Grok was the only one that thought about the person who has to press a key
It tells you how to start before you have to guess, and any arrow key restarts it after you die.
Settled by: the file itself, loaded in a sandboxed frame and played on a keyboard · the brief, verbatim“Press any arrow key to start” is written on the board when it loads. That sounds like nothing. It’s the difference between a game that works inside somebody else’s page and one that looks broken for a few seconds until you find the button. Grok’s second build has neither the hint nor the keyboard restart, so run 1 is the one embedded here.
Grok The same reliability, with manners on top. One of the three forgets to tell you how to start, which is friction rather than failure.
Claude built the best-looking game in the set and then shot two of its three in the foot
The snake has a face. When you hit the top or bottom wall it tells you the glass has sealed.
Settled by: the file itself, loaded in a sandboxed frame and played on a keyboard · the brief, verbatimClaude built a terrarium. Pale teal snake, warm yellow food, a little face on the head, and a game-over message that reads “The glass has sealed”. Nobody asked for any of that and it’s the nicest artefact of the twelve by a wide margin.
It’s also the only one of Claude’s three that runs on this page. Two of its three builds die in the frame, and the cause is a few paragraphs down.
Claude Best game, worst odds. One of three survives the frame, and both casualties fell to the same optional line.
Claude wrote the game I most wanted to keep, and two thirds of the time it wouldn’t have survived leaving the chat window.
Gemini’s good build is genuinely good, which makes the other two worse
A clean board and a clear "Wall Collision!" when you clip the top. This one behaves exactly as briefed.
Settled by: the file itself, loaded in a sandboxed frame and played on a keyboard · the brief, verbatimRun 1 is tidy, well presented and correct on all three rules. It also needed a second look before I’d call it alive: it opens on an empty board, which looks identical to a board that’s never going to fill in, so the only honest way to grade it was to prod it and watch a snake appear. Gemini’s third build needed a far bigger correction, and that one is at the foot of this post.
Gemini One of three, and the set’s only total failure. Run 2 announced itself as a complete, self-contained file inside a file that cannot parse.
pass · partial · mistake · confidently wrong
Four dead Start buttons, and every one of them is a shipping problem
Four of the twelve games won’t start on this page: two of Claude’s three builds, and two of Gemini’s. Press their buttons. Nothing happens, and that’s the exhibit.
Three of these four are complete, correct, playable games. They only die here, in a frame with the shutters down.
Three of them carry a line like this, unguarded, near the top of the script:
// Claude's version of the line that kills three of the twelve
let best = Number(localStorage.getItem('shySnakeBest') || 0);That reads the browser’s saved-data store to remember your high score. Nobody asked for a high score. In a frame this locked down, touching that store throws an error before anything else runs, and the whole script dies with it. A try around that line and all three would get off the start line. Each of them has a second unguarded call further down, so they’d still trip later on, but you would at least get to play first.
And then there’s the fourth, which is a different animal entirely
that is the code file, not the chat window. What follows that line is an entire second copy of the document inside a markdown code fence, sitting in the middle of a live script.
The file was pulled byte-for-byte through Gemini’s own Download button, so this isn’t a copy-paste accident at my end. A browser reads a script block all the way through before it runs any of it, so one broken statement two thirds of the way down means none of it runs, not even the perfectly good code above the break. Click Start and you get “startGame is not defined”. The game announces its own completeness in the same breath as it fails to exist.
Nobody fell for the speed-up, which is the part I expected them to fail
My third rule was deliberately backwards. Just about every Snake game gets faster as you eat; I asked for one that gets slower for five seconds every fifth meal. It’s the sort of instruction a model pattern-matching on “snake speed rule” would quietly correct on your behalf.
The trap I built caught nobody. The cliché was the easy thing to write and not one of the four reached for it, which makes this the least dramatic paragraph in the post and the one I was most pleased to write.
I have to be straight about the limit here: none of the twelve reached the fifth meal in graded play, because of the shy-food problem above. That verdict is read from the code, not from the screen. If you manage to eat five in one of the embeds, you’ll be the first, and I’d genuinely like to hear about it.
Which one I’d hand to somebody else, and which one I’d only look at
- ChatGPTThe one I’d trust to hand to somebody else. Three clean deliveries, no decoration, no surprises when it leaves your machine.
- GrokEqually reliable, and thinks about the player. Says what to press before you need it, and restarts on any arrow key instead of making you hunt for a button.
- ClaudeThe one I’d want to look at. Also the one most likely to add something nobody asked for that breaks it somewhere else.
- GeminiOne good build, one dead one, and one that shipped a chat preamble as code. One in three is worse odds than a coin toss.
- Open what you’re given before you believe it. Four of these twelve look completely finished from the chat window and are not.
- A rule followed perfectly is not a rule that was checked. Twelve builds, one silly instruction, no pushback from anybody.
- The extras are where the breakage lives. Every failure here came from something nobody asked for.
Ask the model what your rule will do, not just whether it can do it. One follow-up question would have caught the whole thing before a line of code got written, and it costs nothing to ask.
What this isn’t: a verdict on how well these models code. It’s one shot from a chat box with no follow-ups, which is how most people use them, and nothing more than that. Inside a coding tool that can run the file and watch it fail, the storage line would announce itself in the console on the very first load.
How I ran this, if you want the boring bit
Same prompt, typed into four assistants on the evening of 6 August 2026. Free tiers with one declared exception, a fresh chat every time, memory turned off where the setting exists, three goes each, first complete reply kept, no follow-ups and no regenerating. One Gemini attempt sat at zero output for over five minutes and was abandoned for a fresh chat, so nothing generated was thrown away, because it never generated anything.
Three rule changes from classic Snake, all three required:
1. The food is shy. Each time the snake's head ends a move exactly one square away from the food, the food hops to a random empty square.
2. The walls disagree. Going off the left or right edge wraps around to the other side. Hitting the top or bottom edge ends the game.
3. Milestone slowdown. Every fifth piece of food eaten makes the game noticeably SLOWER for five seconds, then speed returns to normal.
The grading, committed before the first reply
The grading criteria went into a spec file that was committed at 18:39 that evening, three minutes before the first reply came back and before I had read a single one: does it load in a sandboxed frame, does it play, does the food hop, do the side walls wrap while the top and bottom kill, does the milestone slow it down, and does it reach the internet. Every game was graded by playing it on a real keyboard, and every one was checked for outside network requests twice, once by reading the file and once by watching what it asked for while running. All twelve are clean: zero external references, zero requests. The marks beside each run are the site’s standing verdict states, with a fourth added here for a game that is correct in the file and dead in the frame.
The tiers and the files
Tiers, exactly as captured.
- ChatGPT: free, auto-routed, no version string shown.
- Claude: free plan, Sonnet 5 Medium, delivered as an Artifact.
- Grok: free tier, Fast model, Private Chat mode, so no chat links survive.
- Gemini: verbatim from the capture record, “this Lane A account is Ben’s own Google AI Pro subscription (‘Ben Dixon Pro’ shown in the account badge); genuine free-tier Gemini is not directly testable through this profile (no separate free Google account is logged in). Default model was 3.1 Pro (‘Advanced math & code’) on load; switched to 3.6 Flash (‘All-around help’) as the closest available proxy to what a free-tier user gets by default.”
All twelve files are on this site, exactly as they arrived. Four of them never appear above, because this post shows one build per model and the test ran three. ChatGPT 1 2 3 · Claude 1 2 3 · Gemini 1 2 3 · Grok 1 2 3. Nothing was tidied, reformatted or patched. If I had quietly fixed the high-score line, you would be able to tell.
One absence and one correction
Perplexity sat this one out, deliberately. On the day I ran this, its app-building product was behind the paywall, so putting free Perplexity in this column would have been marking it against a promise it doesn’t make. It gets tested on the things it says it does.
One correction worth telling you about. Gemini’s third build was first graded unplayable: seven attempts to steer it, seven deaths against the top wall. That was wrong, and it was my rig’s fault. The snake reaches the fatal wall in about a second, faster than an automated tool can press a key and read the screen back. Re-run by hand at human speed, it steers perfectly. The machine grading the game was slower than the game. Two of Claude’s builds got the opposite correction in the same session: they rendered a start screen, looked alive, and were already dead.
A picture of a game is not a game, and pressing the button is the only way to tell.
The short version
The test. Four AI assistants, one brief, three goes each, 6 August 2026. Build a Snake game in one file with three odd rules: shy food, side walls that wrap while the top and bottom kill, and a slowdown every fifth meal.
What happened. Twelve games, and thirty-five of the thirty-six rule changes landed exactly as written. Eight survive being embedded in a hardened frame: ChatGPT three of three, Grok three of three, Claude one of three, Gemini one of three. Every failure happened on delivery. The game logic held every time.
The bit that matters. The shy-food rule, followed exactly as written, makes the game close to unwinnable. Three meals were recorded across twelve games, and nothing ever reached the fifth. Not one model said a word about it.
Common questions
- Can AI build a playable game?
- Yes. Eleven of the twelve games I got back were real, playable Snake, and thirty-five of the thirty-six rule changes landed exactly as written. Nothing broke in the game logic. Four of the twelve broke on delivery: one shipped with a chat preamble pasted into the code, three included an optional high-score line that kills the page inside a locked-down frame.
- Which AI wrote the best game?
- Claude wrote the prettiest one by a distance, and it's the terrarium you can play about halfway down this post. ChatGPT and Grok wrote the most reliable ones: three builds each, all three loaded and played inside the hardened frame here. Claude and Gemini managed one of three.
- Why is the game so hard to win?
- Because of my own rule, followed exactly. The food hops to a new square whenever the snake's head ends a move one square away from it, and the square next to the food is the square you have to cross to eat it. All four implemented it, and not one of them flagged what it would do.
- Does this tell me how good these models are at coding?
- Only at this: one shot, one message, no follow-ups, no error to react to. That is how most people use a chat box, so it's worth knowing. It isn't a test of what these models do inside a coding tool where they can run the file and see it fail.
Ben tests how far you can trust the main AI assistants, and publishes exactly where they get things wrong. Every post here is a first-hand test with the receipts, including the times a tool simply wasn’t worth the trust. About Ben →
The site tests how far you can trust the main AI assistants, on real decisions. Start with the Prompt Stack for the four-stage framework, free and ungated, or the Bluff Filter for the paste-ready version with a real before and after.