AI Tests · Test
AI knew the new paternity leave rule. Some still got the pay wrong
- 5 min read
This site is the AI Reliability Tracker: we check six AI apps’ answers every month.
Free, with Confidently Wrong every fortnight.
Copilot said the new starter would likely get statutory paternity pay too. The 26-week rule that says no still applies.
“you are likely to qualify for statutory paternity pay as well.”
Not on GOV.UK’s rule. The pay still needs 26 weeks’ continuous work for the employer, and the person in our question falls short.
Paternity leave became a day-one right in April 2026. A new rule is where you’d expect the apps to trip.
They didn’t. Every answer that searched had the new leave rule right. The slips were on the money, where the 26-week rule still applies.
Two runs, six apps
We asked six apps three times each in fresh chats, in tracker run 1 (3 to 5 October 2026) and run 2 (11 October). Every answer searched except Gemini’s two run 1 misses. Run 1: ChatGPT logged out, Claude paid (Max), Perplexity paid (Pro), Gemini and Grok free. Run 2: all free. The question is sealed. Grading: how we test.
Right on the leave, shaky on the pay
| Assistant | Run 1 leave | Run 1 pay mistakes | Run 2 leave | Run 2 pay mistakes |
|---|---|---|---|---|
| ChatGPT | 3 of 3 | 2 of 3 | 3 of 3 | 0 of 3 |
| Claude | 3 of 3 | 0 of 3 | 3 of 3 | 0 of 3 |
| Copilot | 3 of 3 | 1 of 3 | 3 of 3 | 1 of 3 |
| Gemini | 1 of 3 | 0 of 1 | 3 of 3 | 1 of 3 |
| Grok | 1 of 1 answered | 1 of 1 | 3 error cards | error card |
| Perplexity | 3 of 3 | 2 of 3 | 3 of 3 | 1 of 3 |
| Total | 14 of 16 | 6 of 14 | 15 of 15 | 3 of 15 |
pass · partial · miss · confidently wrong
Pay mistakes: answers right on the leave that said, implied or marked the 26-week pay test as met.
Claude is the row to stare at, because it kept the two halves apart. One of its answers put it in nine words: “The 26-week service rule was dropped for leave only.”
- ChatGPTLeave right six times. Two pay mistakes, one hedged “appear likely”.
- ClaudeBoth halves right, six of six.
- CopilotLeave right six times. One pay mistake per run, both hedged “likely”.
- GeminiOld leave rule twice in run 1. One “Qualified” table in run 2.
- GrokOne answer in six: leave right, pay wrong. Five error cards.
- PerplexityLeave right six times. Three pay mistakes, each conditional.
What paternity pay still needs
Who qualifies for the leave. Employees, from their first day of employment.
Who qualifies for the pay. 26 weeks’ continuous employment with the employer up to any day in the qualifying week (the 15th week before the baby is due), staying employed up to the birth, plus earnings and notice conditions.
If you qualify for Statutory Paternity Pay (SPP), it’s £194.32 a week, or 90% of average weekly earnings if lower, taxed like wages.
Three ways the pay slipped
Perplexity’s run 1 slips came with a condition: “so you should also qualify for SPP provided you meet the earnings threshold.” The condition is earnings. It counts the 26 weeks as passed, and on GOV.UK’s rule they aren’t.
Grok’s one run 1 answer worked out the 26 weeks itself, got it wrong, and announced: “you will meet the service requirement for SPP.” A calculator marking its own sum with a gold star.
“Statutory Paternity Pay (SPP) … Qualified”
A few lines lower: “you will not qualify for Statutory Paternity Pay (SPP) from the government scheme due to the length-of-service requirement for pay.” Approved in the box, declined underneath.
Gemini’s two leave mistakes
Two of Gemini’s three run 1 answers skipped the search and served the old leave rule. One closed: “No, you are not entitled to statutory Paternity Leave or Statutory Paternity Pay from this employer”.
What this shows, and what it doesn’t
One question doesn’t show AI gets every new law right, or that pay is always where it slips. It shows where to look first. Plans changed between runs, so they aren’t a before and after.
One Grok try worked for ten minutes, gathered 40 sources, then said “Grok was unable to finish replying.” A librarian who fetches forty books and goes home.
The opposite failure, an old rule served as current, is in our ISA test.
Summary
The short version
Common questions
- Can I take paternity leave if I've just started a new job?
- Yes. Since the April 2026 change, GOV.UK says employees can take Paternity Leave from their first day of employment, for one or two weeks. In our two tracker runs, every AI answer that searched the web got this part right.
- Will I get Statutory Paternity Pay in a new job?
- Not unless you've worked for the employer continuously for at least 26 weeks up to any day in the qualifying week, the 15th week before the baby is due, are still employed there up to the birth, and meet the earnings and notice conditions. GOV.UK still sets that rule, and it's where the AI answers went wrong.
- Which AI apps got the pay rule wrong?
- In run 1, 6 of the 14 right answers said or implied the 26-week pay test was met: ChatGPT, Perplexity, Copilot and Grok. In run 2 it was 3 of 15: Copilot, Gemini and Perplexity, one each. Claude said no statutory pay on all six of its tries.
Confidently Wrong, every fortnight.
Which AI told the truth this month, with the receipts, plus an AI story worth retelling in between.
Free. Leave whenever you like.