The trick: Lab Not Field
The Home Office piloted AI summaries of asylum interviews and reported 23 minutes saved per case with 9% inaccurate.
After rollout, 45 of the 203 summaries experts checked failed. They checked about 0.3% of them.
The Home Office's pilot evaluation said AI interview summaries could save 23 minutes per case, with 9% found inaccurate.
Before you read on. Your call?
TRUE, BUT
45 of 203
After rollout, Home Office experts failed 45 of 203 sampled summaries, 22%, per figures Open Rights Group obtained.
The twist
Those 203 were about 0.3% of the summaries produced, and a summary that omitted details could still pass.
There’s more to this story.
Membership opens the full investigation, the strongest counterargument and what to do with what you’ve learned.
30 days free, then A$89 a year. Payment card required. Cancel before the trial ends to avoid the first charge.
Review my 30-day free trial →- At sign-up: 30 days free. Payment card required. One introductory trial per customer.
- Your trial ends 30 days after you start. Unless you cancel before then in Account → Manage subscription, we charge A$89 for the first year. It renews automatically at A$89/yr until cancelled.
- You can ask for a full refund within 14 days after any annual payment, renewals included, with no reason needed, by emailing hello@bskiller.com. This voluntary refund does not limit your rights under the Australian Consumer Law.
By starting your trial, you agree to the Terms.
Inferno Tech Pty Ltd t/a BS Killer, ABN 27 647 413 474
Get tomorrow’s check free by email.
Couldn't check your access. That's on us.
The trick has a name
We call it Lab Not Field: it works in the test conditions, not the deployed ones. You'll see it again. Learn to spot it →
Receipts
- Supports gov.uk:
the pilots suggest that the ACS tool could save 23 minutes per case when reviewing interview transcripts
- Supports gov.uk:
On average, the test group reviewed transcripts 23 minutes quicker than the comparison group (a 32% time saving).
- Supports gov.uk:
A small proportion of summaries produced (9%) were deemed to be inaccurate or had missing information
- Context gov.uk:
full evaluation after deployment is recommended to capture the impact of the tools
- Refutes theregister.com:
45 of 203 AI-generated interview summaries failed initial expert checks after the system was rolled out in April 2026.
- Context openrightsgroup.org:
Just 0.3% of total generated summaries between April-July were checked by Home Office Subject Matter Experts (SME).
- Context theregister.com:
While those figures don't prove that AI caused the slowdown
- Context theregister.com:
summaries could pass the checks even if they omitted important details, as long as the information they did include was accurate.
- Context theregister.com:
The Home Office did not respond to The Register's questions.
- Refutes theregister.com:
That's a 22 percent failure rate, compared with the 9 percent rejection rate recorded during the original pilot.
Open the Receipts Pack → What each source proves, every figure traced, and what would change our verdict.
