The trick: Self-Marked
Google's sign language model beat every previously reported score on the benchmark.
Google wrote the benchmark.
Google DeepMind says SL2T is the most capable sign language translation model to date, scoring 70 BLEURT zero-shot on FLEURS-ASL, significantly higher than any previously reported score, and ships it as sign-to-text dictation in Gboard and Live Transcribe on Pixel 11
Before you read on. Your call?
TRUE, BUT
70
The model is probably the real thing. The superlative was graded on a Google-built exam, and the 'on your phone' feature phones a server with every sentence you sign.
There’s more to this story.
Membership opens the full investigation, the strongest counterargument and what to do with what you’ve learned.
30 days free, then A$89 a year. Payment card required. Cancel before the trial ends to avoid the first charge.
Review my 30-day free trial →- At sign-up: 30 days free. Payment card required. One introductory trial per customer.
- Your trial ends 30 days after you start. Unless you cancel before then in Account → Manage subscription, we charge A$89 for the first year. It renews automatically at A$89/yr until cancelled.
- You can ask for a full refund within 14 days after any annual payment, renewals included, with no reason needed, by emailing hello@bskiller.com. This voluntary refund does not limit your rights under the Australian Consumer Law.
By starting your trial, you agree to the Terms.
BS Killer is published by Inferno Tech Pty Ltd, ABN 27 647 413 474.
Get tomorrow’s check free by email.
Couldn't check your access. That's on us.
The trick has a name
We call it Self-Marked: graded by the party that benefits from the grade. You'll see it again. Learn to spot it →
Receipts
- Supports deepmind.google:
SL2T is the most capable sign language translation model to date according to key benchmarks like FLEURS-ASL (sd-test).
- Supports deepmind.google:
SL2T achieves a remarkable zero-shot score of 70 BLEURT, which is significantly higher than any previously reported score.
- Refutes deepmind.google:
only these geometric coordinates are sent to the server for translation, allowing the original video to be discarded immediately
- Context siliconangle.com:
It then sends the coordinates of these locations to its cloud-based server, avoiding the need to upload any actual video, making it faster and more secure for users.
- Context arxiv.org:
we introduce FLEURS-ASL, an extension of the multiway parallel benchmarks FLORES (for text) and FLEURS (for speech) to support their first sign language (as video), American Sign Language, translated by 5 Certified Deaf Interpreters
Open the Receipts Pack → What each source proves, every figure traced, and what would change our verdict.
