The trick: Self-Marked
Google calls EmbeddingGemma 2 best-in-class for its size.
The 9.92-point code gain, from 68.76 to 78.68, is the lab's own measurement.
EmbeddingGemma 2 is a best-in-class open model for natively multimodal embeddings.
Before you read on. Your call?
TRUE, BUT
9.92
Google reports a 9.92-point MTEB Code gain, from 68.76 to 78.68, and the developer guide calls that 14% higher than EmbeddingGemma 1. The model card says among the strongest under 1B parameters.
The twist
CodeSOTA says these are publisher figures, that they do not establish a rank against scores whose suite revision is unspecified, and that its page has no CodeSOTA run for this model.
There’s more to this story.
Membership opens the full investigation, the strongest counterargument and what to do with what you’ve learned.
Get tomorrow’s check free by email.
30 days free. Payment card required. One introductory trial per customer.
Your trial ends on 30 days after you start. Unless you cancel before then in Account → Manage membership, we charge A$89 for the first year. It renews automatically at A$89/yr until cancelled.
You can ask for a full refund within 14 days after any annual payment, renewals included, with no reason needed, by emailing hello@bskiller.com. This voluntary refund does not limit your rights under the Australian Consumer Law.
By starting your trial, you agree to the Terms.
BS Killer is published by Inferno Tech Pty Ltd, ABN 27 647 413 474.
Couldn't check your access. That's on us.
The trick has a name
We call it Self-Marked: graded by the party that benefits from the grade. You'll see it again. Learn to spot it →
Receipts
- Supports blog.google:
EmbeddingGemma 2 is a best-in-class open model for natively multimodal embeddings
- Supports blog.google:
Achieves leading scores among sub-1B multimodal embedders for its size across benchmarks like MTEB (Massive Text Embedding Benchmark) Code and MAEB (Massive Audio Embedding Benchmark), while matching or outperforming many larger models across text, vision, and audio tasks.
- Supports blog.google:
delivering a significant 9.92-point improvement on code performance (in MTEB Code, from 68.76 to 78.68)
- Context ai.google.dev:
among the strongest multimodal embedding models under 1B parameters
- Context developers.googleblog.com:
EmbeddingGemma 2 scores 14% higher than EmbeddingGemma 1 on MTEB (Code)
- Refutes codesota.com:
This page has no CodeSOTA corpus-specific run for EmbeddingGemma 2.
- Context codesota.com:
No comparable score for this model exists in the historical table below.
- Context codesota.com:
These named v2/v1 results remain separate: they do not establish a rank against scores whose suite revision is unspecified.
- Context codesota.com:
The card describes 740M total parameters, with a selectively loadable 270M text component
Open the Receipts Pack → What each source proves, every figure traced, and what would change our verdict.
