Start free trial

TRUE, BUTMoved Ruler

Built to resist contamination: 80.3%

Move to the benchmark built to resist contamination and the top model sits at 80.3%, and GPT-5.6 Sol lands at 64.6%.

“Frontier models have all but solved software engineering: on SWE-bench Verified, the coding benchmark quoted in launch posts, the top models now sit near 96%, with Claude Opus 5 leading at 96%.”

Frontier-model coding marketing built on SWE-bench Verified (Anthropic, OpenAI and coverage, August 2026)

Recorded 19 August 2026. Our ruling means: literally true, materially misleading.

Read the story →

Receipts pack

The evidence file for this verdict

6 receipts: 1 support the claim, 1 contradict it, 4 add context. 4 figures on the record, 4 traced to a quoted receipt.

The receipts

Each source, the line we quote from it, its date and which way it cuts. Free to read and to cite.

  1. 1Supports the claimbenchlm.ai ↗

    Claude Opus 5 leads the SWE-bench Verified leaderboard on BenchLM's August 2026 update with 96%

    Look up an archived copy ↗

  2. 2Contextbenchlm.ai ↗

    suggesting this benchmark is nearing saturation for frontier models

    Look up an archived copy ↗

  3. 3Contradicts the claimbenchlm.ai ↗

    Claude Mythos 5 leads the SWE-bench Pro leaderboard on BenchLM's August 2026 update with 80.3%

    Look up an archived copy ↗

  4. 4Contextcodingfleet.com ↗

    Pro uses actively maintained repositories with no public ground-truth leakage

    Look up an archived copy ↗

  5. 5Contextcodingfleet.com ↗

    above GPT-5.6 Sol (64.6%), at $2/$6 per 1M tokens

    Look up an archived copy ↗

  6. 6Contextweb.archive.org ↗

    First, 59.4% of audited problems contain flawed test cases that reject correct solutions.

    Look up an archived copy ↗

Members' extras

What each receipt proves, and what would change our mind.

  • What each receipt above proves, in our words
  • The figures table, each number traced to the receipt that states it
  • The trick behind the claim, explained, with the guide to spotting it
  • What would change our verdict
  • Our call on the record, with its due date

The verdict, the opening and every receipt above stay free. Membership adds our analysis to the evidence file for every checked story.

First membership: 30 days free, then A$89 a year. One introductory trial per customer. Card required; renews annually until cancelled. Cancel before the trial ends to avoid the first charge.