The trick: Moved Ruler
AI task length is doubling every seven months, METR's chart says.
A new preprint finds the same tenfold jump is much easier from 3 to 30 minutes than from 30 minutes to 5 hours, and a METR author warns the error bars run about a factor of two.
METR's March 2025 post says the length of tasks AI agents can complete has been consistently exponentially increasing over the past 6 years, with a doubling time of around 7 months.
Before you read on. Your call?
TRUE, BUT
10x
On 8 October a two-author arXiv preprint recomputing the time horizons on 228 tasks and 26 AIs. Their abstract says the fitted curve from human time to AI difficulty is nearly flat from 2 to 30 minutes and close to linear elsewhere, so a jump from 3 to 30 minutes is much easier than one from 30 minutes to 5 hours despite the same multiplier.
The twist
A METR author wrote in January 2026 that many people overstate the precision of the measurements and draw conclusions the evidence does not fully support, with error bars historically about a factor of 2 each way. The abstract does not say whether the trend itself changes.
There’s more to this story.
Membership opens the full investigation, the strongest counterargument and what to do with what you’ve learned.
30 days free, then A$89 a year. Payment card required. Cancel before the trial ends to avoid the first charge.
Review my 30-day free trial →- At sign-up: 30 days free. Payment card required. One introductory trial per customer.
- Your trial ends 30 days after you start. Unless you cancel before then in Account → Manage subscription, we charge A$89 for the first year. It renews automatically at A$89/yr until cancelled.
- You can ask for a full refund within 14 days after any annual payment, renewals included, with no reason needed, by emailing hello@bskiller.com. This voluntary refund does not limit your rights under the Australian Consumer Law.
By starting your trial, you agree to the Terms.
BS Killer is published by Inferno Tech Pty Ltd, ABN 27 647 413 474.
Get tomorrow’s check free by email.
Couldn't check your access. That's on us.
The trick has a name
We call it Moved Ruler: two methods, two answers, one of them quoted. You'll see it again. Learn to spot it →
Receipts
- Supports metr.org:
We show that this metric has been consistently exponentially increasing over the past 6 years, with a doubling time of around 7 months.
- Supports metr.org:
If we plot this on a logarithmic scale, we can see that the length of tasks models can complete is well predicted by an exponential trend
- Refutes arxiv.org:
Hence, a time-horizon jump from 3 min to 30 min is much easier than one from 30 min to 5 hours despite the same multiplier of
- Context arxiv.org:
On 228 tasks and 26 AIs, we recompute the time horizons using splines and item-response theory to relax the assumption that the AI difficulty of a task depends linearly on the log of human time.
- Context arxiv.org:
human time to AI difficulty; it is nearly flat in a region from 2--30 min but close to linear elsewhere
- Context arxiv.org:
we contribute time-horizon point estimates that perform better under a cross-validated suite of proper scoring rules
- Context arxiv.org:
We suggest that time horizons be interpreted together with the diagnostic plots
- Context metr.org:
many people to some extent both overstate the precision of our time horizon measurements and draw conclusions I don't think the evidence fully supports
- Context metr.org:
Claude Opus 4.5 has a 50%-time horizon of around 4 hrs 49 mins (95% confidence interval of 1 hr 49 mins to 20 hrs 25 mins)
- Context metr.org:
Error bars have historically been a factor of ~2 in each direction
- Context metr.org:
Time horizon is not the length of time AIs can work independently
- Context metr.org:
While I still believe in the core results
- Context metr.org:
This page is no longer actively updated.
- Context metr.org:
Measurements above 16 hrs are unreliable with our current task suite
- Context metr.org:
The static figures and some claims in the text (e.g. the doubling time) reflect the state of the data at the time of original publication.
- Context epoch.ai:
Why don't AIs automate more real-world tasks if they can handle 1-hour ones?
- Context arxiv.org:
Thu, 8 Oct 2026 17:59:50 UTC
- Supports metr.org:
We propose measuring AI performance in terms of the length of tasks AI agents can complete.
Open the Receipts Pack → What each source proves, every figure traced, and what would change our verdict.
