For eleven weeks, the AI evaluation nonprofit METR sat inside Anthropic with access to internal R&D data, the kind of access frontier labs almost never grant. Their report, published November 5, lands on a number people will be quoting for a while: Anthropic is likely improving model capabilities on well-specified tasks at roughly twice the rate it did between March 2024 and October 2025.
The main driver, in METR’s assessment, is Anthropic using its own AI to do AI research. The loop safety researchers have theorized about for a decade has a measured rate now, an error bar, and a footnote saying most of the evidence is confidential.
What METR actually measured
The access ran from August 12 to October 29. METR compared Anthropic’s rate of capability improvement during that window against the lab’s March 2024 to October 2025 baseline, on tasks with clear success criteria — the kind you can actually score.
The headline estimate is a 2x speedup. To their credit, METR publishes the uncertainty instead of hiding it: the true figure could plausibly be 1.3x, or 3x. That is a wide band. It is also a band that never dips below “meaningfully faster than before.”
Two findings make the attribution interesting. First, METR attributes most of the acceleration to AI-assisted research itself; other factors dominated the picture in late 2025 but they do not explain the current slope. Second, METR looked for bottlenecks and diminishing returns and found neither. Whatever is fueling this run, they saw no sign it is about to flatten.
The part nobody outside can check
Here is where the report gets strange. Anthropic redacted a majority of the evidence METR found most informative, on the grounds that it is “confidential information that could accelerate other AI developers.” Safety lead Sam McCandlish defended the cuts as avoiding “handing out a roadmap for others to go faster.”
So the public conclusion rests on evidence the public cannot see, evaluated by an evaluator who says so openly. METR flags this as far from ideal. I believe them, and I also find the tension more interesting than the headline number. Anthropic is simultaneously saying “trust our external assessor” and “not with the receipts.” Both positions are defensible. Holding both at once is the entire governance problem of frontier AI compressed into one PDF.
Model 4, hiding in the footnotes
Some details leaked through the redactions anyway. Anthropic has internal models ahead of the public Claude Opus 5.5. One of them, called “Model 4” in the report, cleared 50 percent on all 15 benchmarks METR tested. All fifteen. If your mental model of the current frontier is whatever was announced last, it is stale by some unknown number of generations.
Then there is the admission from METR’s own Ryan Greenblatt: he expected a slower speedup this year and says he was “seriously wrong.” When the professional calibrators are surprised downward, the rest of us should widen our intervals.
One more thread worth pulling. Anthropic’s S-1 filing defines a danger threshold as “dramatically accelerated progress attributable to AI,” and a Monday update suggested that point may be close. Read those two documents together and the picture gets uncomfortable: the company’s own paperwork may be describing conditions that its own external assessment says are materializing.
What this means if you build on these models
Most people reading this do not run evals on frontier labs. You run a business on top of their outputs, which is a different kind of exposure.
Capability velocity is now a planning variable. If the models underneath your product improve twice as fast as they did last year, a few things follow. Any “which model should we standardize on” decision depreciates faster than it used to. Any prompt architecture or agent scaffold tuned to a specific model’s quirks ages faster too. And the gap between your private evals and the vendor’s public benchmark claims gets harder to reason about when an unknown number of stronger internal models sit ahead of the public frontier.
The redactions reach you indirectly. If the strongest evidence about AI progress rates stays confidential inside the lab it concerns, then everyone outside, your competitors included, is planning on filtered data. Humility about forecasts is the rational response. Yours included.
What to do now
- Read the METR report yourself before reading commentary about it (including this). It is unusually readable for the genre, and the redaction disclosures are listed explicitly rather than buried.
- Add an eval-cadence line to your AI roadmap: re-benchmark your core workflows quarterly. A 2x capability slope will roll over your assumptions mid-year if you let it.
- Treat model choice as a lease, not a purchase. Keep an adapter layer between your product and the underlying model so swapping is a config change, not a rebuild.
- Watch the “dramatically accelerated progress” language in Anthropic’s S-1 updates. If a company flags its own danger threshold, take the flag seriously.
The number everyone will remember is 2x. The number I keep thinking about is fifteen, as in fifteen out of fifteen benchmarks cleared by a model you cannot use yet, inside a company that just told an outside evaluator most of the evidence has to stay secret. Progress like this used to be a prediction. Now it is an audit finding.
