At a glance
- Anthropic says a METR team is receiving deeper internal access.
- Publication rights and confidentiality boundaries both matter.
- An evaluation arrangement is not a guarantee of model safety.
The announcement
On October 15, Anthropic described an arrangement giving a METR team employee-like access, along with a publication process and restrictions on some information. Source 1
Ask three separate questions
What information can evaluators inspect? What can they publish independently? What remains outside their access? The source describes different partnership tiers, so one team’s access should not be attributed to every evaluation organization.
Our analysis: request the evidence boundary
When procuring models, a single score or certification-like phrase is insufficient. Ask which tasks were tested, under what conditions, what failed and whether the results will be revisited. Keep undisclosed areas explicitly unknown.
The arrangement is not a finding
This is a report about access and process. It does not establish that a model is free of risk. Our article summarizes Anthropic’s account; it does not publish conclusions on behalf of METR. Future findings should be read from the evaluators themselves.
Application testing remains necessary
Connecting a model to company documents, tools and permissions introduces new failure modes. Teams still need workflow-specific acceptance checks, review boundaries and recovery paths, even when model-level evaluation becomes more extensive.
Sources & verification
Primary materials consulted for this article. Vendor measurements are not independent Youjie tests.
- An update on independent evaluators ↗Anthropic · Source date 2026-10-15 · Checked Oct 16, 2026