Anthropic Grants METR Employee-Level Access, Elevating AI Risk Assessment Transparency

The AI safety evaluation field is witnessing a rare experiment in transparency. According to RSS sources, Anthropic is granting independent research organization METR (Model Evaluation & Threat Research) employee-level internal access, allowing its researchers to deeply examine model development processes, internal risk records, and incident handling mechanisms. More critically, Anthropic has waived its right to block METR from publishing its evaluation conclusions.

What Employee-Level Access Actually Means

"Employee-level access" is the most noteworthy detail of this collaboration. Typically, external research institutions' evaluations of AI companies remain at the level of API calls, public documentation, or limited black-box testing. Employee-level access, however, means METR researchers can access training logs, internal communications, risk incident reports, and even parts of the decision-making process—just like an Anthropic insider.

This depth is nearly unprecedented in the AI industry. Even in safety collaborations with government agencies, AI companies typically retain the right to filter sensitive information. By choosing to open up to this level, Anthropic is essentially exposing the "back kitchen" of its internal governance directly to external auditors.

METR (Model Evaluation & Threat Research) is a nonprofit research organization focused on independent evaluation of frontier AI model capabilities and risks. It originated from the evaluation team at ARC (Alignment Research Center), which received early access to GPT-4 before its release and conducted red-team testing on it. METR's work centers on measuring model performance in high-risk capabilities such as autonomous replication, cyberattacks, and deception.

In standard AI safety evaluation practice, black-box testing refers to evaluators only being able to observe model behavior through API inputs and outputs, without access to training data, weight updates, or internal decision records, while white-box access allows evaluators to examine the model's internals. The scope Anthropic has opened goes even beyond white-box, extending to organizational-level governance documents and incident records—a first for the industry.

Waiving the Veto: Trust or Strategy

Even more significant than the access itself is Anthropic's decision to waive its review veto over METR's publications. According to the original information, METR will independently publish its assessment of Anthropic's risks, incidents, and practices, and Anthropic has no right to block it.

This means that if METR discovers vulnerabilities or governance deficiencies in Anthropic's model safety, it can independently disclose them to the public. For a company competing at the frontier of model capabilities, this requires considerable confidence—or rather, it is a strategy of converting transparency itself into a competitive moat.

Against the backdrop of growing calls for AI regulation, voluntarily relinquishing narrative control may be more effective at building public trust than passively accepting regulation.

In past collaborations between AI companies and external evaluators, publication veto has been an almost default clause. Companies typically require reviewing draft evaluation reports and have the right to demand removal of content involving trade secrets or safety-sensitive information, or even to block publication outright. Collaborations between OpenAI and Microsoft, and between DeepMind and external auditors, all retain varying degrees of review authority.

By waiving this right, Anthropic means that METR's evaluation reports will enter the public domain in a form completely unmodified by Anthropic. This arrangement is known in academic auditing as unrestricted publication right, but it is extremely rare in commercial AI labs, as reports could expose competitive disadvantages or trigger regulatory scrutiny.

The Value and Tension of Independent Evaluation

As a third-party evaluator, METR's core value lies in its independence. Employee-level access upgrades evaluation from "interviews + document review" to "immersive observation," capable of capturing risk signals that are filtered out or unrecognized in official reports.

But this arrangement also contains inherent tension. While researchers gain deep access, how do they maintain the independence of their analytical framework? Although Anthropic has waived its publication veto, the access itself is still granted by Anthropic. Whether this power asymmetry affects the sharpness of the evaluation warrants continued observation.

From an industry perspective, this collaboration provides a reference sample for "how AI companies can coexist with independent evaluators." If METR's evaluation reports can maintain substantive critical rigor, this will become a milestone in AI safety governance; if the reports turn out mild, it could weaken the demonstrative significance of such arrangements.

Implications for AI Safety Governance

Frontier AI labs' safety practices have long faced the trust dilemma of "self-evaluation." Safety reports published by companies themselves, no matter how detailed, struggle to fully dispel external doubts about their selectivity. Introducing independent institutions with deep access is one viable path to breaking this dilemma.

Anthropic's move also exerts a certain pressure on peers: when one company is willing to open up to the employee level and waive its veto, it becomes harder for other labs to justify refusal when facing similar demands.

Of course, a single collaboration is not enough to define an industry standard. Whether METR's forthcoming evaluation content is sufficiently in-depth and reveals substantive issues is the true yardstick for testing whether this mechanism is effective.

The self-evaluation dilemma is a core challenge in AI safety governance: model developers are both players and referees, and their published safety reports are inevitably questioned for selective disclosure. Similar to how financial auditing requires external accounting firms to be independent of the companies they audit, the AI safety field is also exploring similar third-party checks and balances.

Existing attempts include: evaluation agreements signed between the UK AI Safety Institute (AISI) and multiple labs, the US NIST's AI risk assessment framework, and third-party compliance assessments stipulated in the EU AI Act. However, the access depth of these arrangements typically falls far short of employee-level access, and most still retain the company's right to filter information. If the Anthropic-METR collaboration proves effective, it could push the industry toward establishing a norm of deeper external access.

One-line Summary

Anthropic grants METR employee-level access and waives publication veto, setting a precedent for deep transparency in AI safety evaluation.

Paragraph Summary

Anthropic is granting independent research organization METR employee-level internal access, allowing its researchers to examine training logs, internal risk records, and decision-making processes—a nearly unprecedented arrangement in the AI industry. More significantly, Anthropic has voluntarily waived its publication veto over METR's evaluation reports, meaning METR can independently publish its findings without Anthropic's approval. This arrangement upgrades AI safety evaluation from traditional black-box testing and limited document review to immersive observation, providing a reference sample for how AI companies can collaborate with independent evaluators. However, since the access is still granted by Anthropic, whether this power asymmetry affects the critical rigor of the evaluation—and whether the forthcoming report is sufficiently in-depth—will be the true yardstick for testing the mechanism's effectiveness.