AI Safety Evaluation Moves to Third Parties: Anthropic Grants METR Employee-Level Access

AI safety evaluation is shifting from model vendors' "self-reporting" toward independent external verification. Anthropic's official account recently disclosed that the company has committed to having external evaluators review its safety practices and has granted an "employee-level access" to a team from the evaluation organization METR, while also negotiating similar collaborations with multiple organizations. This statement sends a clear signal: frontier AI labs' safety commitments are attempting to achieve verifiability through third-party evaluation mechanisms.

From Commitment to Mechanism: How Third-Party Evaluation Is Implemented

In its statement, Anthropic said the company "has committed to allowing evaluators to assess our safety practices," has held discussions with multiple organizations, and "granted an employee-like access to a team from METR_Evals." The company also attached an explanatory article detailing how it works with third-party evaluators.

The statement is not information-dense, but its direction is clear. So-called "employee-level access" means external evaluation teams can access information and system permissions close to those of internal personnel, rather than relying solely on demos or summary materials provided by the vendor. For safety evaluation, the depth of access directly determines the credibility of evaluation conclusions—an evaluation that only reads public documents is entirely different in value from one that can observe actual processes, tools, and data flows.

Voluntary Commitments and the Verifiability Dilemma of Frontier AI Labs

In recent years, major frontier AI labs have successively released Responsible Scaling Policies or similar safety frameworks, committing to take corresponding safety measures when model capabilities reach specific thresholds. However, such commitments are mostly self-set by vendors, who also self-assess whether they are met, lacking an external audit component, and are therefore often criticized as "self-regulation." Third-party evaluation mechanisms are precisely a response to this dilemma: by introducing independent institutions, the judgment of "whether commitments are honored" is partially outsourced. But the effectiveness of third-party evaluation depends on three conditions—depth of access, independence of evaluation methods, and degree of public disclosure of conclusions. Anthropic's grant of employee-level access mainly addresses the first condition; the latter two remain to be seen.

Why "Employee-Level Permissions" Deserve Attention

AI safety evaluation has long faced a structural challenge: evaluators need sufficiently deep access to discover real problems, while model vendors, citing trade secrets, data privacy, and security risks, are often willing to open only limited windows. The level of permission Anthropic has granted this time has a certain demonstrative significance in this context.

A deeper implication lies in the independence of evaluation. If the scope, methods, and conclusions of evaluation are all led by the evaluated party, third-party evaluation can easily degenerate into public relations endorsement. Granting employee-level access, at least at the execution level, gives the evaluation team the conditions for independent observation and leaves room for subsequent public disclosure of evaluation methods. Of course, the specific boundaries of the access opened, whether evaluation conclusions are made public, and how discovered problems are handled—these key details cannot be confirmed from a single tweet.

METR's Background and Working Methods

METR (Model Evaluation & Threat Research) is an independent research organization focused on systematic measurement of frontier AI model capabilities and risks, with roots traceable to long-term research accumulation in AI safety evaluation methods. METR's evaluation work typically revolves around dimensions such as model autonomy in real tasks, deceptive behavior, cybersecurity, and biological risks, emphasizing quantifiable model risk levels through reproducible experimental designs. Unlike vendor self-assessment, METR's methodology requires evaluators to be able to directly invoke models, design controlled experiments, and observe model performance on unseen tasks—which is why "employee-level access" is practically meaningful to it: only with permissions close to those of internal personnel can evaluators bypass vendor-preset demo interfaces, independently construct test scenarios, and obtain raw data.

Parallel Negotiations with Multiple Institutions: The Embryo of an Evaluation Ecosystem

Anthropic mentioned it is "discussing with multiple organizations," indicating that its third-party evaluation is not an exclusive collaboration with a single institution but an attempt to build a multi-evaluator framework. For frontier labs, multi-institutional participation can disperse the perspective limitations of a single evaluator; for the evaluation ecosystem, more institutions gaining deep access opportunities helps accumulate comparable methods and standards.

METR (Model Evaluation & Threat Research) is one of the more closely watched evaluation organizations in this field, with work focused on systematic measurement of model capabilities and risks. Anthropic's choice of it as the first deep collaboration partner is related to its accumulation in evaluation methods, but the identities, evaluation scope, and timelines of other partners have not yet been disclosed.

Major Participants in the AI Safety Evaluation Ecosystem

Beyond METR, there are several types of institutions in the AI safety evaluation field: one type is academically rooted evaluation research teams, focusing on methodology and reproducible benchmarks; another is security companies specializing in red-team testing and adversarial evaluation, focusing on discovering specific vulnerabilities; and another is evaluation frameworks promoted by governments or international organizations, such as the UK AI Safety Institute and the US AI Safety Institute, focusing on standard-setting and policy alignment. The value of parallel participation by multiple institutions lies in complementary perspectives—academic teams provide methodological rigor, security companies provide practical findings, and government agencies provide standard anchors. Anthropic's mention of "discussing with multiple organizations," if a division of labor can be formed among different types of institutions, would be more conducive to cross-validation of evaluation conclusions than exclusive collaboration with a single institution.

Questions Yet to Be Answered

The substantive value of this commitment depends on subsequent details. Which systems and processes exactly does the evaluation team's access cover? Will evaluation results be published in the form of public reports? If evaluation finds deficiencies in safety practices, is the vendor obligated to disclose or remediate? The answers to these questions determine whether third-party evaluation is a genuine accountability mechanism or a limited information-sharing arrangement.

From an industry trend perspective, frontier AI labs have in recent years faced persistent external questioning about the verifiability of safety commitments. Entrusting part of the evaluation work to external institutions is a common path to respond to such questioning, but the path itself does not constitute an answer. What Anthropic has disclosed this time is a phased action; the actual outputs and impact of the evaluation still need to be observed.

One-line Summary

Anthropic commits to external evaluation of safety practices, opening employee-level access to the METR team to advance the implementation of third-party evaluation mechanisms.

Paragraph Summary

Anthropic officially disclosed that it has committed to having external evaluators review its safety practices and has granted employee-level access to a team under METR, while negotiating similar collaborations with multiple organizations. The article analyzes that employee-level access means external evaluators can deeply engage with internal systems and processes rather than relying solely on public materials provided by the vendor, which is critically important to the credibility of evaluation conclusions. Parallel negotiations with multiple institutions indicate an intent to build a multi-evaluator framework and accumulate comparable methods and standards. However, key details such as the specific boundaries of evaluation access, whether conclusions are made public, and how discovered problems are handled have not been disclosed, and whether third-party evaluation is a genuine accountability mechanism or a limited information-sharing arrangement remains to be seen.