Why AI Safety Testing Falls Short: The Urgent Need for Unified Evaluation Standards

Current AI evaluation frameworks still have obvious shortcomings in maturity and standardization. Irene Solaiman of Hugging Face points out that the industry is shifting from simple model-level benchmarks to agentic system evaluation, but the lack of consensus on testing protocols makes results hard to reproduce. This view reveals a core contradiction in the AI safety field: model capabilities are iterating rapidly, while evaluation methods have yet to catch up.

The Dilemma of Comparing Results: Everyone Speaks Their Own Language

The industry has yet to establish a unified standard for comparing different test results. Each company follows its own evaluation methodology, making horizontal comparisons across models and systems difficult. More critically, test repeatability—running the same method and getting the same results—remains one of the most contentious issues.

According to a report by Hacks.gr, Solaiman describes the current situation as a dual struggle with maturity and standardization of evaluation frameworks. The lack of consensus not only undermines research credibility but also makes it difficult for regulators and the public to judge whether a given AI system is truly safe. For high-risk scenarios that require rigorous safety arguments, this uncertainty is especially damaging.

Benchmarks and Reproducibility

Benchmarks are the most common form of AI evaluation, referring to measuring a model's performance on specific tasks using fixed datasets and scoring rules, such as MMLU and GSM8K. Their value lies in providing quantifiable horizontal reference, but only on the premise that test protocols, prompt templates, decoding parameters, and scoring scripts are fully disclosed and consistent. In reality, organizations often apply different preprocessing to the same benchmark, and "benchmark contamination"—where test questions leak into training data and inflate scores—even occurs. Reproducibility thus becomes the bottom line for testing evaluation credibility: if a third party cannot reproduce the results using the same method, the scores lose their comparative meaning. This is also the deeper reason why Solaiman emphasizes the lack of consensus on testing protocols.

The Controversy Over External Audits: Who Audits and How

Amid ongoing safety concerns, placing external reviewers inside AI companies has gradually become popular. Anthropic chose the consulting firm Accenture to take on this role, but the decision drew criticism in the tech industry—some questioned whether Accenture is suitable for AI safety review work.

Behind the controversy is a deeper issue: the effectiveness of external audits depends on the auditor's depth of understanding of AI systems. If auditing institutions lack sufficient technical capability, reviews may become mere formalities. Solaiman's argument implies that what the industry needs is not just "someone to audit" but a set of operational and verifiable audit standards.

AI External Audits and Third-Party Evaluation

External audits refer to reviews of an AI system's safety, compliance, and governance processes by third-party institutions independent of the developer, including code review, red-team testing, document verification, and continuous monitoring. Unlike financial audits, AI audits have no generally accepted standards yet, and the auditor's qualifications, degree of methodology disclosure, and boundaries of responsibility remain undefined. The advantage of involving consulting firms is process management and compliance experience; the shortcoming is limited understanding of frontier issues such as internal model mechanisms and adversarial attacks. Therefore, the substance of the controversy is not "whether to audit" but how audit capability is certified and how audit conclusions are accepted by regulators.

From Model to System: The Shift in Evaluation Targets

Solaiman emphasizes that the direction of auditing needs to change: examining only the AI model itself is no longer enough. Evaluation is shifting toward examining the entire system, including the tools and software that support its operation. The reason is that a less capable model equipped with a more effective support system may outperform a more capable model with insufficient support.

This judgment has direct implications for safety evaluation. The attack surface is no longer limited to model weights and inference interfaces but extends to the entire technology stack. However, system-level testing is considered costly and difficult to implement. Solaiman states that the industry needs to build more consensus on rules and evaluation standards before these tests can deliver substantive value for system safety and reliability.

Agentic Systems and System-Level Evaluation

An agentic system refers to a software system driven by large models that can autonomously plan, invoke tools, execute multi-step tasks, and interact with the environment, such as agents equipped with browsers, code interpreters, and memory modules. Shifting the evaluation target from models to systems means examining whether tool calls are correct, whether permission boundaries are breached, whether errors accumulate across multi-step tasks, and whether external dependencies introduce new attack surfaces. System-level evaluation typically requires building simulated environments and end-to-end scenarios, costing far more than static question-answering benchmarks, and results are heavily influenced by orchestration logic, making standardization difficult. This is exactly the crux of why system-level testing struggles to deliver substantive safety value when the industry underinvests in rule consensus.

The Cost of Missing Standardization

Taken together, the core problems of current AI safety testing can be summed up in three points: inconsistent comparison standards, questionable audit mechanisms, and prohibitively high costs of system-level evaluation. These three are intertwined—without unified standards, audit results lack comparability; without comparability, investment in system-level testing is hard to recoup.

Solaiman's position is clear: the industry needs more standardization. This is not a patch for technical details but a necessary step for the evaluation system to move from "fragmented" to "interoperable." As AI systems grow increasingly complex and application scenarios continue to expand, the maturity of evaluation standards will directly determine the credibility of safety commitments.

One-line Summary

AI safety evaluation lacks unified standards and needs to shift from models to system-level auditing.

Paragraph Summary

The article focuses on the maturity and standardization of AI safety testing and evaluation. Irene Solaiman of Hugging Face points out that the industry is shifting from model-level benchmarks to agentic system evaluation, but testing protocols lack consensus, making results hard to reproduce and compare horizontally. External audit mechanisms are also questionable; Anthropic's hiring of Accenture for review sparked doubts, reflecting the contradiction between auditors' technical capabilities and the lack of standards. The evaluation target is expanding from a single model to the entire system, including tools and software support, because a weaker model with a more effective support system may perform better overall—but this expands the attack surface and drives up testing costs. The article identifies three major problems: inconsistent comparison standards, questionable audit mechanisms, and prohibitively high system-level evaluation costs, arguing that standardization is the precondition for moving from fragmentation to interoperability.