Anthropic Donates AI Alignment Tool Petri to Meridian Labs: A New Chapter for Open-Source Safety Evaluation

Petri Open-Source Donation: Core Event Recap

Anthropic recently announced the official donation of its open-source AI alignment testing tool Petri to Meridian Labs (@meridianlabs_ai), ensuring the tool's development can continue independently. Alongside this transition, the two organizations jointly released a major update with significant improvements across three dimensions: adaptability, realism, and depth.

A leading AI company proactively handing over an internal alignment tool to an independent organization is uncommon in the industry, which is why it has attracted widespread attention in the AI safety community.

What Is Petri? Understanding the AI Alignment Testing Tool

Petri is an open-source tool developed by Anthropic, specifically designed to test and evaluate the alignment performance of AI models.

"Alignment," in simple terms, means ensuring that an AI system's behavior remains consistent with human intentions, values, and safety standards. This is one of the most critical topics in current AI safety research. The alignment problem originated from early thinking about the potential risks of superintelligence, systematically articulated by scholars like Nick Bostrom and Stuart Russell. From a technical perspective, alignment research encompasses multiple subfields: Value Learning studies how to let AI infer true human preferences from behavior; Interpretability research attempts to understand the internal decision-making mechanisms of models; and RLHF (Reinforcement Learning from Human Feedback) is currently the core technical pathway for mainstream large model alignment training—both Claude and the GPT series employ this approach.

Within this technical framework, Petri belongs to the "evaluation" layer—it doesn't train models but systematically verifies whether alignment outcomes meet standards after training is complete, similar to automated testing frameworks in software engineering.

The value of alignment testing tools lies in their ability to systematically verify whether AI models deviate from expected behavior across various scenarios, including:

• Whether they produce harmful outputs
• Whether they correctly understand and follow human instructions
• How they perform in edge cases and extreme scenarios

For large model developers and AI safety researchers, tools like Petri are essential for discovering potential risks and improving model safety.

The Technical Ecosystem of AI Safety Evaluation Tools

The AI safety evaluation tool ecosystem that Petri operates in is rapidly maturing. Several related frameworks are currently developing in parallel: EleutherAI's Language Model Evaluation Harness focuses on capability benchmarking; METR (formerly ARC Evals) focuses on evaluating dangerous capabilities of frontier models; and HarmBench is a red-teaming benchmark specifically targeting harmful content generation. Petri's positioning within this ecosystem leans more toward systematic testing of alignment behavior rather than pure capability assessment.

Notably, the industry has long struggled with the pain point of "benchmarks disconnected from real deployment scenarios"—cases where models perform excellently on standard test sets yet expose alignment issues in real user interactions are all too common. The "realism" dimension emphasized in this Petri update is a direct response to this problem.

Why Anthropic Chose to Donate Rather Than Self-Maintain

Anthropic's decision to donate Petri to Meridian Labs rather than continue maintaining it internally involves several important considerations:

Ensuring Evaluation Independence

An alignment tool maintained by an independent organization carries greater credibility when evaluating AI models from different companies. If the tool remained under Anthropic's control, it would inevitably face questions about being "both player and referee."

Donating internal tools to independent foundations or organizations has well-established precedents in the tech industry: Sun Microsystems donated Java to the Eclipse Foundation, Facebook open-sourced React and handed maintenance to the community, and Google donated Kubernetes to the CNCF (Cloud Native Computing Foundation). The core logic of this model is: the original developer trades exclusive control for broader community participation, higher credibility, and more sustainable long-term maintenance. For AI safety tools, independence is particularly critical—a "safety evaluation tool" solely controlled by a commercial company will naturally face questions about the neutrality of its conclusions, while endorsement from an independent institution effectively resolves this trust issue.

Leveraging Community Power to Accelerate Iteration

Open-source tools operated by dedicated teams often receive more sustained community contributions and faster iteration cycles. As an independent institution focused on AI safety, Meridian Labs is positioned to mobilize broader developer resources.

Driving Industry-Wide Safety Standards

AI alignment is not a problem for any single company—it's a challenge the entire industry must face collectively. Opening up the tool helps establish industry-level AI safety evaluation standards and enables more institutions to participate in alignment research.

Meridian Labs and the Rise of Independent AI Safety Institutions

The independent AI safety research institutions represented by Meridian Labs are a rapidly growing emerging force in the AI governance ecosystem in recent years. These institutions typically operate as non-profits or independent research organizations, not affiliated with any single commercial company, with diversified funding sources (including foundation grants, government contracts, and corporate donations). Similar institutions include: METR, which focuses on frontier AI risk assessment; the AI Now Institute, which researches AI policy and governance; and the Frontier Model Forum, jointly established by multiple leading AI companies.

These institutions fill the gap between commercial companies and government regulators, playing an increasingly important role in developing industry self-regulatory standards and providing independent technical assessments. Anthropic's choice of Meridian Labs as Petri's new custodian also signifies recognition and endorsement of this independent institutional ecosystem.

Petri Major Update: Three Key Upgrades in Adaptability, Realism, and Depth

The update released in collaboration with Meridian Labs focuses on improvements across three key dimensions:

Adaptability

The updated Petri can more flexibly accommodate AI models of different types and scales, significantly lowering the testing barrier. Whether for large language models or small-to-medium specialized models, researchers and developers can more conveniently apply Petri to their projects.

Realism

Test scenarios now more closely mirror real-world usage contexts. Overly idealized tests often fail to expose problems models encounter in actual deployment, and this update—through more realistic scenario design—can more effectively uncover potential alignment risks. This improvement directly addresses the industry's concern about "benchmarks disconnected from real deployment scenarios."