The Battle Over AI Standards: Why People Outside the Labs Deserve a Voice

AI Development Shouldn't Be Left to the Labs Alone

Artificial intelligence is reshaping society at an unprecedented pace — yet the power to determine its direction remains concentrated in the hands of a small number of AI labs. A perspective that sparked debate on Twitter cuts right to the heart of this problem: people outside AI labs should have genuine input into how this technology develops, and there needs to be a clear way to assess whether that development is proceeding safely.

The demand sounds straightforward, but it strikes at the most sensitive nerve in AI governance today — who gets to define safety? Who is qualified to oversee it? As technical capabilities grow stronger while decision-making becomes increasingly closed off, the foundation of public trust is being quietly eroded.

The Core Value of Standards: Preventing Power Concentration

The proposal emphasizes that standards in the AI space should help prevent the concentration of power. This point is especially critical given the current industry landscape. Leading labs have built near-impenetrable moats through their advantages in compute, data, and capital, while emerging companies and open-source model developers are at a clear competitive disadvantage.

The proposal explicitly states that good standards should ensure new companies and open-source model companies can compete. The underlying logic is straightforward: if safety standards are designed as hurdles that only giants can clear, then "safety" risks becoming a tool to squeeze out competitors, ultimately leading to market monopolization. A genuinely healthy standards framework should preserve the openness of competition while safeguarding safety.

The Situation for Open Source

Open-source models have long been an important force for breaking technological monopoly. They lower the barriers to innovation, enabling more researchers and developers to participate in cutting-edge exploration. Yet open-source models are often the first targets when safety regulation tightens. How to strike a balance between oversight and openness is a challenge that any standards-setting process cannot sidestep.

In practice, open-source models face a core tension: closed-source labs can intervene in model behavior after deployment through access controls, terms of service, and API rate-limiting — while open-weights models, once released, can be run locally by anyone with safety guardrails removed. This leads some regulators to equate "controllability" with "closed-source." Meta's LLaMA series, Mistral, and others have all triggered regulatory controversy for this reason. It's worth noting that even the definition of "open source" in AI is contested — releasing model weights alone is fundamentally different from releasing full training data, code, and pipelines. If future safety standards treat "revocability" or "centralized monitoring" as prerequisites for compliance, the open-source ecosystem will face systemic exclusion. This is precisely the real-world context behind the proposal's insistence that standards should not create unfair barriers to entry.

Making Evidence Comparable, Making Failures Instructive

The proposal also puts forward a pragmatic goal: standards should help countries and companies compare evidence and learn from failures.

This point is often overlooked, but it's enormously valuable. Today, every lab uses its own methods to evaluate model safety, with no unified measurement framework — making it nearly impossible for outsiders to judge the relative safety levels of different models. A shared set of standards would make evaluation results comparable, and allow the industry as a whole to accumulate collective lessons from incidents and failures, rather than each organization working in isolation and repeating the same mistakes.

Learning from failure is a hallmark of any mature engineering discipline. High-risk industries like aviation and pharmaceuticals have built rigorous incident-reporting and data-sharing mechanisms. AI, as a technology with far-reaching consequences, equally needs this kind of transparency and collaborative culture.

The AI safety evaluation field currently suffers from severe "benchmark fragmentation." Labs design their own red-teaming exercises and publish their own safety reports, but the methodologies, test sets, and scoring criteria vary widely — making cross-lab comparisons by external parties nearly impossible. The aviation parallel is particularly instructive: the International Civil Aviation Organization (ICAO) mandates that member states share accident and incident data, with all major failures required to enter an international database, creating a cross-border collective learning mechanism. The pharmaceutical industry's mandatory adverse-event reporting follows the same logic. AI currently lacks an equivalent mandatory disclosure mechanism — unexpected capability emergence, the spread of jailbreaking techniques at scale, and systemic biases appearing in deployment are typically exposed first by external researchers rather than the labs themselves. If unified standards incorporated "failure reporting" as a compliance requirement, it would fundamentally transform how information flows across the industry.

A U.S.-Led Governance Path

The proposal takes a clear position: the United States should lead this standards-setting effort. This reflects the geopolitical reality of AI governance — technical standards are also carriers of international influence and agenda-setting power.

Whoever leads standards development largely defines the rules of the game. For the U.S., establishing widely accepted AI safety standards first would both consolidate its technological leadership and shape the global governance framework. Of course, this raises another layer of questions: can standards led by a single country genuinely represent the world's diverse interests? That remains an open question.

The geopolitics of technical standards is nothing new. The 5G standards battle, semiconductor export controls, and competition for seats in organizations like ISO/IEC are all classic examples of technology and politics intertwining. In AI governance, the EU has already established a risk-tiered regulatory framework through the AI Act, and is leveraging the "Brussels Effect" — whereby regulations from large markets naturally spread through global supply chains — to influence global practice. China, meanwhile, is actively participating in rule-making through domestic regulations like the Interim Measures for the Management of Generative AI Services. If the U.S. were to lead an independent standards regime, it would effectively mean global AI governance moving toward "standards fragmentation," with companies facing multiple overlapping compliance requirements. The tension between "U.S. leadership" and "multilateral coordination" is therefore an unavoidable structural problem for this proposal to navigate.

The Real-World Significance of Transparent Governance

The overall thrust of this proposal points toward a vision of AI governance that is more transparent, more open, and more accountable. It attempts to answer several key questions: who decides the direction of technological development, how safety is externally verified, how competitive dynamics stay healthy, and how lessons learned get shared.

For ordinary people and small businesses, such a governance framework means more participation and more right to be informed. For the industry as a whole, it may be an essential safeguard against winner-take-all dynamics and a means of sustaining long-term innovation.

However, there is still a long road from vision to implementation. The specific content of standards, enforcement mechanisms, oversight bodies, and how to balance safety with innovation all require extensive negotiation in practice. This Twitter proposal reads more like the beginning of a conversation than a final answer. It reminds us that the future of AI should not be treated as a purely technical question — it is a public matter that requires broad participation from society as a whole.