AI Extinction Risk and Bioweapon Threats: Real or Overhyped?

Could AI Actually Destroy Humanity?

Debate over whether artificial intelligence poses an existential threat to humanity has been heating up. MIT Technology Review recently hosted an online roundtable that tackled the question on many people's minds: could AI actually end human civilization?

These concerns aren't without basis. As large language models have made rapid capability gains, a growing number of researchers and policymakers have begun taking so-called "existential risk" seriously. These discussions are no longer confined to science fiction — they've entered mainstream tech media and public policy conversations.

Two Perspectives on Existential Risk

The debate around AI extinction risk typically falls into two opposing camps. One side argues that as AI systems grow more capable and increasingly autonomous, the absence of effective alignment and oversight could eventually make them impossible to control. The other side contends that this kind of "doomsday narrative" tends to be overblown, distracting attention from the real, present-day harms AI is already causing — bias, privacy violations, and misinformation.

MIT Technology Review's roundtable aimed to address these questions for the public, combining expert perspectives with reader concerns to foster a rational rather than emotionally charged discussion. This kind of open dialogue, led by a credible tech publication, helps cut through extreme narratives and build a more balanced understanding among general audiences.

Alignment is a core technical concept in AI safety — it refers to the research effort to ensure that an AI system's goals, behaviors, and values remain consistent with human intentions. Current large language models are trained using techniques like reinforcement learning from human feedback (RLHF), but as model capabilities grow, researchers worry that models may behave in unexpected ways in scenarios they weren't explicitly designed for. Existential risk proponents argue that if a future artificial general intelligence (AGI) far exceeding human capabilities were to have even a slight misalignment between its objective function and human well-being, the consequences could be irreversible. This is the backdrop behind why organizations like OpenAI and DeepMind have established dedicated alignment research teams — and why the topic remains a focal point of debate across academia and industry.

Bioweapons: A More Concrete Concern

Compared to the abstract proposition of "AI wiping out humanity," the risk of AI being misused to develop bioweapons feels considerably more tangible and immediate. As AI applications in life sciences, protein design, and chemical synthesis continue to expand, there are growing fears that these tools could lower the technical barrier to creating biological threats — making dangerous work that once required specialized teams far more accessible.

This issue is one of the focal points in current AI safety discussions. It bridges two critical domains: the proliferation of AI capabilities and the protection of biosecurity. For regulators, figuring out how to advance scientific progress while guarding against potential malicious misuse has become an urgent and complex challenge.

AI's penetration into the life sciences is perhaps best exemplified by the protein structure prediction breakthrough — DeepMind's AlphaFold2 essentially solved the protein folding problem in 2021, a challenge that had stumped biologists for fifty years, dramatically accelerating drug discovery. Yet the same capabilities could be used to design gain-of-function mutations in pathogens. In 2022, researchers at Carnegie Mellon and other institutions found that certain large language models, when unconstrained, could provide detailed instructions for synthesizing dangerous compounds — raising widespread alarm in the biosecurity community. Both the U.S. Biosecurity Commission and the UK government have designated "AI-assisted biological threats" as an emerging risk category requiring priority assessment, and the topic featured prominently on the agenda at the 2023 Bletchley Park AI Safety Summit in the UK.

How to Think Rationally About AI Risk

In the face of relentless discussion about AI risk, what the public needs is fact-based judgment — not to be swept along by extreme narratives. Existential risk and present-day harms are not an either/or choice; they represent different points on the AI safety spectrum, and both deserve serious attention.

Open public forums organized by credible institutions are an important way to bridge the gap between expert consensus and public understanding. By translating complex technical issues into accessible public discourse, we can avoid falling into the twin traps of blind panic and blind optimism, and instead drive meaningful governance and research.

As AI capabilities continue to evolve, the debate over where its risk boundaries lie will only intensify. Maintaining an open yet cautious perspective may be the most rational stance available to us right now.