When Millions of AI Agents Meet: A Deep Dive into Distributed Intelligence Collaboration and Security Challenges

From Language Models to AI Agents

Not long ago, AI assistants were essentially just large language models—you asked them questions, they gave you answers, but they couldn't execute tasks on your behalf. With the arrival of the AI Agent era, this is undergoing a fundamental transformation.

In the latest episode of Google DeepMind's podcast, senior research scientist Nenad Tomashev explored a critical question: when millions of AI Agents not only work for us but also trade, negotiate, and delegate tasks to each other, what happens? Does this signal the birth of a new type of economy, a new path toward AGI? And how do we ensure all of this remains safe?

The Essential Difference Between Agents and Large Language Models

Agents aren't a new concept. Long before large language models appeared, researchers were already training agents to collect items and complete tasks in simulated 3D environments. But today's AI Agents are fundamentally different from their predecessors.

Nenad points out that the core distinction is: a large language model simply gives you a response, while an Agent observes the state of the world and takes action. Although modern Agents still use large language models under the hood to formulate action plans, they're wrapped in an execution framework that can chain decisions together and autonomously complete multi-step tasks.

Take planning a wedding as an example: a large language model will give you a list of catering vendors and venue suggestions, but all the email communication still falls on you. An AI Agent, on the other hand, can be authorized to access your Gmail, automatically send emails, book venues, and contact vendors—ideally, you wouldn't need to lift a finger.

Of course, "ideally" is a very important qualifier. Every action an Agent takes isn't 100% accurate, and the more complex the task, the higher the expected failure rate. Even more dangerous is automation bias: after an Agent performs well several times in a row, humans unconsciously let their guard down and stop verifying, which is when errors slip through unnoticed.

AI Agents Accelerating Scientific Discovery: The Future of Automated Labs

For Nenad, the most exciting application of AI Agents is accelerating scientific discovery. Currently, many scientists are already using language models as co-ideators or tools for assisting with formal derivations. But true scientific automation requires far more than that.

Investments are already being made in developing autonomous research laboratories. In these scenarios, Agents need to be able to schedule experiments, analyze results, and adjust plans based on feedback. Unlike software development where unit tests can verify code, scientific research requires running physical experiments to obtain feedback—making the "closed loop" far more complex and dangerous.

Even for seemingly safe tasks like designing batteries, an Agent might design an experimental protocol that causes overheating, resulting in hardware damage. Therefore, reliable safety protocols are indispensable.

However, Nenad also acknowledges that current AI systems have clear limitations in the scientific domain: what they excel at is combinatorial closure over existing knowledge—replicating skills humans already possess, recombining them, and bridging relatively small gaps. But we haven't yet seen these models make truly profound, paradigm-shifting scientific discoveries.

Multi-Agent Delegation and Intelligent Division of Labor

When tasks are sufficiently complex, a single Agent may not be able to independently complete all components. This introduces the concept of inter-Agent delegation: one Agent hands off part of the work to another Agent through established Agent-to-Agent protocols.

Nenad points out that many current multi-Agent systems are more about parallelization rather than true delegation—tasks are randomly split up, with each Agent completing their portion independently, lacking an intelligent division-of-labor framework. This could lead to one Agent buying wine while another buys cups, without either realizing what's actually needed is wine glasses, with no communication between them.

True intelligent delegation requires solving several key problems:

• Reliability assessment: Which Agents are worth delegating to? Can their capabilities be certified?
• Failure management: The delegating party needs to anticipate and handle various possible failure scenarios
• Verifiability: A formal "contract" needs to be established between the delegator and the delegatee
• Reversibility judgment: Distinguishing between reversible tasks (just re-execute) and irreversible tasks (like spending money on purchases), investing more caution in the latter

An interesting reversal is that in some scenarios, AI delegates tasks to humans. In medical imaging, when an AI system is uncertain about a diagnosis, it flags the case and passes it to a human radiologist for review. This AI-human team collaboration model has proven highly effective.

Agent Traps and Cybersecurity: Analyzing New Attack Surfaces

As more and more AI Agents operate on the internet, malicious actors are setting up various Agentic Traps.

These traps take many forms:

Hidden Prompt Injection: Web pages contain hidden instructions that are invisible to humans but readable by Agents, altering the Agent's goals and behavior. Since Agents typically consume the raw format of a page rather than its visual rendering, these hidden tokens may be executed unknowingly.

Dynamic Cloaking: Malicious websites display completely different page content depending on whether the visitor is a human or an Agent. By analyzing behavioral patterns on the page, they can precisely determine the visitor's identity and then inject targeted misleading content.

Wallet Attacks: There have already been cases where experimenters had funds stolen after authorizing Agents with wallet access.

Nenad emphasizes that Agents' use of the web may have already surpassed that of humans—this is a first in history. This means malicious actors have a larger attack surface and stronger motivation.

The counterstrategy is Defense in Depth: no single solution can address all problems. Instead, layered mitigation measures are needed—authenticating web content, establishing resource trust mechanisms, Agent-side protections, underlying model safety, meaningful human review, the principle of least privilege, and more.

Cognitive Monoculture and Systemic Risks of the Agent Economy

A particularly concerning issue is Cognitive Monoculture. Currently, there are only a handful of mainstream large language models on the market (Claude, ChatGPT, Gemini, etc.), and they tend to make similar decisions.

When hundreds of thousands or even millions of AI decision-makers are suddenly deployed, and they tend to make similar decisions, failure points become highly correlated. This is analogous to systemic risk in financial markets—high-frequency trading algorithms have caused flash crashes, and the Agent economy faces similar or even greater risks.

Even more concerning is that Agents might coordinate indirectly through the environment without direct communication, forming a kind of implicit "collusion." This requires us to develop anti-collusion measures to guard against such systemic threats.

Distributed Intelligence: An Alternative Path to AGI

At the end of the interview, Nenad offered a profound perspective: perhaps the future of AGI isn't a single omnipotent super-model, but rather a society of experts.

He uses chess as an example: Gemini can play chess, but a dedicated chess engine is faster, more accurate, and cheaper because it focuses on only one thing. Human society works the same way—no single person can master everything; we achieve collective intelligence through division of labor and collaboration.

This means the future AI architecture might be: a general-purpose "connective tissue" layer responsible for coordination and orchestration, while specific tasks are executed by certified expert Agents. Each expert Agent is cheaper, more reliable, and more efficient.

This distributed intelligence architecture poses entirely new challenges for safety and alignment. Current alignment methods primarily target individual models, but when 10,000 Agents interact in complex ways, how do you align the entire system? Nenad believes that economic incentive mechanisms might be a starting point—just as human society coordinates behavior through economic incentives.

Replicating Not Human Intelligence but Human Civilization

As host Hannah Frost summarized at the end of the show: perhaps replicating human-level intelligence isn't the ultimate goal. The real direction is replicating civilization-level intelligence—an Agent society composed of specialists and generalists, delegators and executors.

This vision is both exciting and unsettling. Exciting because of the scientific breakthroughs and efficiency leaps it could bring; unsettling because the pace of this transformation far exceeds any previous industrial revolution, and the window of time we have to adapt is shrinking rapidly. As Nenad put it: "We need to be very, very careful about everything."