DecArena: A Large-Scale Benchmark for Decentralized Multi-Agent Collaboration

DecArena exposes the scalability and privacy bottlenecks of centralized multi-agent systems, proposing a graph-constrained benchmark and the GraphHarness decentralized collaboration approach.
The article discusses the scalability and privacy issues of centralized multi-agent systems (MAS) that rely on global orchestrators, and introduces the DecArena benchmark: under graph-constrained communication, agents observe only local states and interact only with neighbors, with distributed tool-calling tasks at a tool scale exceeding 100,000. Experiments show that state-of-the-art LLM agents struggle to discover non-local resources and collaborate effectively, and standard RL cannot reliably compensate. The authors propose GraphHarness, combining topology-aware warmup with self-improving LLM-based routing, improving task success rate by 48.5, collaboration quality by 30.6, and reducing cost by 34.0 relative to the baseline. The conclusion is that decentralized MAS require explicit harness mechanisms rather than relying solely on model capability.
Why Do Centralized Multi-Agent Systems Struggle to Scale?
Personal AI agents are rapidly gaining traction, but the vast majority of multi-agent systems (MAS) rely on a global orchestrator to schedule all agents, resources, and communication. This approach exposes two fundamental problems as scale increases: scalability bottlenecks and privacy risks—the orchestrator requires broad access to all agents' states and communication content, so once deployed in real environments, it amounts to concentrating all collaboration data into a single point of failure.
The DecArena work shifts the perspective to decentralized environments: with no global visibility, agents can only see their own local state and can only communicate with neighbors. This is closer to how collaboration works in human organizational structures—you don't need to know everyone in the company, but you need to know how to find the people who can help you solve your problem.

The DecArena Benchmark: Distributed Tool Calling Under Graph Constraints
The paper's core premise is graph-constrained communication: each agent observes only its local state and can interact only with its neighbors. The task takes the form of synthetic but scalable distributed tool calling, with a tool scale of over 100,000. This means no single agent can directly see the global tool library; it must rely on proactive discovery to find non-local tools and skills, then collaborate with other agents to complete the task.
This setup captures the key difficulty of decentralized MAS: resources are out of view, collaboration paths do not pre-exist, and agents must simultaneously solve two problems—"finding resources" and "forming teams."
Experimental Findings: Existing LLM Agents Clearly Struggle
Experimental results show that current state-of-the-art LLM agents struggle to discover relevant non-local resources and to collaborate effectively under decentralized settings. More notably, standard reinforcement learning (RL) methods cannot reliably overcome this limitation. The authors conclude that even with SOTA large models, decentralized MAS require more explicit harness mechanisms to support effective collaboration, rather than expecting collaboration capabilities to "emerge" from the model on its own.
Graph-constrained communication can be formalized as follows: the set of agents forms a graph $G=(V,E)$, where each agent $i$'s observation at time $t$ is its local state $s_i^t$ plus neighbor messages, and the action space is limited to sending requests or tool calls to neighbors in $N(i)$. In contrast, a centralized orchestrator is equivalent to a fully connected graph plus a global state aggregator. The core constraint of graph constraints lies in the limited communication radius: the collaboration path length between any two agents is bounded by the graph diameter, and each step along the path introduces information decay and latency. Therefore, whether a task can be completed depends not only on an individual agent's reasoning ability but also on whether the graph topology matches the task dependency structure—this is precisely the motivation for the topology-aware warmup introduced by GraphHarness.
Proactive discovery is fundamentally different from traditional tool retrieval. In a centralized setting, a retriever can scan the global tool library and return top-k candidates; under graph constraints, an agent can only see its local tool set and must extend its visibility hop by hop by sending probe requests to neighbors, with each probe consuming communication budget and tokens. This reduces the problem to a partially observable exploration-exploitation trade-off: converging too early on local neighbors may miss the optimal tool, while over-exploring drives up costs. The reason standard RL struggles to be effective in the paper is largely that the reward signal is sparse and delayed—successful invocation of non-local tools is often only visible after multi-hop collaboration, and credit assignment spans multiple agents and multiple time steps, making stable estimation difficult for conventional policy gradients.
GraphHarness: Topology-Aware Warmup + Self-Improving Routing
To address these issues, the authors propose a strong baseline, GraphHarness, which combines two mechanisms:
- Topology-aware warmup: enables agents to build awareness of the graph structure in the early stage, reducing blind exploration.
- Self-improving LLM-based routing: the model itself iteratively optimizes routing decisions for requests, directing tasks to more suitable neighbors.
Compared with the baseline, GraphHarness improves task success rate by +48.5, collaboration quality by +30.6, while reducing cost by +34.0. These numbers suggest that the bottleneck in decentralized collaboration may not lie in model capability itself, but rather in the lack of suitable collaboration scaffolding—explicitly injecting structural information and replacing global orchestration with a routing mechanism can significantly change the outcome.
Self-improving routing means that routing decisions are generated by the LLM itself and iteratively optimized with interaction feedback, rather than being hard-coded rules or static hashing. A typical implementation has the model output the next-hop choice based on the current local state, historical routing success rates, and neighbor capability descriptions; after each task round, the successful/failed paths are fed back to the model as context, allowing it to adjust routing preferences in subsequent rounds. The fundamental difference from a centralized orchestrator is that routing is distributed and locally informed: each agent maintains only its own view of the routing policy, and there is no global routing table. Topology-aware warmup is equivalent to injecting a graph structure prior during the cold-start phase of the routing policy, preventing the model from uniformly randomly probing neighbors early on due to lack of structural information, thereby reducing communication overhead during the warmup period.
Implications for Multi-Agent System Design
The value of this work is not just a benchmark. It points to a design orientation: as the number of agents and the degree of personalization increase, the cost of a centralized orchestrator will become increasingly high, and decentralized collaboration cannot simply be left to LLMs to improvise. An explicit harness layer—including topology awareness, routing policies, and discovery mechanisms—may be a necessary condition for making decentralized MAS truly usable. For teams building multi-agent applications, this means treating "collaboration infrastructure" as a first-class citizen in design, rather than as glue code attached to model calls.
FAQ
What does DecArena evaluate?
It evaluates large-scale decentralized multi-agent collaboration under graph-constrained communication, with tasks in the form of distributed tool calling and a tool scale exceeding 100,000.
How large are GraphHarness's improvements?
Relative to the baseline: task success rate +48.5, collaboration quality +30.6, cost +34.0 (the cost item is the improvement magnitude reported in the paper).
Why is standard reinforcement learning insufficient?
The paper's experiments show that RL cannot reliably overcome the resource discovery and collaboration challenges in decentralized environments, and explicit harness mechanisms are needed.
One-line Summary
DecArena reveals the scalability and privacy bottlenecks of centralized multi-agent systems and proposes a graph-constrained benchmark and the GraphHarness decentralized collaboration solution.
Paragraph Summary
The article discusses the scalability and privacy issues caused by centralized multi-agent systems (MAS) relying on global orchestrators, and introduces the DecArena benchmark: under graph-constrained communication, agents observe only local states and can interact only with neighbors, with tasks being distributed tool calling at a tool scale exceeding 100,000. Experiments show that existing state-of-the-art LLM agents struggle to discover non-local resources and collaborate effectively, and standard reinforcement learning cannot reliably compensate. The authors propose GraphHarness, combining topology-aware warmup with self-improving LLM-based routing, achieving a task success rate improvement of 48.5, collaboration quality improvement of 30.6, and cost reduction of 34.0 relative to the baseline. The conclusion points out that decentralized MAS require explicit harness mechanisms rather than relying solely on model capability.
Related articles

AI Writes Your Code—Do You Still Need to Learn Git? Five Reasons
AI coding agents handle Git operations automatically—is it still necessary to learn Git? Five reasons: team communication, reviewing agent PRs, monitoring code changes, fallback when AI goes down, and the joy of learning—plus how the focus of learning Git is shifting in the agent era.

Anthropic Reportedly Prices IPO at $83 Per Share, Valuing Company at $1.5 Trillion
Anthropic reportedly prices IPO at $83 per share, above the expected range, with a fully diluted valuation of $1.5 trillion. This article analyzes the signal significance of the pricing and its impact on the AI industry's financing environment.

Open-Source Code Hosting Platform Codeberg Suffers Outage, Sparking Debate in Developer Community
Open-source code hosting platform Codeberg.org suffered an outage, sparking Hacker News debate over nonprofit platform stability and open-source infrastructure dependency risks.