PHANTOM-Bench
A benchmark for quantifying the phenomenon of phantom consensus in multi-agent LLM systems; through a staged design it distinguishes final agreement from the truthfulness of that agreement, and evaluates the degree to which social signals distort collective decision-making.
Timeline (last 90 days)
The researchers introduced PHANTOM-Bench, a benchmark specifically designed to quantify phantom consensus.
PHANTOM-Bench decomposes the decision process into multiple independently observable stages: isolated private judgment, public decision after ambiguous social signals, estimation of the group's private judgments, transparency restoration, and control conditions.
The experiments cover nine closed-source and open-source models, and ambiguous public signals persistently degrade decision quality and can turn an originally correct group private judgment into an incorrect public collective outcome.
In the most susceptible setting, public accuracy drops by as much as 47.8 percentage points.
In the most susceptible setting, the collective rationality failure rate reaches 47.3%.
Models make numerous errors when reporting the group's private beliefs, and failures at the public stage coexist with distortion in belief estimation.
Different models differ in their sensitivity to ambiguous signals and explicit peer signals, and phantom consensus is not a single mechanism but a superposition of multiple social reasoning defects.
The study recommends that developers of multi-agent decision systems retain records and visibility of private judgments in the architecture, and include assessment of the accuracy of group belief estimation in the evaluation metrics.
All Facts (8)
The study recommends that developers of multi-agent decision systems retain records and visibility of private judgments in the architecture, and include assessment of the accuracy of group belief estimation in the evaluation metrics.
50%UnverifiedPHANTOM-Bench decomposes the decision process into multiple independently observable stages: isolated private judgment, public decision after ambiguous social signals, estimation of the group's private judgments, transparency restoration, and control conditions.
50%UnverifiedThe experiments cover nine closed-source and open-source models, and ambiguous public signals persistently degrade decision quality and can turn an originally correct group private judgment into an incorrect public collective outcome.
50%UnverifiedIn the most susceptible setting, public accuracy drops by as much as 47.8 percentage points.
50%UnverifiedIn the most susceptible setting, the collective rationality failure rate reaches 47.3%.
50%UnverifiedModels make numerous errors when reporting the group's private beliefs, and failures at the public stage coexist with distortion in belief estimation.
50%UnverifiedDifferent models differ in their sensitivity to ambiguous signals and explicit peer signals, and phantom consensus is not a single mechanism but a superposition of multiple social reasoning defects.
50%UnverifiedThe researchers introduced PHANTOM-Bench, a benchmark specifically designed to quantify phantom consensus.
50%