[KongchangAI]
BenchmarkPHANTOM-Bench

PHANTOM-Bench

A benchmark for quantifying the phenomenon of phantom consensus in multi-agent LLM systems; through a staged design it distinguishes final agreement from the truthfulness of that agreement, and evaluates the degree to which social signals distort collective decision-making.

Timeline (last 90 days)

Nov 9

The researchers introduced PHANTOM-Bench, a benchmark specifically designed to quantify phantom consensus.

Unverified50%
Nov 9

PHANTOM-Bench decomposes the decision process into multiple independently observable stages: isolated private judgment, public decision after ambiguous social signals, estimation of the group's private judgments, transparency restoration, and control conditions.

Unverified50%
Nov 9

The experiments cover nine closed-source and open-source models, and ambiguous public signals persistently degrade decision quality and can turn an originally correct group private judgment into an incorrect public collective outcome.

Unverified50%
Nov 9

In the most susceptible setting, public accuracy drops by as much as 47.8 percentage points.

Unverified50%
Nov 9

In the most susceptible setting, the collective rationality failure rate reaches 47.3%.

Unverified50%
Nov 9

Models make numerous errors when reporting the group's private beliefs, and failures at the public stage coexist with distortion in belief estimation.

Unverified50%
Nov 9

Different models differ in their sensitivity to ambiguous signals and explicit peer signals, and phantom consensus is not a single mechanism but a superposition of multiple social reasoning defects.

Unverified50%
Nov 9

The study recommends that developers of multi-agent decision systems retain records and visibility of private judgments in the architecture, and include assessment of the accuracy of group belief estimation in the evaluation metrics.

Unverified50%

All Facts (8)

Source Articles