[KongchangAI]
Conceptposition bias

Position Bias

A systematic bias in which an LLM judge, in pairwise comparison, tends to select the candidate answer in a fixed position regardless of content quality; swapping the positions of the options can reverse the judgment.

Timeline (last 90 days)

Nov 5

In language model evaluation, position bias typically manifests as a preference for a fixed position (such as first or second), and this preference is unrelated to the text content.

Unverified50%
Nov 5

A common method for identifying position bias is to swap the order of candidates and observe whether the choice flips; if the flip rate is significantly above the random level, position sensitivity is indicated.

Unverified50%
Oct 15

Position bias means the judge model's judgment of information in the context is affected by where that information appears: content near the end is often given higher weight, while key warnings that appear early are diluted in long conversations.

Unverified50%

All Facts (3)

Source Articles