Solution Length
The number of reasoning steps or the text length required to produce an answer. As a confidence-related signal, it is associated with problem difficulty.
Timeline (last 90 days)
The association strength of perturbation inconsistency, semantic entropy, and solution length with overall difficulty is significantly higher than that of token-level uncertainty.
Even when analyzing only the questions that the model answered correctly, the associations of perturbation inconsistency, semantic entropy, and solution length with overall difficulty remain significant.
All Facts (2)
The association strength of perturbation inconsistency, semantic entropy, and solution length with overall difficulty is significantly higher than that of token-level uncertainty.
50%UnverifiedEven when analyzing only the questions that the model answered correctly, the associations of perturbation inconsistency, semantic entropy, and solution length with overall difficulty remain significant.
50%