[KongchangAI]
Conceptperturbation inconsistency

Perturbation Inconsistency

An uncertainty measurement method that applies slight semantics-preserving perturbations to the input and observes the stability of model outputs, reflecting the model's behavioral stability within a neighborhood of the input space.

Timeline (last 90 days)

Nov 17

The association strength of perturbation inconsistency, semantic entropy, and solution length with overall difficulty is significantly higher than that of token-level uncertainty.

Unverified50%
Nov 17

Even when analyzing only the questions that the model answered correctly, the associations of perturbation inconsistency, semantic entropy, and solution length with overall difficulty remain significant.

Unverified50%
Nov 17

Perturbation inconsistency measures uncertainty by applying slight, semantics-preserving perturbations to the input (such as paraphrasing, adding or removing irrelevant words, and adjusting word order) and observing whether the model's output remains stable.

Unverified50%
Nov 17

Perturbation inconsistency aligns with findings in model robustness research.

Unverified50%

All Facts (4)

Source Articles