Numerical Representation Invariance
An evaluation metric for language models that measures whether a model produces consistent answers across numerically equivalent expressions, such as decimals, fractions, percentages, and scientific notation. It is used to assess whether a model genuinely understands quantities or merely relies on surface-level formatting patterns.
Timeline (last 90 days)
The arXiv paper 'Same Quantity, Different Answer' systematically examines the numerical representation invariance problem in large language models.
The five identity-preserving transformations include decimal notation, fractions, percentages, English number words, scientific notation, and equivalent unit conversions.
Systematic magnitude errors are more insidious and more dangerous than random errors, because the answers appear numerically plausible in structure yet deviate by an order of magnitude.
All Facts (3)
The arXiv paper 'Same Quantity, Different Answer' systematically examines the numerical representation invariance problem in large language models.
50%UnverifiedThe five identity-preserving transformations include decimal notation, fractions, percentages, English number words, scientific notation, and equivalent unit conversions.
50%UnverifiedSystematic magnitude errors are more insidious and more dangerous than random errors, because the answers appear numerically plausible in structure yet deviate by an order of magnitude.
50%