Floating-Point Arithmetic
A method of computation in computers for representing approximate values of real numbers, following the IEEE 754 standard. It is subject to issues such as rounding errors and non-determinism.
Timeline (last 90 days)
Rounding errors in floating-point arithmetic and the nondeterminism of parallel execution are classic difficulties for formal verification.
Transformations common in compiler optimization, such as reassociation and constant folding, are safe in the integer domain but may change numerical results in the floating-point domain.
IEEE 754 floating-point numbers do not satisfy the associative and distributive laws of the real numbers, and rounding errors accumulate and propagate.
The randomness of LLM inference comes from at least three independent sources: the sampling strategy (temperature, top-p), the accumulated error of floating-point operations, and changes in the reduction order of matrix multiplications caused by changes in batch size.
All Facts (4)
Rounding errors in floating-point arithmetic and the nondeterminism of parallel execution are classic difficulties for formal verification.
50%UnverifiedTransformations common in compiler optimization, such as reassociation and constant folding, are safe in the integer domain but may change numerical results in the floating-point domain.
50%UnverifiedIEEE 754 floating-point numbers do not satisfy the associative and distributive laws of the real numbers, and rounding errors accumulate and propagate.
50%UnverifiedThe randomness of LLM inference comes from at least three independent sources: the sampling strategy (temperature, top-p), the accumulated error of floating-point operations, and changes in the reduction order of matrix multiplications caused by changes in batch size.
50%