heterodyne
An open-source quantization analysis tool embedded within llama.cpp that compares output distribution differences across quantization configurations token by token and layer by layer. It supports numerical baseline comparisons and reproducible experiment records.
Timeline (last 90 days)
heterodyne is a tool inside llama.cpp that compares two configurations token by token and layer by layer to locate the source of quantization drift.
heterodyne calls each configuration an "arm", i.e., a GGUF file plus runtime settings that may change its numerics (K/V cache type, flash attention, ubatch size).
heterodyne's numerical noise floor is estimated by re-running arm A with odd batch splits, representing harmless floating-point computation-order differences on the same machine.
heterodyne uses llama.cpp scheduler's cb_eval callback to copy or overwrite tensors when computation reaches a specified graph node, achieving activation transplanting.
This attention-versus-MLP contribution decomposition requires two extra rounds of computation per layer per block, and 12 blocks took 29 minutes on an i7-1355U laptop.
All Facts (5)
heterodyne is a tool inside llama.cpp that compares two configurations token by token and layer by layer to locate the source of quantization drift.
50%Unverifiedheterodyne calls each configuration an "arm", i.e., a GGUF file plus runtime settings that may change its numerics (K/V cache type, flash attention, ubatch size).
50%Unverifiedheterodyne's numerical noise floor is estimated by re-running arm A with odd batch splits, representing harmless floating-point computation-order differences on the same machine.
50%Unverifiedheterodyne uses llama.cpp scheduler's cb_eval callback to copy or overwrite tensors when computation reaches a specified graph node, achieving activation transplanting.
50%UnverifiedThis attention-versus-MLP contribution decomposition requires two extra rounds of computation per layer per block, and 12 blocks took 29 minutes on an i7-1355U laptop.
50%