KER
Keyword Error Rate (KER), an ASR evaluation metric in which each keyword occurrence is an independent evaluation unit and a keyword counts as retained only when the entire phrase is correct; it quantifies whole-phrase retention capability.
Timeline (last 90 days)
KeyASRBench proposes an occurrence-based Keyword Error Rate (KER), computed on frozen whole-phrase annotations, treating each keyword occurrence as an independent evaluation unit and counting it as preserved only if the entire phrase is completely correct.
In the KeyASRBench evaluation, the language-specific MER/WER range is 5.31% to 12.91%, while the KER range for the same set of systems is 13.61% to 33.81%.
On the 30-minute control set, KeyASRBench's KER range is 10.65% to 27.31%, lower than the challenge set but still significant in absolute terms, demonstrating that keyword errors are prevalent in real transcriptions.
KeyASRBench currently covers Chinese and English and a fixed keyword set, and its control set is 30 minutes in size; its value lies in proposing a reproducible KER framework rather than providing a universally applicable industry threshold.
All Facts (4)
KeyASRBench proposes an occurrence-based Keyword Error Rate (KER), computed on frozen whole-phrase annotations, treating each keyword occurrence as an independent evaluation unit and counting it as preserved only if the entire phrase is completely correct.
50%UnverifiedIn the KeyASRBench evaluation, the language-specific MER/WER range is 5.31% to 12.91%, while the KER range for the same set of systems is 13.61% to 33.81%.
50%UnverifiedOn the 30-minute control set, KeyASRBench's KER range is 10.65% to 27.31%, lower than the challenge set but still significant in absolute terms, demonstrating that keyword errors are prevalent in real transcriptions.
50%UnverifiedKeyASRBench currently covers Chinese and English and a fixed keyword set, and its control set is 30 minutes in size; its value lies in proposing a reproducible KER framework rather than providing a universally applicable industry threshold.
50%