Automatic Speech Recognition

Homophone confusion is a classic hard problem in automatic speech recognition (ASR) systems; words with similar phonemes such as spell and smell can hardly be distinguished by sound alone when semantic or task context is lacking.

Mainstream ASR systems typically require thousands to tens of thousands of hours of human-annotated audio to achieve usable accuracy.

Semi-supervised federated learning (SSFL) trains an initial model using a small amount of labeled seed data on the server side, then uses a teacher model to generate pseudo-labels for large volumes of unlabeled client data, thereby optimizing the model without uploading raw audio.

The output of speech recognition is a sequence, and errors in pseudo-labels accumulate across the entire output sequence, where a recognition deviation at one position may trigger a chain of misjudgments in subsequent tokens.

In ASR scenarios, the teacher model is typically an end-to-end speech recognition model based on CTC or an Attention-based Encoder-Decoder.

An arXiv paper (arXiv:2609.12122) proposes using acoustic masking combined with automatic speech recognition (ASR) models to automatically quantify each consonant's contribution to word intelligibility.

The research method works by silencing one consonant in a word at a time and testing whether the ASR model can still correctly recognize that word.

MMR measures whether a machine can still recognize the word rather than whether a human listener can still understand it, and there are systematic differences between ASR model recognition behavior and human perception.

AI real-time caption translation typically involves two layers of technology: an ASR model first converts speech into source-language text, which is then translated into the target language by an NMT model.

Automatic speech recognition can still make errors in scenarios involving technical terminology, multiple speakers, and accents, directly affecting retrieval quality.