MOS
A widely used subjective evaluation metric for speech quality, in which human listeners rate speech samples on a scale of 1 to 5. MOS is broadly applied to assess the naturalness of text-to-speech and speech enhancement systems.
Core Facts
Timeline (last 90 days)
It is generally believed that an MOS above 4.0 is already close to human naturalness, and beyond 4.5 the differences between systems often require more refined testing to distinguish.
WER (word error rate) measures the difference between ASR transcription and reference text, and MOS (mean opinion score) measures the naturalness of synthesized speech; these were the two core metrics for speech evaluation in the past.
In voice agent scenarios, neither WER nor MOS is sufficient: WER only covers comprehension and does not reflect intent understanding; MOS only measures the naturalness of single-sentence synthesis and does not measure multi-turn dialogue context consistency or emotional expression.
In speech synthesis, the evaluation standard for quantization quality is generally comparison with the output of the original FP16 model, commonly measured using similarity metrics and subjective audio naturalness ratings (such as MOS).
Emotion itself is highly subjective, and cross-cultural and cross-individual differences are difficult for a single MOS to fully capture.
Audio dimension scoring involves Time to First Frame (TTFF), end-to-end latency, speech synthesis naturalness (MOS score), and ASR word error rate (WER) under background noise.
The mainstream metric for evaluating speech naturalness is Mean Opinion Score (MOS), where human listeners rate samples on a scale of 1 to 5.
All Facts (7)
The mainstream metric for evaluating speech naturalness is Mean Opinion Score (MOS), where human listeners rate samples on a scale of 1 to 5.
75%UnverifiedIt is generally believed that an MOS above 4.0 is already close to human naturalness, and beyond 4.5 the differences between systems often require more refined testing to distinguish.
50%UnverifiedWER (word error rate) measures the difference between ASR transcription and reference text, and MOS (mean opinion score) measures the naturalness of synthesized speech; these were the two core metrics for speech evaluation in the past.
50%UnverifiedIn voice agent scenarios, neither WER nor MOS is sufficient: WER only covers comprehension and does not reflect intent understanding; MOS only measures the naturalness of single-sentence synthesis and does not measure multi-turn dialogue context consistency or emotional expression.
50%UnverifiedIn speech synthesis, the evaluation standard for quantization quality is generally comparison with the output of the original FP16 model, commonly measured using similarity metrics and subjective audio naturalness ratings (such as MOS).
50%UnverifiedEmotion itself is highly subjective, and cross-cultural and cross-individual differences are difficult for a single MOS to fully capture.
50%UnverifiedAudio dimension scoring involves Time to First Frame (TTFF), end-to-end latency, speech synthesis naturalness (MOS score), and ASR word error rate (WER) under background noise.
50%