MQM
A translation quality evaluation framework that categorizes and annotates error types, such as accuracy and fluency, weights them by severity, and computes a score. It does not rely on reference translations and is regarded as an evaluation approach closer to human judgment.
Timeline (last 90 days)
A new paper systematically tested LLM performance on the two mainstream approaches to machine translation quality evaluation, multidimensional quality metrics (MQM) and error span annotation (ESA), concluding that both humans and models face challenges.
MQM classifies, counts, and weights errors by error type (such as accuracy and fluency) and severity (such as minor and major), while ESA requires annotators to precisely circle the text spans corresponding to errors.
MQM and ESA abandon reliance on reference translations and directly annotate errors in the translation, so they are regarded as approaches closer to human judgment, but at the cost of sharply rising annotation costs.
All Facts (3)
A new paper systematically tested LLM performance on the two mainstream approaches to machine translation quality evaluation, multidimensional quality metrics (MQM) and error span annotation (ESA), concluding that both humans and models face challenges.
50%UnverifiedMQM classifies, counts, and weights errors by error type (such as accuracy and fluency) and severity (such as minor and major), while ESA requires annotators to precisely circle the text spans corresponding to errors.
50%UnverifiedMQM and ESA abandon reliance on reference translations and directly annotate errors in the translation, so they are regarded as approaches closer to human judgment, but at the cost of sharply rising annotation costs.
50%