BLEU

Mainstream approaches to agent evaluation include LLM-as-Judge, trajectory matching, and outcome verification.

BLEU is commonly used for machine translation evaluation, and ROUGE is widely applied to summarization tasks; both measure text quality by statistically calculating n-gram overlap

BLEU is a natural language generation evaluation metric proposed in the 1990s that measures quality by calculating the n-gram overlap between generated text and reference translations, and remains the standard benchmark in the field of machine translation.

Current mainstream LLM evaluation methods include reference-based metrics (such as BLEU and ROUGE), model-based evaluation (LLM-as-a-Judge), and human preference-based evaluation (such as the ELO rating of Chatbot Arena).

BLEU is sensitive to morphological variation, and Hindi, Urdu and Sindhi all have rich morphological inflection, where the same meaning can have multiple morphological realizations.

On the Hindi-to-Urdu translation task, TITAN achieves a 33.75% BLEU improvement over the strongest multilingual baseline model.

BLEU measures translation quality by counting the overlap rate of n-grams (usually 1- to 4-grams) between the model output and the reference translation, and imposes a length penalty on overly short outputs; its value ranges from 0 to 100, but it is not a linear scale.

Automatic metrics such as BLEU and ROUGE require the existence of a high-quality reference translation, and many real-world scenarios do not have this condition.

Automatic metrics such as BLEU and ROUGE only do surface matching, cannot identify semantic errors, and cannot distinguish error severity.

ROUGE基于召回率统计n-gram重合度，BLEU基于精确率并加入长度惩罚，两者都只看字面重叠，对同义替换敏感。

Traditional document parsing evaluations commonly use CER (Character Error Rate) or BLEU scores, metrics that are statistically reasonable but difficult to map to business-relevant questions.

BLEU and ROUGE are automated evaluation metrics based on reference answers, primarily used for translation or summarization tasks.

The study adopts a dual-track evaluation approach: using BLEU to measure surface similarity and an LLM-based judge to measure semantic similarity.

Measuring the performance of SimulST systems typically uses two dimensions: translation quality (BLEU score) and latency (Average Lagging AL, Differentiable AL), which together form the quality–latency frontier curve.

BLEU was originally designed for machine translation and measures quality by calculating n-gram overlap rates.

Prior to DoGBench, assessing the quality of AI-generated documentation mostly relied on subjective judgment or general-purpose text metrics such as BLEU and ROUGE.

General-purpose text metrics such as BLEU and ROUGE focus only on literal word matching and cannot measure whether users can actually complete tasks by following the documentation.

Existing generative AI evaluation methods rely on text-overlap metrics such as BLEU and ROUGE or LLM-as-judge, but have the limitation of being unable to capture narrative logic and visual layout when evaluating presentations.

Both BLEU and perplexity are form-driven metrics that observe only surface text and cannot determine whether a response is genuinely grounded in the context semantically and logically.

BLEU scores by counting the n-gram overlap between model output and reference translations, and was originally designed for machine translation.