The VLM Evaluation Trap: Clinical Terminology Erasure and Hallucinated Bias Behind High Scores

When High Scores No Longer Mean High Quality

Vision-Language Models (VLMs) have shown remarkable potential in Radiology Report Generation (RRG) in recent years—they can read chest X-rays and automatically generate seemingly professional diagnostic reports. However, a disturbing phenomenon is emerging: these models score high on standard evaluation metrics while silently erasing critical clinical terminology and even introducing biased hallucinated content.

Vision-Language Models are a class of deep learning models that simultaneously process visual and textual information. Their typical architecture combines a visual encoder (such as a Vision Transformer) with a language decoder (such as a GPT-series architecture). In the radiology report generation task, the model must receive medical images as input, understand the anatomical structures and pathological features within them, and then generate structured natural language reports. The difficulty of this task lies in requiring not only precise visual perception but also rich medical knowledge and standardized clinical expression. In recent years, with the development of large-scale pretraining techniques, models like BiomedCLIP and Med-PaLM have made significant progress in this field, but their actual clinical reliability has always been questionable.

This finding comes from a study focused on chest X-ray report generation (paper available at arXiv:2603.01625). The research team noticed during actual use of VLMs for report generation that mainstream evaluation metrics have systematic flaws: they reward repetitive template text, reports lacking specific clinical terminology, and reports that conclude findings as "normal." In other words, models have learned "how to score high on exams" rather than "how to write truly useful clinical reports."

Evaluation Metric Flaws: Rewarding Mediocrity, Penalizing Expertise

The core of this problem lies in the misalignment between evaluation metrics and clinical value. The natural language generation metrics widely used in the RRG field (such as BLEU, ROUGE, CIDEr, etc.) fundamentally measure surface-level overlap between generated text and reference text.

From a technical standpoint, BLEU (Bilingual Evaluation Understudy) scores by calculating n-gram precision between generated and reference texts; ROUGE (Recall-Oriented Understudy for Gisting Evaluation) focuses on recall, measuring how many n-grams from the reference text are covered by the generated text; CIDEr (Consensus-based Image Description Evaluation) introduces TF-IDF weighting, attempting to reduce the influence of high-frequency words and boost the weight of rare words. However, the fundamental limitation of these metrics is that they are essentially statistical text-matching tools incapable of understanding the clinical importance at a semantic level—a missing critical lesion term and a change in an irrelevant article may carry the same weight in the eyes of these metrics.

Why Repetitive Templates Score High

Radiology reports contain abundant high-frequency, fixed expression patterns, such as "heart size is normal," "lung fields are clear," and "no obvious abnormalities observed." These phrases appear extremely frequently in training data, and models can achieve considerable matching scores by consistently outputting these "safe" templates when compared against reference reports.

Why Rare but Critical Terms Get Erased

What truly reflects clinical value are often those rare but critical terms—specific lesion descriptions, uncommon anatomical abnormalities, subtle pathological signs. Because these words appear infrequently in the data, models tend to "avoid" generating them, since the risk of generating them incorrectly is high while their contribution to overall metric scores is small. The researchers pointedly note that this makes generated reports appear "repetitive and bland," ultimately rendering them clinically useless.

This is a classic case of metric optimization bias (a manifestation of Goodhart's Law): when we use an imperfect proxy metric to measure a goal, the model learns to "game the system," optimizing the metric itself rather than what we actually care about. Goodhart's Law was originally proposed by British economist Charles Goodhart in 1975, stating that "when a measure becomes a target, it ceases to be a good measure." In machine learning, this phenomenon is called "metric hacking" or "reward hacking." For example, in reinforcement learning, agents may find shortcuts to maximize the reward function rather than truly completing the intended task. In the RRG scenario, models maximize overlap with reference text by generating high-frequency templates—essentially a form of "benign manipulation" of evaluation metrics. It's not malicious cheating; rather, the optimization objective itself is biased.

Silent Erasure and Hallucinated Bias: Two Hidden Failure Modes

The study identifies two VLM behavior patterns that warrant high vigilance:

Terminology Erasure: Models systematically omit clinically meaningful vocabulary, causing reports to become homogenized. This erasure is "silent"—it leaves no negative traces on traditional metrics and may actually improve scores by reducing "opportunities for error."

Hallucinated Bias: Models introduce biased terminology into reports that doesn't exist in the original images. This is more dangerous than simple information omission because it actively injects erroneous information into reports, potentially misleading clinical decisions.

Hallucination is one of the core challenges facing current large language models. In general scenarios, hallucinations may manifest as fabricated citations or facts; in medical scenarios, the harm of hallucinations is enormously amplified. Research shows that VLM hallucinations can be divided into "object hallucinations" (generating objects not present in the image) and "attribute hallucinations" (incorrectly describing properties of existing objects). The "hallucinated bias" revealed in this study is more insidious—models don't randomly generate incorrect content but systematically introduce bias in specific directions. This may be related to annotation biases in training data, uneven population distributions, and other factors, causing certain diagnostic conclusions to be inappropriately reinforced or suppressed.

These two behaviors together constitute the hidden risks of VLM deployment in medical scenarios: a model that performs excellently on paper may actually be doing two terrible things—not saying what should be said, and saying things that shouldn't be said.

How to Measure "What the Model Didn't Say"

To address these issues, the research team proposed a new evaluation framework whose core idea is distilled in the paper's title—"Measuring What VLMs Don't Say."

Traditional metrics only focus on what the model "says" and how close it is to the reference text, while this framework shifts attention to:

• Degree of terminology erasure: How many clinically critical terms did the model omit that should have appeared?
• Introduction of biased terms: How much biased content did the model add that shouldn't be there?

By quantifying these two dimensions, researchers can reveal model deficiencies hidden by traditional metrics. This provides a perspective more aligned with real clinical value for evaluating medical AI systems: a report's value lies not only in what it gets right, but also in whether it omits critical information or fabricates false content.

It's worth noting that the medical AI evaluation field is exploring multiple improvement directions. Tools like RadGraph evaluate the accuracy of entities and relationships in reports by constructing radiology knowledge graphs; the F1-RadGraph metric can measure precision and recall at the clinical entity level; label extraction tools like CheXbert can convert reports into structured disease labels for comparison. Additionally, clinical validity assessment (such as having radiologists perform blinded scoring) remains the gold standard despite its high cost. The "measuring what wasn't said" framework proposed in this study represents a complementary perspective—not only evaluating the correctness of generated content but also systematically tracking information completeness and bias direction.

Implications for Medical AI Deployment

The significance of this research extends far beyond the specific task of radiology report generation—it raises profound warnings for the entire medical AI field and the broader generative AI evaluation ecosystem.

Evaluation Systems Need Restructuring

When deploying AI in high-risk domains, we cannot simply rely on generic NLG metrics. These metrics were designed for general text generation and cannot capture the completeness and accuracy of domain-specific knowledge. Medical, legal, financial, and other specialized fields all need to establish customized evaluation frameworks that reflect real-world value.

Beware of "Score-Chasing" Optimization on Benchmarks

When researchers and engineers single-mindedly pursue topping leaderboards (benchmarks), they likely cultivate models that are skilled at test-taking rather than truly competent. This reminds the entire industry: metrics are means, not ends. A model's high score on a benchmark should never be directly equated with its reliability in real-world scenarios.

Pay Attention to AI Systems' "Silent Failures"

The most dangerous AI failures are often not the obvious errors but this kind of "silent erasure"—systems quietly losing critical information while appearing to function normally on the surface. In medical diagnosis, an omitted lesion description could mean a delayed diagnosis. Identifying and measuring these hidden failures should be a core component of responsible AI development.

Conclusion

This research tears away the facade behind VLMs' impressive data in medical applications: high benchmark scores may be nothing more than a disguise. When models learn to "please" evaluation metrics by using repetitive templates and omitting rare terminology, they actually move further and further from true clinical value.

For any team hoping to introduce generative AI into high-risk professional domains, this case serves as a wake-up call—before trusting model outputs, we must first examine: what exactly are our evaluation standards measuring? And more importantly, what are they not measuring?