When an LLM Says "Very Likely": The Gap Between Linguistic Uncertainty and Human Understanding

Humans Express Uncertainty Through Words — But LLMs Are Speaking a Different Language

In everyday communication, people naturally use verbal markers to convey how confident they are about something — words like "maybe," "probably," or "almost certain." These expressions map to rough probability ranges, reflecting a form of metacognition: we know what we know, we know what we don't, and we adjust our behavior and information-seeking accordingly.

Yet most mainstream methods for LLM Uncertainty Quantification (UQ) rely on computationally expensive likelihood calculations or multi-sample consistency checks — rather than reading confidence directly from the model's own linguistic output. A new paper on arXiv (arXiv:2610.00083) addresses exactly this gap: when an LLM says "very likely," what probability does that actually correspond to internally? And does that number align with what a human would mean by the same phrase?

Metacognition is a psychological concept describing "cognition about one's own cognitive processes," systematically introduced by psychologist John Flavell in the 1970s. In the domain of uncertainty expression, researchers have long established that humans share relatively stable probability mappings for terms like "possible," "probable," and "almost certain" — for example, "very likely" consistently maps to a 75%–90% probability range across multiple cross-cultural studies. This finding has been widely applied in intelligence analysis (such as the U.S. Office of the Director of National Intelligence's standardized language guidelines) and medical risk communication, with the explicit goal of reducing decision errors caused by linguistic ambiguity. Uncertainty Quantification (UQ) is a core topic in machine learning, aimed at enabling models to not only produce predictions but also reliably estimate their confidence in those predictions — a foundational capability for building trustworthy AI systems.

Core Finding: A Systematic Mismatch Between Machine and Human Confidence Semantics

The research team compiled a corpus of human uncertainty markers from the psychology and decision science literature, then used it as a benchmark to evaluate LLMs. The results showed significant divergence: the numerical uncertainty levels that LLMs associate with these linguistic markers differ substantially from human expectations.

In other words, when a model says "very likely," the probability range it has in mind is not the same one a human defaults to. This misalignment is not random noise — it is a systematic confidence discrepancy. For applications that increasingly rely on natural language interaction with AI, this is a long-overlooked yet consequential problem: users may interpret a model's phrasing through the lens of human semantic conventions, leading to systematic misjudgments.

Why Verbal Markers Matter More Than Raw Probability Scores

From a cognitive science perspective, accurate linguistic uncertainty expression reflects metacognitive monitoring — a clear awareness of one's own knowledge boundaries. A model that can appropriately say "I'm not very sure" is more aligned with human collaborative and decision-making intuitions than a black box that simply outputs a probability of 0.73. Verbal markers are therefore not just rhetorical flourishes in model output — they are potential reliability signals, provided we can calibrate them.

METHODNAME: Reverse-Engineering the True Meaning of Markers from Output

The paper proposes a new optimization-based algorithm (referred to in the paper as METHODNAME) with an elegant core idea: rather than estimating uncertainty through repeated sampling, it directly learns an optimal "uncertainty profile" from the LLM's own outputs.

Specifically, the algorithm fits a "marker-to-uncertainty" mapping for each linguistic marker, such that it best explains the model's observed accuracy rates. This allows the algorithm to reverse-engineer how much probability mass each verbal marker should carry within the model's internal representation.

Two Key Contributions of This Approach

First, it sidesteps the high computational cost of multi-sample approaches, offering a more lightweight path to UQ. Second, it enables direct marker-level comparison between humans and LLMs — decomposing the confidence semantic mismatch to pinpoint exactly which words and which probability ranges diverge. This interpretability is especially valuable for downstream model calibration and human-AI alignment work.

Conventional LLM uncertainty quantification methods fall into two main categories: likelihood-based methods, which read log-probabilities from model output tokens but require access to internal model parameters (making them infeasible for closed-source models like GPT-4); and ensemble/sampling-based methods such as Self-Consistency, which estimate uncertainty by sampling the same question multiple times and aggregating the answer distribution — accurate, but at the cost of multiplied inference overhead. The optimization-based approach proposed in this paper represents a third path: treating verbal markers as spontaneously generated "soft confidence signals" and calibrating their semantics by fitting the mapping between markers and actual accuracy rates. It requires neither repeated sampling nor access to internal parameters, offering potential advantages in both efficiency and applicability.

Implications for AI Reliability

This research touches on an underappreciated safety issue: as we increasingly trust the "confidence" or "hesitation" that models express in natural language, how reliable are those expressions really? If a model's "very likely" actually corresponds to an accuracy rate far below human expectations, users will unknowingly absorb additional risk.

Calibrating verbal markers to align with true accuracy rates would mean that future AI systems could express uncertainty in ways that better match human intuition — enabling safer decision-making and information-seeking. This has direct practical relevance for high-stakes AI applications in medicine, law, finance, and similar domains.

It should be noted that the algorithm is referred to with the placeholder METHODNAME in the paper; its official name, experimental scope, and cross-model generalization capabilities are yet to be fully disclosed in the final version. As a newly published preprint, its findings are best treated as exploratory signals pointing in a promising direction, rather than settled conclusions.

Real-world cases in high-risk domains already illustrate why miscalibrated linguistic uncertainty deserves serious attention. Research shows that clinicians using AI-assisted diagnostic tools tend to treat high-confidence verbal expressions (such as "most likely X") as more credible than numerical probabilities — even when both convey identical information. If a model's linguistic confidence is systematically misaligned with its actual accuracy, this "overtrust" effect becomes significantly amplified. Similar dynamics exist in legal and financial contexts: when compliance officers rely on the wording in AI-generated summaries to assess risk levels, calibration errors in verbal markers translate directly into decision risk. This is precisely why "linguistic uncertainty alignment" is increasingly recognized as an independent subfield of AI safety research — not merely an afterthought to model output quality.