Cognitive Thermometers: How Machine Learning Measures Semantic Complexity

A New Answer to an Old Question

How does the human mind represent semantic categories? Why do natural languages favor certain meanings while avoiding others? These questions have long puzzled linguists and cognitive scientists. A newly published arXiv paper (arXiv:2610.10724) offers an intriguing perspective: treating machine learning models as "cognitive thermometers" for measuring semantic complexity.

Traditionally, researchers have relied on logical definability and logical complexity to explain why certain semantic concepts appear more frequently in language. For instance, quantifiers like "all" and "some" are relatively simple, while more complex quantifier combinations are comparatively rare. This type of explanation assumes that logically simpler concepts are more readily adopted and widely used by language.

But the paper's authors point to a fundamental flaw in this approach — it is heavily dependent on the choice of logical language. Whether you use first-order logic, modal logic, or some other formal system to describe a concept, the resulting "complexity" can differ substantially. And the choice of logical language often lacks sufficient theoretical motivation, making the entire measurement framework feel somewhat arbitrary.

Semantic typology and logical definability are two foundational concepts for understanding this paper. Semantic typology is a branch of cross-linguistic comparative research whose central question is: among the world's thousands of languages, which differ enormously in vocabulary and grammar, why do certain semantic structures (such as the "all/some" quantifier opposition) appear nearly universally, while other logically valid semantic structures are exceedingly rare? Researchers believe there are cognitive or information-theoretic universal constraints at work.

Logical definability is the traditional tool for characterizing these constraints: how short a first-order logic formula is needed to express a given quantifier or semantic concept is treated as a proxy for its "cognitive accessibility." For example, "all x satisfying P" can be written directly using first-order universal quantification, while expressions requiring nested quantifiers or counting are considerably more complex. The underlying assumption of this framework is that the human mind prefers logically concise concepts, causing language to converge toward semantic fragments with lower definability.

Machine Learning as a "More Neutral" Measure

The paper's core claim is that machine learning offers a complexity metric that is relatively less dependent on human assumptions. Rather than debating which logical symbol system to use to define a semantic concept, one can simply observe how much effort a learning model needs to acquire it.

The elegance of this approach lies in transforming "complexity" from an abstract symbolic definition into an operationalizable measure of learning difficulty. A concept that requires more training data, longer training time, or a more complex model architecture to master is, in some sense, more "complex." Because this measure presupposes no particular logical framework, the authors describe it as a "somewhat more agnostic" approach.

The "cognitive thermometer" metaphor is apt: a thermometer doesn't concern itself with the molecular motion underlying heat — it simply provides a unified, comparable reading. Similarly, machine learning models can provide a unified complexity "reading" for different semantic concepts, bridging the gap between symbolic logic and connectionist AI.

At a technical level, this approach is typically operationalized through sample complexity or convergence speed. Researchers design learning tasks — for example, having a model induce the extension of a quantifier from positive and negative examples — then record how many training samples or training rounds are needed to reach a certain accuracy level. Higher sample complexity signals that a concept is "harder" for the learning algorithm, which is treated as evidence of greater semantic complexity.

It's worth noting that different machine learning architectures — from simple perceptrons to recurrent neural networks — may produce different difficulty rankings for the same concept. This is precisely the source of the model selection problem mentioned in the paper's more cautious sections. The paper's implicit claim is that when multiple architectures agree on a concept's difficulty ranking, that agreement itself constitutes evidence for the concept's objective complexity — analogous to multiple thermometers corroborating each other's readings.

Logic and Learning: Convergence and Divergence

The paper reviews a body of emerging evidence showing that logical methods and machine learning methods frequently reach consistent conclusions when measuring relative complexity. That is, concepts judged simple by logic tend to be learned faster by machine learning models. This convergence is itself a powerful piece of evidence — two completely different theoretical paths pointing to the same conclusion strengthens the credibility of that conclusion.

When the Two Diverge

More interesting are the cases where the two diverge. The paper argues that when logical complexity and learning difficulty give different verdicts, machine learning is often the better explanation.

The reasoning is this: language is acquired and used by human minds, and human learning mechanisms themselves have connectionist characteristics. If a concept is logically simple but proves difficult to learn, then its distribution across languages should align more closely with the "hard to learn" prediction than the "logically simple" prediction. This effectively transfers some of the explanatory authority in semantic typology from symbolic logic to the learning process.

Implications for Semantic Typology

Semantic typology studies cross-linguistic patterns in meaning expression — which meanings are frequently encoded, and which are universally avoided. By bringing machine learning into this domain, the paper aims to provide a unified complexity framework that can accommodate the insights of symbolic logic while drawing on the learning dynamics of connectionist models.

This integrative orientation reflects a broader trend in cognitive science: the attempt to bridge the longstanding methodological divide between symbolicism and connectionism. Symbolic logic excels at precise definition and interpretability, while connectionist models more closely resemble actual human learning and generalization behavior. Treating machine learning models as "thermometers" means researchers can use a common scale to compare the predictions of both paradigms and make principled judgments when they conflict.

The methodological opposition between symbolicism and connectionism was one of the most central debates in cognitive science and AI during the latter half of the 20th century. Symbolicism holds that intelligence can be described through formal rules and logical inference, with classic Prolog programs and Montague semantics as representative achievements. Connectionism holds that intelligence emerges from the parallel interaction of large numbers of simple units (neurons), with various neural networks as representative achievements. In linguistics, the generative grammar tradition leans toward symbolicism, while distributional semantics and neural language models belong to the connectionist camp.

The core tension between the two paradigms lies in their different modes of generalization: symbolic systems generalize systematically through rules (if "all" holds in context A, the inference to B follows mechanically), while connectionist models generalize through statistical similarity — sometimes capturing human intuition but rarely providing interpretable rules. Positioning machine learning as a "thermometer" rather than a "theory" is a pragmatic integration strategy — it doesn't require resolving the fundamental divide between the two paradigms, but simply borrows the learning behavior of connectionist models as a measurement tool.

A Few Cautionary Reflections

This paper is an opinion and review piece — it proposes a research program rather than presents empirical conclusions. Several questions merit further pursuit:

• The circularity of model selection: Machine learning claims to be more neutral than logical methods, but model architecture, training strategy, and data distribution are equally human choices. Does this simply replace the question of "which logic to choose" with "which model to choose"?
• Correlation vs. causation: A correlation between learning difficulty and semantic distribution does not directly imply that learning mechanisms shaped language. More fine-grained mechanistic evidence is needed.
• The interpretability trade-off: One advantage of logical complexity is transparency and interpretability, whereas the "learning difficulty" reading of a neural network is often difficult to reverse-engineer into the concept's internal structure.

Nevertheless, the "cognitive thermometer" framework offers a valuable conceptual tool for bridging the symbolic and neural traditions. It reminds us that measuring the complexity of meaning need not rely on a single ruler — and that the convergences and divergences between different rulers are themselves important clues to understanding human language and cognition.