Why Do Latent Reasoning Probes Fail? Minimal Pairs Reveal the Truth of "Present but Hard to Read"

The Blind Spot of Probe Research: Not Reading It Doesn't Mean It Isn't There

Probe research on large models' "latent reasoning" often rests on a default assumption: if some piece of information cannot be read out at the position where the answer begins, then the model does not internally represent that information. But a paper titled Present but Hard to Read: Probing Latent Reasoning with Minimal Pairs directly challenges this.

The researchers ran a minimal experiment with the recurrent model Ouro-1.4B and the standard Transformer Qwen2.5-1.5B: make questions differ by only a single number while keeping everything else identical. The result: this number does indeed exist at the last prompt token, and is represented in an encoding shared across questions—with a readout accuracy of about 0.9. However, when the probe looks at only one prompt at a time (i.e., the usual setup), even after training on 5,249 questions, Ouro's readout rate is only 0.36, and Qwen's is just 0.19.

The contrast between these numbers shows that the failure of traditional probes reflects more the limitations of the reading method than the absence of information inside the model.

Probe: a method that trains a simple classifier (such as linear regression) on a model's internal representations to detect whether specific information (such as part of speech, facts, or intermediate computation results) is encoded in a linearly readable form in the hidden states. The usual approach: collect activation vectors as the model processes a large number of inputs, train a probe on these vectors to predict the target label, then test the probe's readout accuracy on new inputs. However, traditional probes often use only the last token state of a single prompt for training and evaluation, which implicitly assumes that the information is linearly separable at that position. The core of this paper is precisely to challenge this assumption: when the signal is weak in the single-prompt setting, a probe's failure may stem from the reading method rather than missing information.

Not Too Crowded with Information, but Too Weak a Signal

A natural explanation is that the model's state carries too much information, so the target number is "squeezed" and hard to distinguish. But the experiments reject this "crowding hypothesis."

The researchers found that the state shift caused by a change in number is actually very small, and random noise can also land on equally "busy" directions. By contrast, if that number is replaced with a random word, the state change is much larger and easier to read out. In other words, the problem is not too much information, but that the trace left by this copied number is itself very weak.

What's more, because every changing token leaves a trace, a successful probe result is also ambiguous—it may have read the computed value, or it may have merely read a copy or mixture of the input.

Minimal pairs is a classic experimental design in linguistics and psycholinguistics: construct two sentences that are identical except for one key component (for example, differing only in a number), and compare the difference in the model's internal representations of the two sentences to isolate that component's contribution. In this study, minimal pairs let the researchers precisely measure the state change caused by a single number token, without interference from other factors. This design helps distinguish "whether the information exists" from "whether the information is readable": if the state difference between minimal pairs does exist but is very small, then even if the probe cannot read it out, one cannot conclude that the model does not encode that information.

A Protocol for Distinguishing "Computed Values" from "Input Traces"

To address this ambiguity, the authors propose a testing protocol whose core is a synthetic null, used to simulate generic mixture effects. This protocol can test whether a probe is reading an intermediate value computed by the model or a product of input copies or mixtures.

In a pre-registered study on two-step problems, this protocol captured, before Ouro wrote out the last digit of the answer, a weak signal about the unwritten intermediate value a+b, and the sign of the signal changed with the type of operation. However, the study honestly notes that mixture effects alone can produce a considerable portion of this "sign-following" phenomenon.

Synthetic null: a statistical control method in which researchers artificially construct a "null hypothesis" dataset where the target signal is replaced with random noise or pseudo-signals generated by a mixture model, in order to simulate the false positives a probe might produce due to input traces or mixture effects. By comparing the probe's performance on real data with its performance under the null, one can judge whether the information read out by the probe is a genuine computed value or merely reflects a copy or mixture of input tokens. This method helps improve the reliability of probe conclusions and avoid mistaking mixture effects for the model's internal computation.

Readable but Unused: The Last Mile of Latent Reasoning

The most intriguing result of the study comes from intervention experiments. When the researchers, starting from the first recurrence, "hold" a question's intermediate value to the corresponding value of another question, the written digit does not shift toward the other question's answer. But if the same patch is applied to the answer itself, the digit does move.

This means that although that intermediate value can be read out, it is not actually used by the model. The authors sum it up in one sentence: the value is readable but not used.

This finding raises a dual methodological caution for latent reasoning research: a probe failing to read something does not mean it isn't there; and a probe reading something does not mean it is used. Understanding a model's internal representations requires asking both questions at once: "is it readable?" and "is it used?"

Intervention experiment: a commonly used tool in causal inference—by artificially changing some activation value inside the model (for example, replacing an intermediate representation with the corresponding value from another input), observe whether the model's output changes accordingly. If the output changes, the representation is used by the model in computation; if the output does not change, the representation exists but is not used. The "patch" in this paper refers to this kind of activation replacement operation. Intervention experiments can distinguish "readable" from "used": a probe can only show that information can be decoded, while intervention is needed to prove that the information causally affects the model's behavior.

Implications for Probe Methodology

This work does not negate the value of probe research but pushes it to a more refined level. The minimal-pairs design lets researchers isolate the contribution of a single token, and the synthetic null provides a necessary control baseline. For research that relies on probe conclusions to infer what a model "knows," these controls should become standard practice.

From a broader perspective, the two findings—"present but hard to read" and "readable but unused"—together paint a picture more complex than "the model internally has/doesn't have some information." The representation, reading, and use of latent reasoning are three links that need to be tested separately.