Neuralese
A term referring to the non-human-readable continuous vector-space representations that large language models use during internal reasoning. Distinct from natural-language chain-of-thought, Neuralese involves passing high-dimensional vectors directly through latent space across multiple reasoning steps, producing intermediate states that humans cannot directly interpret or supervise.
Timeline (last 90 days)
Rob Miles argues that obfuscated chains of thought are merely "opaque chains of thought" and not true Neuralese.
True Neuralese refers to the language used for communication between neurons, achieved by skipping de-embedding and sampling and directly feeding high-dimensional vectors back into the model—i.e., opaque recursion.
Neuralese refers to the non-human-readable representational form in continuous vector space that models use during internal reasoning.
Once reasoning shifts into Neuralese, the supervision channel based on CoT output is severed, as humans cannot read high-dimensional vectors nor intervene in reasoning in a timely manner.
When training models with a purely outcome-oriented approach, optimization pressure spontaneously tends to push reasoning toward more efficient but unreadable Neuralese forms.
Implicit latent-space reasoning may bring faster inference speeds, stronger capability in handling complex problems, and fewer token consumption, but there is a fundamental trade-off between transparency and performance.
All Facts (6)
Rob Miles argues that obfuscated chains of thought are merely "opaque chains of thought" and not true Neuralese.
50%UnverifiedTrue Neuralese refers to the language used for communication between neurons, achieved by skipping de-embedding and sampling and directly feeding high-dimensional vectors back into the model—i.e., opaque recursion.
50%UnverifiedNeuralese refers to the non-human-readable representational form in continuous vector space that models use during internal reasoning.
50%UnverifiedOnce reasoning shifts into Neuralese, the supervision channel based on CoT output is severed, as humans cannot read high-dimensional vectors nor intervene in reasoning in a timely manner.
50%UnverifiedWhen training models with a purely outcome-oriented approach, optimization pressure spontaneously tends to push reasoning toward more efficient but unreadable Neuralese forms.
50%UnverifiedImplicit latent-space reasoning may bring faster inference speeds, stronger capability in handling complex problems, and fewer token consumption, but there is a fundamental trade-off between transparency and performance.
50%