COCONUT
A continuous chain-of-thought framework proposed by Meta in which a model uses the final hidden state of one reasoning step directly as the input embedding for the next step, bypassing token decoding and re-encoding. This approach enables reasoning within latent space and serves as a representative experimental instance of Neuralese research.
Timeline (last 90 days)
AI safety research organization Redwood Research published an analysis article pointing out that latent-space reasoning architectures represented by COCONUT and full-bandwidth Transformers may greatly weaken the currently most effective AI supervision tool — chain-of-thought (CoT).
COCONUT completely replaces textual CoT with a chain of continuous 'latent thoughts'; the model no longer outputs readable reasoning steps but instead reasons in a continuous vector space, causing the supervisory value of CoT to drop directly to zero.
The COCONUT framework has demonstrated stronger performance on certain mathematical reasoning tasks.
COCONUT (Chain of Continuous Thought) propagates the reasoning chain directly in continuous latent space, feeding the hidden state produced by the previous reasoning step directly as input to the next, bypassing the encode-decode loop from hidden state to token and back to hidden state.
COCONUT outperforms traditional CoT on certain logical reasoning tasks, particularly those requiring breadth-first search (BFS)-style reasoning, because continuous space naturally supports parallel exploration of multiple reasoning paths.
All Facts (5)
COCONUT (Chain of Continuous Thought) propagates the reasoning chain directly in continuous latent space, feeding the hidden state produced by the previous reasoning step directly as input to the next, bypassing the encode-decode loop from hidden state to token and back to hidden state.
60%UnverifiedAI safety research organization Redwood Research published an analysis article pointing out that latent-space reasoning architectures represented by COCONUT and full-bandwidth Transformers may greatly weaken the currently most effective AI supervision tool — chain-of-thought (CoT).
50%UnverifiedCOCONUT completely replaces textual CoT with a chain of continuous 'latent thoughts'; the model no longer outputs readable reasoning steps but instead reasons in a continuous vector space, causing the supervisory value of CoT to drop directly to zero.
50%UnverifiedThe COCONUT framework has demonstrated stronger performance on certain mathematical reasoning tasks.
50%UnverifiedCOCONUT outperforms traditional CoT on certain logical reasoning tasks, particularly those requiring breadth-first search (BFS)-style reasoning, because continuous space naturally supports parallel exploration of multiple reasoning paths.
50%