Ouro-1.4B
A recurrent language model with a looped structure and approximately 1.4B parameters, used for research on latent reasoning and internal representations.
Timeline (last 90 days)
The experiments use the recurrent model Ouro-1.4B and the standard Transformer Qwen2.5-1.5B, constructing minimal-pair questions that differ by only one number.
At the last prompt token, the numerical information does exist and is represented in a way shared across questions, with a readout accuracy of about 0.9.
When the probe looks at only one prompt at a time (the conventional setting), even after training on 5,249 questions, Ouro's readout rate is only 0.36 and Qwen's only 0.19.
In a pre-registered study of two-step problems, this protocol captures a weak signal about the unwritten intermediate value a+b before Ouro writes the last digit of the answer, and the sign of the signal follows the type of operation.
When a question's intermediate value is patched from the first loop onward with the corresponding value from another question, the written digits do not move toward the other question's answer; applying the same patch to the answer itself does move the digits, indicating that the intermediate value is readable but not actually used by the model.
All Facts (5)
The experiments use the recurrent model Ouro-1.4B and the standard Transformer Qwen2.5-1.5B, constructing minimal-pair questions that differ by only one number.
50%UnverifiedAt the last prompt token, the numerical information does exist and is represented in a way shared across questions, with a readout accuracy of about 0.9.
50%UnverifiedWhen the probe looks at only one prompt at a time (the conventional setting), even after training on 5,249 questions, Ouro's readout rate is only 0.36 and Qwen's only 0.19.
50%UnverifiedIn a pre-registered study of two-step problems, this protocol captures a weak signal about the unwritten intermediate value a+b before Ouro writes the last digit of the answer, and the sign of the signal follows the type of operation.
50%UnverifiedWhen a question's intermediate value is patched from the first loop onward with the corresponding value from another question, the written digits do not move toward the other question's answer; applying the same patch to the answer itself does move the digits, indicating that the intermediate value is readable but not actually used by the model.
50%