Unverified50% confidenceOpinionExact time
Karen Zou pointed out that the repaired small model was actually getting stuck in repetitive output on 93% of the problems, and that Astra found the loop and cut it off rather than genuinely improving math ability.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/4/2026
First Seen
Valid until: 2/2/2027
Sources
Related Entities
Related Claims
UnverifiedThe "Astra fixes Qwen's math ability" experiment used Python and an evaluation loop to find a steering vector that changed only a tiny fraction of the weights, raising the math benchmark score from single-digit percentage points to nearly half.67% similarUnverifiedIn the experiment, Astra modified the weights of the small Qwen model, raising the MATH500 score from 7.9% to 48.1%, while the share of answers hitting the trial-run limit fell from 93% to 33%.66% similarUnverifiedIn a math repair experiment, a small open-source model scored only 7.9% on the Math 500 test; Astra raised the score to 48.1% by locating and subtracting a direction within two layers (about 130,000 weights, an edit ratio of 0.016%), and its instruction-following ability also improved roughly twofold.65% similarUnverifiedIn a sandwich determination demo, the model returned 94% true and 6% false, with the entire decision taking 93 milliseconds.61% similarUnverifiedIn an ablation experiment, after replacing the model's latent thinking segments with segments from unrelated questions, accuracy dropped by 45%–51%, indicating that latent thinking carries substantive reasoning.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1135780API
curl https://kongchang.com/api/v1/knowledge/claims/1135780MCP
get_claim(id=1135780)