Unverified50% confidenceBenchmarkExact time
KITE / Step Scale Transformer (September 23, 2025) splits the architecture into a Prefiller tower and a Decoder tower, and the 67B MoE version reduces inference cost by 31.6% compared with a 63B classic Transformer while achieving better quality.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/19/2026
First Seen
Valid until: 1/17/2027
Sources
Related Entities
Related Claims
UnverifiedSwitch Transformer and the Mixtral series have widely adopted the MoE architecture.65% similarUnverifiedMiniMax adopts a Transformer architecture combined with a Diffusion Model.65% similarUnverifiedTransformer autoregressive generation is divided into the Prefill stage (compute-intensive, with higher GPU utilization) and the Decode stage (bandwidth-intensive, generating tokens one at a time)64% similarUnverifiedThe Universal Transformer proposed by Dehghani et al. in 2019 was the first to design Transformer layers as repeatable recurrent units, and introduced an adaptive computation time mechanism to dynamically determine the number of recurrent iterations.63% similarUnverifiedAlthough modern large models each have their own optimizations in detail, over 95% of their core architecture is based on the Transformer architecture.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1070134API
curl https://kongchang.com/api/v1/knowledge/claims/1070134MCP
get_claim(id=1070134)