FlashTopK
An efficient TopK selection operator for sparse attention mechanisms. It substantially reduces the time consumed by the attention computation bottleneck and improves the inference efficiency of large models.
Timeline (last 90 days)
Tencent Hunyuan, together with the University of Edinburgh, officially open-sourced the FlashTopK operator, aimed at accelerating the TopK filtering bottleneck in DeepSeek-style sparse attention.
FlashTopK reduces the time spent on the TopK filtering step from 5.32 seconds in the traditional implementation to 1.62 seconds, a reduction of about 70%.
According to measured data from the repository, FlashTopK is 1.7 to 3.8 times faster than comparable mainstream implementations in the inference stage, and 1.3 to 2 times faster in absorption training.
FlashTopK is currently adapted to the GLM5 series as well as the Hunyuan Hi4 preview version.
All Facts (4)
FlashTopK reduces the time spent on the TopK filtering step from 5.32 seconds in the traditional implementation to 1.62 seconds, a reduction of about 70%.
50%UnverifiedAccording to measured data from the repository, FlashTopK is 1.7 to 3.8 times faster than comparable mainstream implementations in the inference stage, and 1.3 to 2 times faster in absorption training.
50%UnverifiedFlashTopK is currently adapted to the GLM5 series as well as the Hunyuan Hi4 preview version.
50%UnverifiedTencent Hunyuan, together with the University of Edinburgh, officially open-sourced the FlashTopK operator, aimed at accelerating the TopK filtering bottleneck in DeepSeek-style sparse attention.
50%