D-Flash
投机解码变体方案,完全并行化草稿生成,速度快但各token互相独立,容易产生后缀衰减问题
Core Facts
Timeline (last 90 days)
The new version combines D-Flash2, MTP, and N-gram speculative decoding, and the combination of all three produces a roughly twofold decoding acceleration effect.
Muse Glimmer采用模块化架构:约28B文本参数负责推理与规划,2B用于工具调用,2B ViT负责视觉理解,另有D-Flash小模型用于加速
Muse Glimmer的D-Flash投机解码方案一次生成16个候选token由主模型并行验证
On the RTX 5090, Muse Glimmer's generation speed increases from 75 tokens per second to 233 tokens per second, a speedup of approximately 3.1×.
All Facts (4)
On the RTX 5090, Muse Glimmer's generation speed increases from 75 tokens per second to 233 tokens per second, a speedup of approximately 3.1×.
65%UnverifiedThe new version combines D-Flash2, MTP, and N-gram speculative decoding, and the combination of all three produces a roughly twofold decoding acceleration effect.
50%UnverifiedMuse Glimmer采用模块化架构:约28B文本参数负责推理与规划,2B用于工具调用,2B ViT负责视觉理解,另有D-Flash小模型用于加速
50%UnverifiedMuse Glimmer的D-Flash投机解码方案一次生成16个候选token由主模型并行验证
50%