[KongchangAI]
Concept分布覆盖专家剪枝 / Distribution Coverage Expert Pruning

DCEP

An MoE expert pruning method that preserves distributional geometry by minimizing weighted coverage error, making the pruned experts' output distribution approximate the original distribution.

Timeline (last 90 days)

Nov 2

Distribution Coverage Expert Pruning (DCEP) preserves distributional geometry by minimizing weighted coverage error.

Unverified50%
Nov 2

Both PEEP and DCEP introduce router weights as priors to balance the global structural relations among experts and the local importance of individual experts.

Unverified50%
Nov 2

At expert retention ratios of 75% and 50%, PEEP and DCEP surpass existing state-of-the-art baselines on most benchmarks and achieve the best average performance on general tasks and mathematical reasoning tasks.

Unverified50%
Nov 2

The greedy optimization strategies provided by PEEP and DCEP are computationally efficient and suitable for practical deployment.

Unverified50%

All Facts (4)

Source Articles