[KongchangAI]
Concept投影能量专家剪枝 / Projection Energy Expert Pruning

PEEP

An MoE expert pruning method that preserves functional subspaces by maximizing weighted projection energy, using router weights as a prior to select a structure-preserving subset of experts.

Timeline (last 90 days)

Nov 2

Projection Energy Expert Pruning (PEEP) preserves functional subspaces by maximizing weighted projection energy.

Unverified50%
Nov 2

Both PEEP and DCEP introduce router weights as priors to balance the global structural relations among experts and the local importance of individual experts.

Unverified50%
Nov 2

At expert retention ratios of 75% and 50%, PEEP and DCEP surpass existing state-of-the-art baselines on most benchmarks and achieve the best average performance on general tasks and mathematical reasoning tasks.

Unverified50%
Nov 2

The greedy optimization strategies provided by PEEP and DCEP are computationally efficient and suitable for practical deployment.

Unverified50%

All Facts (4)

Source Articles