[KongchangAI]
PaperJanus-MOPD 框架

Janus-MOPD

A framework for dynamic analysis of geometric representations in multi-teacher on-policy distillation for large reasoning models, revealing domain heterogeneity and subspace locking by tracking the evolution of the student model's internal representation space.

Timeline (last 90 days)

Nov 13

The paper proposes the Janus-MOPD framework, which tracks student learning dynamics from the perspective of internal geometric representations to reveal the deeper causes of domain heterogeneity.

Unverified50%
Nov 13

Janus-MOPD finds that the student exhibits a three-stage learning dynamic of expansion-consolidation-optimization (ECO).

Unverified50%
Nov 13

In the expansion stage, the student's representation space shows increased representation variance and rising effective dimensionality; in the consolidation stage, it shows a stabilizing representation distribution and increased intra-class compactness; in the optimization stage, it shows subtle directional adjustments and subspace alignment.

Unverified50%
Nov 13

The learning process does not proceed linearly but goes through three stages of capability expansion, stable consolidation, and fine-grained optimization, each corresponding to a different pattern of representation space change.

Unverified50%
Nov 13

In terms of spatial dimensions, the researchers summarized two types of subspace locking phenomena: intra-domain and cross-domain.

Unverified50%
Nov 13

Janus-MOPD's geometric tracking method provides a diagnostic tool that can be used to analyze bottlenecks in multi-domain training and to guide data allocation and loss function design.

Unverified50%

All Facts (6)

Source Articles