[KongchangAI]
· 2 min read· 1,116 words

Janus-MOPD: Geometric Tracking of Student Learning Dynamics in Multi-Teacher Distillation

Janus-MOPD: Geometric Tracking of Student Learning Dynamics in Multi-Teacher Distillation

Janus-MOPD reveals the spatiotemporal mechanisms behind students' difficulty in balancing multi-domain capabilities in multi-teacher distillation from a geometric representation perspective.

This article introduces the Janus-MOPD framework, which aims to explain why student models struggle to simultaneously master multi-domain capabilities in multi-teacher on-policy distillation (MOPD). Existing research mostly analyzes from the angles of uneven external supervision efficiency and hard assignment, while this work turns to the learning dynamics of the student's internal representations. In the temporal dimension, students go through three stages—Expansion-Consolidation-Optimization (ECO)—with inherent limitations in learning efficiency and scalability; in the spatial dimension, intra-domain and cross-domain subspace locking phenomena are identified, explaining why learning directions are clear yet cross-domain conflicts exist. Janus-MOPD provides a unified formulation of learning dynamics and a geometric tracking methodology, offering a theoretical basis for improving multi-teacher distillation algorithms and designing training strategies, and serving as a practical tool for diagnosing bottlenecks in multi-domain training.

Multi-teacher on-policy distillation (MOPD) is a key post-training technique for building multi-domain large reasoning models (LRMs), but student models often struggle to master skills across multiple domains simultaneously. A recent paper proposes the Janus-MOPD framework, which tracks student learning dynamics from the perspective of internal geometric representations, revealing the deeper causes of domain heterogeneity.

rss source: Janus-MOPD

The Challenge of Multi-Teacher Distillation: Why Do Students Struggle to Balance Multiple Domains?

Multi-domain capability is essential for practical large reasoning models, making multi-teacher on-policy distillation (MOPD) an important step in the post-training pipeline, with the goal of integrating the capabilities of multiple domain-expert teachers into a single student model. However, due to domain heterogeneity, students typically cannot acquire multi-domain capabilities simultaneously. Existing research has attempted to address this issue from the angles of uneven temporal efficiency of external multi-teacher supervision and spatial hard assignment, but the extent of performance degradation in broader multi-domain training remains unexplained.

This paper shifts the focus to how students internally learn and integrate multi-domain capabilities, proposing Janus-MOPD—a general geometric representation learning dynamics framework for large reasoning models.

On-Policy Distillation is a variant of knowledge distillation whose core idea is that the student model learns during training using its own generated trajectories (rather than fixed teacher-generated data), with teacher models providing token-level supervision signals over these online trajectories. Compared with traditional offline distillation, on-policy distillation can alleviate the mismatch between training data and the student distribution, but it also introduces new challenges: multiple teachers provide supervision of varying strength across different domain samples, and the student must balance multi-source signals within a dynamically generated data stream. In multi-teacher settings, each teacher typically corresponds to a domain expert, and the student must simultaneously fit multiple potentially conflicting distributions, which exacerbates optimization complexity.

Large Reasoning Models (LRMs) refer to large language models that possess multi-step reasoning capabilities through large-scale reinforcement learning or chain-of-thought training. Unlike general conversational models, LRMs emphasize strong performance on tasks requiring long-chain reasoning, such as mathematics, code, and logic. Building multi-domain LRMs typically requires fusing the capabilities of multiple domain experts, and multi-teacher distillation is precisely the post-training means to achieve this goal. Understanding the sources of LRM reasoning capability helps clarify why domain heterogeneity significantly affects distillation outcomes—different reasoning tasks may have fundamentally different structural requirements for the representation space.

Temporal Dimension: The Three-Stage Expansion-Consolidation-Optimization Dynamics

In the temporal dimension, the study finds that students exhibit a three-stage learning dynamic of Expansion-Consolidation-Optimization (ECO), accompanied by inherent limitations in learning efficiency and scalability. Specifically, the learning process does not proceed linearly but goes through three stages: capability expansion, stable consolidation, and fine-grained optimization, each corresponding to different patterns of representation space change. This finding helps understand the internal causes of performance fluctuations in multi-domain training.

Geometric tracking of representation space refers to methods that characterize the evolution of model capabilities by analyzing the distribution, dimensionality, direction, and other geometric properties of internal hidden representations (such as the last-layer hidden states or specific intermediate-layer activations). Common techniques include principal component analysis (PCA), singular value decomposition, representational similarity analysis (CKA), and subspace angle metrics. In Janus-MOPD, geometric tracking is used to observe changes in the student's representation space over training steps: the expansion stage manifests as increased representation variance and rising effective dimensionality; the consolidation stage manifests as stabilized representation distributions and improved intra-class compactness; the optimization stage manifests as subtle directional adjustments and subspace alignment. This approach does not rely on external evaluation metrics but directly seeks signals of learning dynamics from within the model, thereby revealing the structural causes behind performance curves.

Spatial Dimension: Subspace Locking and Cross-Domain Conflict

In the spatial dimension, by tracking the student's representation space, the researchers summarize two types of subspace locking phenomena—intra-domain and cross-domain—and explain why students have clear learning directions yet inherent cross-domain conflicts. Subspace locking means that students tend to map capabilities from different domains onto specific subspaces in the representation space, making it difficult to flexibly transfer and share across domains, thereby causing performance degradation.

Subspace locking describes the phenomenon in which a model encodes capabilities from different domains into relatively fixed and separated subspaces in the representation space during training. Once the representation subspace for a domain forms in early training, subsequent training finds it difficult to reallocate it or share it with other domains, leading to reduced utilization of the representation space. Intra-domain locking refers to samples within the same domain tending to occupy a compact subspace, while cross-domain locking refers to overlap or abnormal angles between subspaces of different domains. This locking gives rise to cross-domain conflict: when the subspace directions of two domains are close but their semantic requirements differ, gradient updates interfere with each other, manifesting as performance improvement in one domain at the expense of another. The concept of subspace locking draws on research on feature subspaces and task interference in representation learning, providing a geometric perspective for understanding negative transfer in multi-task learning.

Contributions and Implications of Janus-MOPD

Janus-MOPD provides a unified formulation of learning dynamics, a geometric tracking methodology, and systematic findings on how students learn and integrate multi-domain capabilities. Overall, this work tracks the continuous evolution of the model's own multi-domain capabilities from within through a Janus perspective. These findings provide a theoretical basis for improving multi-teacher distillation algorithms and designing more effective training strategies, and may inspire future research on model interpretability and continual learning.

For researchers and engineers working on post-training of large models, the geometric tracking method of Janus-MOPD offers a diagnostic tool that can be used to analyze bottlenecks in multi-domain training and guide data allocation and loss function design. As the demand for multi-domain models grows, such internal dynamics research will become increasingly important.

One-line Summary

Janus-MOPD reveals from a geometric representation perspective the spatiotemporal mechanisms that make it difficult for students to balance multi-domain capabilities in multi-teacher distillation.

Paragraph Summary

This article introduces the Janus-MOPD framework, which aims to explain why student models struggle to simultaneously master multi-domain capabilities in multi-teacher on-policy distillation (MOPD). While existing research mostly analyzes from the angles of uneven external supervision efficiency and hard assignment, this work turns to the learning dynamics of the student's internal representations. In the temporal dimension, students go through three stages—Expansion-Consolidation-Optimization (ECO)—accompanied by inherent limitations in learning efficiency and scalability; in the spatial dimension, intra-domain and cross-domain subspace locking phenomena are identified, explaining why learning directions are clear yet cross-domain conflicts exist. Janus-MOPD provides a unified formulation of learning dynamics and a geometric tracking methodology, offering a theoretical basis for improving multi-teacher distillation algorithms and designing training strategies, and serving as a practical tool for diagnosing bottlenecks in multi-domain training.

#multi-teacher distillation#on-policy distillation#large reasoning models#Janus-MOPD#learning dynamics#geometric representation
Share:

Related articles