MoE Routing Telemetry Exposes Privacy: Expert Selection Leaks Training Membership Information

When Routing Information Becomes a New Privacy Attack Surface

Mixture-of-Experts (MoE) architecture has become the mainstream approach for scaling large language model parameters. During inference, MoE models generate substantial routing information — decision data recording which experts handle each token. This information is typically logged for operational purposes such as monitoring, debugging, load analysis, and security auditing.

A newly published arXiv paper, "When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry", argues that this seemingly innocuous operational telemetry actually opens up an entirely new privacy attack surface. The researchers posed a core question: can routing telemetry reveal whether a given sample was used to fine-tune a deployed model? The answer is yes.

The core idea behind MoE architecture is to decompose a large neural network into multiple relatively independent "expert" sub-networks, with a lightweight router (gating network) deciding which experts process each input token. In typical implementations, the model activates only a small subset (usually top-2) from dozens or even hundreds of experts, decoupling parameter scale from computational cost. Frontier models including Mixtral, DeepSeek^Ad, and Switch Transformer all adopt this architecture. Routing decisions themselves form a unique "computational fingerprint" — the same input triggers different expert combinations depending on the model's state, which is the physical basis for routing telemetry carrying meaningful information.

Router-Augmented Membership Inference Attack

The paper's core contribution is a router-augmented membership inference attack. Traditional membership inference attacks primarily rely on output-side signals — such as prediction confidence and loss values. The innovation here lies in combining traditional output-side signals with aggregated routing features.

The approach works as follows: the attacker independently trains several "shadow models" that simulate the target model's fine-tuning behavior, then uses those shadow models to learn a membership classifier, which is subsequently applied to the target model. The additional dimensionality provided by routing telemetry allows the attacker to more accurately determine whether a given sample appeared in the training set.

Consistent Results Across Architectures and Domains

The study was validated across three MoE architectures and three data domains (nine combined settings in total). Results show that, compared to an already-strong output-side ensemble baseline, incorporating routing telemetry consistently improved attack performance across all nine settings: at a 1% false positive rate (FPR), the true positive rate (TPR) improved by 2.7 to 9.4 percentage points. This consistency suggests the leakage is not incidental but a structural characteristic of MoE architecture.

Membership Inference Attack (MIA) is a classic paradigm in privacy auditing, first proposed by Shokri et al. in 2017. Its fundamental assumption is that model behavior on training samples (e.g., lower loss, higher confidence) is distinguishably different from behavior on non-training samples. The shadow model method is the mainstream MIA implementation — attackers train multiple shadow models with the same structure but different data to simulate the target model's training process, thereby obtaining labeled "member/non-member" signals for training a classifier. This paper's innovation is introducing routing activation patterns as an additional feature dimension into this framework, so that even when output-side signal differences are subtle, routing-side pattern differences can provide complementary information to the classifier.

Leakage Persists Across Multiple Fine-Tuning Paradigms

A particularly concerning finding is that this privacy leakage is highly robust. The researchers tested multiple common fine-tuning approaches, and leakage remained consistently observable across the following scenarios:

• Full fine-tuning
• Frozen-router training — even when the router parameters themselves are not updated
• LoRA (Low-Rank Adaptation) — currently the most popular parameter-efficient fine-tuning method
• Instruction tuning

More critically, the attack remains effective under information-constrained conditions: even when the attacker can only obtain discrete expert selection results (rather than full routing weights), limited telemetry data, or even just a single shadow model, leakage persists. This significantly lowers the bar for executing the attack.

Mechanistic Analysis: Leakage Stems from Hidden Representations, Not Router Memory

The most insightful portion of the paper is its mechanistic analysis. Intuitively, one might assume the router "memorizes" training samples — but the research demonstrates that leakage does not require router-specific memory.

The actual mechanism is: the fine-tuning process injects membership information into the model's hidden representations, and the router effectively projects these representations — even when its parameters are frozen, it still "projects out" this membership signal and exposes it in routing decisions.

In other words, the router itself is innocent; it faithfully reflects pre-existing differences in the underlying representations that carry traces of membership information. This explains why frozen-router training cannot prevent leakage.

Limitations of Defensive Measures

The study also tested mitigation through perturbation of telemetry data. The results are sobering: perturbation only reduces this additional leakage when it significantly degrades the fidelity of the telemetry data. This means there is a direct trade-off between defense and the practical value of telemetry — preserving routing information for monitoring and debugging makes it difficult to fully eliminate privacy risks.

Hidden representations refer to the vector encodings of input data at intermediate layers of a neural network. Fine-tuning is essentially a refinement of the pre-trained representation space to better adapt the model to a specific task or data distribution. Research shows this adjustment process leaves statistically distinguishable "memory traces" in the representations — that is, training sample representations exhibit subtle but stable distributional deviations from unseen sample representations. The router operates by applying a linear transformation to these hidden representations and selecting the top-K results, which makes it a natural "representation probe": any information embedded in the representations will be partially reflected in routing decisions. This mechanism is independent of whether the router itself was fine-tuned, explaining the counterintuitive experimental result that a frozen router still leaks information.

Differential Privacy (DP) is currently the most mature privacy protection framework for the machine learning training phase, providing provable privacy guarantees by injecting calibrated noise into gradients or outputs. However, the defensive experiments in this paper target post-hoc perturbation of telemetry data, not DP protection during training. The fundamental difference between the two approaches is: training-phase DP directly suppresses the formation of membership information in representations, while telemetry perturbation only masks signals that already exist. This means combining differentially private training with telemetry access controls may offer greater defensive depth than perturbing telemetry alone — though DP training typically comes at the cost of model performance degradation, and deployment decisions still require balancing utility against privacy risk.

Practical Implications for MoE Deployment

This research transforms a purely operational signal into a privacy threat, with several practical implications for teams deploying MoE models:

1. Telemetry data needs to be reassessed for sensitivity. Routing logs should not be treated as ordinary operational data — they carry a view into the model's internal computation and may leak membership information about training data.
2. Parameter-efficient fine-tuning is not a privacy moat. Lightweight fine-tuning methods like LoRA still embed membership information in representations and cannot be relied upon to evade privacy scrutiny.
3. Access control is more reliable than data perturbation. Since perturbing telemetry costs its practical utility, a more pragmatic solution may be enforcing strict access controls and audit trails on routing telemetry, rather than hoping that adding noise will suffice.

As MoE architecture becomes increasingly prevalent in frontier models, privacy analyses targeting architectural characteristics deserve ongoing attention from security and compliance teams. Any observable projection of a model's internal computation can become a source of information in an adversary's hands.