Unverified50% confidenceFactExact time
In the two-neuron ReLU teacher-student model, the organization of gradient dynamics is dominated by two directions: the bisector direction carries the main signal and is easy to learn, while the splitting direction controls specialization and is hard to learn.
1
Sources
50%
Confidence
Long-term
Relevance
11/2/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedGradient flow trajectories show that the student neurons first collapse to near the saddle point along the bisector direction, then escape and specialize along the splitting direction to the corresponding teacher neurons.76% similarUnverifiedCamerata proposed the concept of "recursive generalization": neurons branch and grow along related directions during training, and each branch has a provenance rather than being forcibly stuffed in as superposition.70% similarUnverifiedNeural networks exhibit permutation symmetry—reordering the neurons in a given layer leaves the network's functionality completely unchanged, as long as the weights of adjacent layers are adjusted accordingly.70% similarUnverifiedThe paper uses a structured three-layer example to illustrate how a small number of trainable first-layer neurons can exploit the fixed downstream block through nonlinear combination.70% similarUnverifiedWhen a neuron splits, having the two child neurons each inherit half of the parent neuron's output weights (e.g., outgoing edge weights multiplied by 0.5) and keeping their parameters identical at the initial moment preserves the network function unchanged at the instant of splitting.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1125634API
curl https://kongchang.com/api/v1/knowledge/claims/1125634MCP
get_claim(id=1125634)