α-RC Metric
The α-RC metric is a family of normalized concentration measures based on Rényi divergence. It compares normalized minor magnitudes against a uniform distribution, takes values in the interval [0,1] and is independent of network architecture, addressing the scale dependency problem of the PR metric.
Timeline (last 90 days)
The paper's authors propose a class of α-RC metrics based on Rényi divergence, which normalize the concentration measure to the [0,1] interval and are independent of network architecture.
The α-RC metrics are bounded: regardless of network architecture, all α-RC metrics fall within the [0,1] interval; 0 means magnitude is uniformly distributed across all minors, and 1 means magnitude is entirely carried by a single minor.
The α-RC metrics are monotonic: metric values are non-decreasing with α.
When α=2, α-RC reduces to an explicit transformation of the Participation Ratio, thereby eliminating scale dependence while retaining PR's interpretability.
The authors trained a single-hidden-layer MLP on synthetic datasets and verified the effectiveness of the α-RC metrics through computational experiments, which show that the new metrics can more clearly reflect the concentration of a network's internal representations and are unaffected by network scale.
The paper's experimental scale is limited, using only synthetic datasets and a single-hidden-layer MLP; experiments on real data have not yet been conducted.
All Facts (6)
The paper's authors propose a class of α-RC metrics based on Rényi divergence, which normalize the concentration measure to the [0,1] interval and are independent of network architecture.
50%UnverifiedThe α-RC metrics are bounded: regardless of network architecture, all α-RC metrics fall within the [0,1] interval; 0 means magnitude is uniformly distributed across all minors, and 1 means magnitude is entirely carried by a single minor.
50%UnverifiedThe α-RC metrics are monotonic: metric values are non-decreasing with α.
50%UnverifiedWhen α=2, α-RC reduces to an explicit transformation of the Participation Ratio, thereby eliminating scale dependence while retaining PR's interpretability.
50%UnverifiedThe authors trained a single-hidden-layer MLP on synthetic datasets and verified the effectiveness of the α-RC metrics through computational experiments, which show that the new metrics can more clearly reflect the concentration of a network's internal representations and are unaffected by network scale.
50%UnverifiedThe paper's experimental scale is limited, using only synthetic datasets and a single-hidden-layer MLP; experiments on real data have not yet been conducted.
50%