Rényi Divergence
Rényi divergence is a family of measures of difference between probability distributions introduced by Alfréd Rényi, parameterized by an order α. As α→1 it reduces to KL divergence, at α=2 it is related to χ² divergence, and as α→∞ it becomes more sensitive to the largest differences.
Timeline (last 90 days)
The paper's authors propose a class of α-RC metrics based on Rényi divergence, which normalize the concentration measure to the [0,1] interval and are independent of network architecture.
Rényi divergence is a family of measures of the difference between probability distributions, proposed by Alfréd Rényi and parameterized by order α.
When α→1, Rényi divergence reduces to KL divergence; when α=2, it is related to χ² divergence; when α→∞, it is more sensitive to the maximum difference between distributions.
All Facts (3)
The paper's authors propose a class of α-RC metrics based on Rényi divergence, which normalize the concentration measure to the [0,1] interval and are independent of network architecture.
50%UnverifiedRényi divergence is a family of measures of the difference between probability distributions, proposed by Alfréd Rényi and parameterized by order α.
50%UnverifiedWhen α→1, Rényi divergence reduces to KL divergence; when α=2, it is related to χ² divergence; when α→∞, it is more sensitive to the maximum difference between distributions.
50%