Rényi Divergence
A generalized family of divergences defined on probability distributions, controlled by a parameter α. As α approaches 1 it reduces to the KL divergence, and as α approaches infinity it approaches the maximum divergence.
Timeline (last 90 days)
Rènyi differential privacy (RDP) is defined based on Rènyi divergence, provides tighter composition of privacy loss than pure differential privacy, and is widely used in machine learning and statistical analysis.
Apart from a few cases such as the Gaussian mechanism, computing the Rènyi divergence usually has no closed-form solution, and numerical computation faces the challenge of high-dimensional integrals.
The paper innovatively adopts the escort distribution to construct the KSD; the escort distribution can highlight the regions of the distribution relevant to privacy loss, making the KSD more sensitively capture changes in the Rènyi divergence.
The authors design a kernel function based on the escort distribution so that the KSD becomes a computable upper bound on the Rènyi divergence, and the tightness of the certificate can be adjusted by choosing different kernel functions.
The paper gives exact identities for the Gaussian mechanism and the Laplace mechanism, namely that the KSD certificate exactly equals the Rènyi divergence (under a specific kernel); for the exponential mechanism it gives an upper bound.
Rènyi divergence is a generalized family of divergences defined on probability distributions, controlled by a parameter α (α>0, α≠1); as α→1 it degenerates to the KL divergence, and as α→∞ it approaches the maximum divergence.
The escort distribution reweights via p(x)^β / Z (β being a parameter), enlarging or shrinking the high-density regions of the original distribution so that the KSD is more sensitive to the exponential term in the Rènyi divergence.
All Facts (7)
Rènyi differential privacy (RDP) is defined based on Rènyi divergence, provides tighter composition of privacy loss than pure differential privacy, and is widely used in machine learning and statistical analysis.
50%UnverifiedApart from a few cases such as the Gaussian mechanism, computing the Rènyi divergence usually has no closed-form solution, and numerical computation faces the challenge of high-dimensional integrals.
50%UnverifiedThe paper innovatively adopts the escort distribution to construct the KSD; the escort distribution can highlight the regions of the distribution relevant to privacy loss, making the KSD more sensitively capture changes in the Rènyi divergence.
50%UnverifiedThe authors design a kernel function based on the escort distribution so that the KSD becomes a computable upper bound on the Rènyi divergence, and the tightness of the certificate can be adjusted by choosing different kernel functions.
50%UnverifiedThe paper gives exact identities for the Gaussian mechanism and the Laplace mechanism, namely that the KSD certificate exactly equals the Rènyi divergence (under a specific kernel); for the exponential mechanism it gives an upper bound.
50%UnverifiedRènyi divergence is a generalized family of divergences defined on probability distributions, controlled by a parameter α (α>0, α≠1); as α→1 it degenerates to the KL divergence, and as α→∞ it approaches the maximum divergence.
50%UnverifiedThe escort distribution reweights via p(x)^β / Z (β being a parameter), enlarging or shrinking the high-density regions of the original distribution so that the KSD is more sensitive to the exponential term in the Rènyi divergence.
50%