eCMI
A refined variant of the Conditional Mutual Information (CMI) framework that measures information leakage of a hypothesis output by a learning algorithm at evaluation points—specifically, the mutual information between the prediction and whether the test point belongs to the training set or a shadow set.
Timeline (last 90 days)
An arXiv paper (arXiv:2610.10600) proves that a learning algorithm can be constructed whose evaluated conditional mutual information (eCMI) reaches O(d) order in the realizable case, thereby recovering the optimal PAC guarantees for VC classes.
eCMI measures the mutual information between the prediction of the hypothesis output by the algorithm on a random test point and the information of whether that test point belongs to the training set or the shadow set.
Steinke and Zakynthinou proved that generalization error can be upper-bounded by some function of eCMI.
The choice of 5 rather than 3 base learners is related to the technical conditions for achieving a precise O(d) upper bound in the eCMI information-theoretic computation, as fewer votes may incur constant factor losses.
All Facts (4)
An arXiv paper (arXiv:2610.10600) proves that a learning algorithm can be constructed whose evaluated conditional mutual information (eCMI) reaches O(d) order in the realizable case, thereby recovering the optimal PAC guarantees for VC classes.
50%UnverifiedeCMI measures the mutual information between the prediction of the hypothesis output by the algorithm on a random test point and the information of whether that test point belongs to the training set or the shadow set.
50%UnverifiedSteinke and Zakynthinou proved that generalization error can be upper-bounded by some function of eCMI.
50%UnverifiedThe choice of 5 rather than 3 base learners is related to the technical conditions for achieving a precise O(d) upper bound in the eCMI information-theoretic computation, as fewer votes may incur constant factor losses.
50%