From Log-Odds to Shapley Values: The Interpretable Geometry of Weighted Naive Bayes

When Classifiers Need to Explain Themselves

Model interpretability has long been one of the most persistent challenges in machine learning. Even for a relatively simple, theoretically well-grounded classifier like Naive Bayes, articulating why it makes a particular prediction still demands a rigorous mathematical framework. A new paper on arXiv — From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier — takes a methodological approach to building a complete "explanation space" for the weighted Naive Bayes classifier.

The paper's core contribution is establishing a formal connection among three things: the model-induced supervised representation, local explanations, and predictive behavior. This connection isn't derived from empirical observation — it's guaranteed by a precise mathematical identity.

From Conditional Log-Likelihoods to Log-Odds

Traditionally, Naive Bayes classifiers define a canonical supervised distance based on conditional log-likelihoods. This distance measures similarity within a generative probabilistic framework, but it doesn't directly correspond to classification decisions — because the model's final judgment depends on relative comparisons between class probabilities.

The paper proposes a discriminative reformulation, replacing log-likelihoods with log-odds as the foundation. Log-odds — the log ratio of one class's probability to another's — are inherently tied to the decision boundary. Compared to generative log-likelihoods, this representation is much closer to the core question of how the model actually makes its choices.

This shift might seem like a technical detail, but it's the pivotal hinge of the entire paper. Moving from a generative to a discriminative perspective allows the subsequent geometric interpretation to map directly onto classification behavior, rather than remaining at the level of data distribution description.

Generative vs. Discriminative — These represent two fundamentally different modeling philosophies in probabilistic classification. Generative models learn the data distribution P(X|Y) for each class and recover the posterior via Bayes' theorem; discriminative models directly model P(Y|X) or the decision boundary, without caring how the data was generated. Classical Naive Bayes is a generative method, so its supervised distance naturally uses conditional log-likelihoods as its basis. Log-odds — log[P(Y=1|X) / P(Y=0|X)] — are the natural discriminative measure: their sign directly determines the classification outcome, their magnitude reflects decision confidence, and they play a central role in discriminative models like logistic regression. Rewriting the Naive Bayes representation space from log-likelihoods to log-odds is equivalent to shifting the same model from "describing how data is generated" to "understanding how the model makes decisions" — a necessary prerequisite for building an interpretable geometry.

The Exact Coincidence of Distance and Shapley Values

The paper's most striking result is a proof that the distance induced by the log-odds representation exactly equals the ℓ1 distance between analytical Shapley value vectors.

Shapley values originate from cooperative game theory and have been widely adopted for feature attribution in machine learning, with the core idea of fairly distributing each feature's contribution to a prediction. In the explainable AI space, methods like SHAP have become mainstream tools. What this paper does is derive analytical Shapley values for the specific case of the weighted Naive Bayes classifier, and then prove that they correspond precisely to the supervised distance.

The significance of this identity is that the geometric structure induced by the model inherently carries interpretability information. In other words, the distance between two samples in this space directly reflects the magnitude of their difference at the level of feature attribution. This provides a formal interpretive reading of the model's geometry — explanation is no longer appended after the fact, but embedded within the model's representation.

Shapley Values — Originally introduced by Lloyd Shapley in 1953 in the context of cooperative game theory, Shapley values address the question of how to fairly distribute the total value created by multiple participants. The core idea: a participant's contribution equals their average marginal contribution across all possible orderings of participation. Translated to machine learning, each "participant" is a feature, the "total value" is the model's prediction output, and Shapley values measure each feature's average contribution to a specific prediction. Because they satisfy four axioms — efficiency, symmetry, dummy, and additivity — Shapley values are considered the theoretically most "fair" form of feature attribution. SHAP (SHapley Additive exPlanations) is their most widely used practical implementation, but for complex models it typically relies on Monte Carlo sampling for approximation, which is computationally expensive. One of this paper's key contributions is providing a closed-form analytical solution for the weighted Naive Bayes case — eliminating approximation error and dramatically reducing computational cost.

Validating the Theory with k-NN

Beyond the theoretical framework, the paper also conducts an empirical comparison. The authors use a k-nearest neighbors (k-NN) classifier as a test platform, comparing the performance of several supervised distances derived from these representations.

The choice of k-NN is well-motivated: distance-based classifiers are highly sensitive to the choice of distance metric, making them ideal for directly testing how different distances perform in practice. Through this setup, the paper places all three levels — supervised distance, local explanation, and predictive behavior — within the same experimental framework, empirically confirming the tight connections among them.

It's worth noting that the paper explicitly frames itself as work conducted "primarily from a methodological perspective." It aims for theoretical rigor and conceptual unification, not for pushing performance numbers on large-scale benchmarks.

Implications for Explainable AI Research

The value of this research is largely in providing a geometric pathway for understanding interpretability. For a long time, feature attribution methods (like Shapley values) and a model's internal representations have typically been treated as separate research objects. This paper shows that, at least for the weighted Naive Bayes family, the two can be unified through a precise mathematical relationship.

This unification opens several potential directions. First, when explanations can be derived analytically from model representations, both computational efficiency and theoretical guarantees are stronger — no sampling approximations needed. Second, the "distance as explanation" perspective might inspire other model families to search for similar geometry-attribution correspondences. Third, it transforms interpretability from an add-on module into an intrinsic component of understanding a model's geometric structure.

Of course, as a methodological paper focused on a single model family, whether these conclusions generalize to more complex modern models (such as deep neural networks) remains an open question. But the approach it demonstrates — understanding explanation by starting from decision geometry — offers a meaningful reference point for the broader explainable AI field.