Neutrosophic Ensemble Classification: Teaching Bearing Fault Detection to "Know What It Doesn't Know"

Why Traditional Fault Detection "Confidence Scores" Are Unreliable

When rolling bearings in industrial equipment fail, they often bring an entire production line to a halt. Machine learning classifiers have become the dominant tool for bearing fault detection — but the scalar confidence scores they output have a fundamental flaw: they conflate "confidently wrong" predictions with "genuinely ambiguous" ones.

A new study published on arXiv (arXiv:2610.06880) points out that the true/false probability pairs (F = 1 − T) commonly used in traditional classification frameworks are algebraically redundant — since the false probability is entirely determined by the true probability, the two provide no independent information. In other words, when a model says "I'm 95% sure," you can't tell whether that number reflects solid evidence or blind overconfidence when facing an unfamiliar operating condition. This is precisely the most dangerous scenario in industrial deployment.

Decomposing Uncertainty into Four Dimensions

The core contribution of this research is applying a "refined neutrosophic decomposition" to an ensemble model consisting of Random Forest, XGBoost, and Logistic Regression, producing four mutually independent indicators:

• T-hat: Evidence strength for the top-ranked class (how much evidence supports the model's most favored category)
• F-hat: Evidence strength for the strongest competitor (how strong the runner-up is)
• I1-hat: Predictive entropy, measuring how dispersed the overall distribution is
• I2-hat: Decision disagreement, measuring how consistently the individual models within the ensemble agree

The value of this decomposition lies in distinguishing different sources of uncertainty. A high F-hat signals the presence of a closely competing class, while a high I1-hat indicates overall weak evidence. Compared to a single scalar confidence score, this four-dimensional characterization allows downstream systems to make more targeted decisions about when to trigger an alert versus when to escalate to human review.

Tested Against Two Rigorous Benchmark Datasets

The study validates its approach on two public bearing datasets: the Case Western Reserve University dataset (CWRU) and the Jiangnan University dataset (JNU, at speeds of 600–1000 rpm). A critical aspect is the use of a leave-one-condition-out protocol — the model never sees a particular load or speed during training, and is then tested specifically on that unseen operating condition. This is far more demanding than the common random train/test split, and much closer to the reality of industrial equipment operating under constantly changing conditions.

Neutrosophic Logic is an extended logical framework proposed by mathematician Florentin Smarandanche in the 1990s. Unlike classical binary logic (true/false) or fuzzy logic (continuous truth values), neutrosophic logic splits the evaluation of any proposition into three independent components: Truth (T), Falsity (F), and Indeterminacy (I), whose sum is not forced to equal 1. This enables simultaneous representation of three distinct epistemic states: evidence in support, evidence against, and absence of evidence. This shares similarities with Dempster-Shafer evidence theory but provides a more flexible algebraic structure. The "refined neutrosophic decomposition" in this study further splits I into I1 (entropy, reflecting global distributional spread) and I2 (intra-ensemble disagreement), forming a four-dimensional representation that allows "evidence conflict" and "evidence absence" — two fundamentally different sources of uncertainty — to be treated separately, providing more granular input for downstream decisions such as alert prioritization and human review thresholds.

The Case Western Reserve University Bearing Dataset (CWRU) is the most widely cited public benchmark in fault diagnosis, released by CWRU's Department of Electrical Engineering around 2000. It covers four states — normal, inner race fault, outer race fault, and rolling element fault — collected under four load conditions (0, 1, 2, and 3 horsepower) at 12 kHz or 48 kHz sampling rates. Its accessibility and clear class structure have made it the de facto standard for benchmarking bearing fault algorithms, but this also creates a risk of overfitting — many studies report near-perfect accuracy under random splits without accounting for cross-load generalization. The Jiangnan University Dataset (JNU) is collected across a wider speed range (600/800/1000 rpm) with greater variation in operating conditions, more closely reflecting variable-speed industrial operation. Cross-speed generalization on JNU is significantly harder than on CWRU. The contrast between the two datasets neatly exposes the widespread pattern of "acing the easy test, collapsing on the hard one."

Honest Results: Brilliance and Collapse Side by Side

What makes this paper most commendable is that it honestly reports the boundary conditions under which the method fails, rather than cherry-picking favorable numbers.

On CWRU, the researchers first corrected a file-to-class mapping error, after which the ensemble model achieved 100% accuracy on three of the four held-out load conditions, and 92.27% on the fourth. Ironically, this created an awkward situation: too few errors remained to enable any meaningful uncertainty analysis.

On JNU with 1000 rpm held out, things deteriorated sharply — the ensemble accuracy collapsed to 40.64%, falling below the majority-class baseline. Interestingly, standalone Logistic Regression (57.91%) generalized far better than the tree-based ensemble. Further analysis showed that I1-hat (predictive entropy) had a robust association with errors beyond T-hat/F-hat, while I2-hat (decision disagreement) contributed little. Even more candidly, the researchers acknowledge that standalone Logistic Regression confidence scores outperformed the full four-dimensional decomposition in certain cases, and they explicitly document this as a boundary condition of the method.

What Actually Works: Cross-Domain Fusion and Physical Features

Beyond the neutrosophic decomposition itself, the paper delivers two additional results of real value.

JS Divergence Fusion of Time and Frequency Domain Models

The researchers fused time-domain and frequency-domain models trained on the same signal, using the Jensen-Shannon divergence between them as an uncertainty measure. This consistently outperformed single-model entropy: on the standard split, the Area Under the Risk Coverage curve (AURC) was 0.29 versus 0.36; on the harder single-condition holdout, it was 0.54 versus 0.73. More critically, in the distribution-shift comparison between CWRU and JNU, this was the only indicator that correctly reflected distributional drift. This suggests that having two different perspectives on the same signal cross-check each other captures genuine epistemic uncertainty better than a single model's internal entropy.

Jensen-Shannon (JS) divergence is a symmetrized version of the Kullback-Leibler (KL) divergence, used to measure the similarity between two probability distributions. KL divergence is asymmetric (KL(P||Q) ≠ KL(Q||P)) and tends toward infinity when the supports don't overlap, while JS divergence avoids these issues by averaging the KL divergence of each distribution against their mixture, yielding a result always bounded within [0, ln2] with better numerical stability. In this study, the time-domain model and frequency-domain model each produce a predictive probability distribution over the same vibration signal. When the two perspectives strongly agree, JS divergence approaches 0, reflecting model confidence; when they diverge significantly, JS divergence rises, signaling that the signal lies near the model's knowledge boundary. This cross-perspective "mutual interrogation" mechanism captures out-of-distribution samples more effectively than a single model computing entropy from its own output distribution.

The Power of Physically Interpretable Features

On the CWRU dataset alone, the researchers used literature-validated bearing fault characteristic frequencies, correctly demodulated via envelope spectrum analysis, and achieved near-perfect separation of most fault classes using just three interpretable features (99.57% accuracy). This is a reminder that in the age of deep learning, physics-based signal processing methods remain highly competitive in specific contexts — and come with built-in interpretability.

Implications for Industrial AI Deployment

The significance of this work extends beyond a new classification metric. Its rigorous experiments convey several important messages:

First, high accuracy under training conditions does not imply generalization — the collapse on JNU from near-perfect to below-baseline is a powerful warning against trusting laboratory metrics. Second, there is no universal key to uncertainty quantification — a complex four-dimensional decomposition does not necessarily outperform a simple baseline, and the method's applicability boundaries must be honestly documented. Third, cross-perspective validation (time domain + frequency domain) and physics-based features may be a more reliable path forward than stacking more models.

The researchers have made all code, logs, and figures publicly available for independent reproduction — and that transparency itself is a mark of maturity in industrial AI research.