Contextual Bias in Agricultural CNNs: Hidden Neuron Concept Induction Reveals Interpretable Features

Concept induction reveals interpretable visual attributes of CNN hidden neurons on contextual patches, finding middle depths most interpretable and bias strength uncorrelated with interpretability.
This study employs concept induction to derive interpretable visual attribute logic expressions from hidden neuron activation patterns, validated across five agricultural datasets, four CNN architectures, three depths, and five patch positions. Results show 27.4% of neuron-feature pairs pass statistical tests, with middle-depth validation (40.2%) significantly higher than deep (12.0%), and edge, Gabor, and Tamura texture features most prominent. Patch position has no dominant effect, and contextual bias strength is not positively correlated with neuron interpretability: the moderately biased Sugarcane dataset has the highest rate (40.5%), while highly biased Cassava has the lowest (19.2%). This suggests concept induction can complement traditional bias detection, helping diagnose model-reliant visual attributes and informing more robust vision system design.
Convolutional Neural Networks (CNNs) sometimes rely on background-dominated image patches during image classification—patches that contain almost no target object yet can still influence the model's judgment. This phenomenon indicates that CNNs exploit contextual information, but prior research has mostly detected this behavior through classification accuracy, with little understanding of what the hidden layers actually represent. A recent study attempts to use Concept Induction methods to reveal the interpretable visual attributes corresponding to hidden neuron activations on contextual image patches, offering a new perspective for understanding CNN decision mechanisms.

The Gap Between Contextual Bias and Hidden Layer Representations
It is nothing new that CNNs exploit background information during classification. Prior research has found that models can classify background-dominated image patches with considerable accuracy, suggesting that models may learn spurious correlations in the data. However, most of these studies remain at the behavioral level: whether and to what extent a model relies on context is typically assessed through accuracy or saliency maps. What hidden-layer neurons actually "see" in contextual image patches, and whether these activations correspond to interpretable visual concepts, has lacked systematic exploration.
This study aims to fill that gap. The authors propose that concept induction can derive logical expressions from neuron activation patterns, thereby describing which visual features are associated with neuron activation. If these expressions are validated on held-out images, it means we have found a reliable connection between neuron activation and interpretable visual attributes. This reveals more about the internal representations of the model than merely observing classification accuracy.
Experimental Design: Multiple Datasets, Architectures, Depths, and Positions
To comprehensively examine this issue, the study used five agricultural datasets, each exhibiting high, medium, or low degrees of contextual bias. In agricultural scenes, crops are often strongly associated with backgrounds (such as soil, weeds, and sky), which easily induces models to take shortcuts, making these datasets an ideal testbed for studying contextual bias.
The study covered four CNN architectures and analyzed them across three network depths (shallow, middle, deep) and five image patch positions (e.g., center, corners, etc.). For each image patch, the authors computed various quantitative image descriptors, including edge features, Gabor features, and Tamura texture features, and encoded these descriptors into a knowledge graph as background knowledge for concept induction. The concept induction algorithm then derived logical expressions characterizing neuron activation. These expressions were subsequently statistically validated on held-out images using the Mann-Whitney U test and Cliff's delta to confirm their generalizability.
The advantage of this design is that it does not rely on manually annotated concepts but automatically mines interpretable feature combinations from data, while controlling for false positives through statistical testing.
Concept Induction is a method for automatically learning interpretable concept descriptions from data. Its core idea is: given a set of positive examples (e.g., image patches with high activation for a neuron) and negative examples (patches with low activation), along with a set of predefined background knowledge (e.g., morphological features, texture descriptors), search for logical rules that can distinguish positive from negative examples through Inductive Logic Programming (ILP). These rules are typically presented as first-order logic clauses, such as "if an image patch has high edge density and coarse texture, then the neuron activates." Unlike black-box models, the rules produced by concept induction are human-readable and can be statistically validated on new data to assess their generalizability. In this study, the authors encoded quantitative image descriptors (edge, Gabor, Tamura, etc.) as facts in a knowledge graph, and the concept induction algorithm derived logical expressions characterizing neuron activation conditions, providing a symbolic explanation for neuron representations.
The Mann-Whitney U test is a non-parametric statistical test used to determine whether two independent samples come from the same distribution. It does not require data to follow a normal distribution and is suitable for ordinal data or continuous but non-normal data. In this study, the authors labeled image patches with neuron activation as positive examples and those without as negative examples, then compared the value distributions of the two groups on a given image descriptor. If the test is significant, it indicates an association between that descriptor and neuron activation. Cliff's delta is an effect size measure that quantifies the magnitude of the difference between two distributions, ranging from [-1, 1], with larger absolute values indicating more pronounced differences. By combining these two statistical tools, researchers can not only determine whether an association is significant but also assess the strength of the association, thereby more reliably validating whether the logical expressions obtained through concept induction have generalizability.
Validation Results: Middle Depths Most Interpretable, Texture Features Prominent
Of all 48,013 tested neuron-feature pairs, 13,164 (27.4%) passed validation. This proportion indicates that a substantial portion of neuron activations can indeed be described by interpretable visual features, but not all—deep-layer neurons may encode more abstract information that is difficult to capture with simple descriptors.
Validation rates differed markedly across network depths: the average validation rate at middle depths was 40.2%, compared to only 12.0% at deep layers. This aligns with intuitive understanding: middle-layer neurons tend to be sensitive to mid-level features such as texture and edges, which happen to be easily captured by image descriptors; deep-layer neurons may correspond to higher-level semantic concepts that are difficult to characterize with simple texture descriptors.
Among the various descriptors, edge features, Gabor features, and Tamura texture features had the highest validation rates. This indicates that neuron activation is most closely associated with these low-to-mid-level visual features, suggesting that contextual bias may be largely driven by texture and edge information.
Interestingly, image patch position did not show dominance across all datasets. That is, regardless of whether patches were taken from the center or corners, validation rates depended more on depth and feature type than on position. This finding challenges the simple assumption that "models primarily rely on the central region."
Gabor features are classic features used for image texture analysis, obtained by convolving an image with a set of Gabor filters at different frequencies and orientations. Each filter corresponds to an edge or texture pattern at a specific scale and orientation, so Gabor features can capture local orientation and frequency information in images and are commonly used in tasks such as texture classification and face recognition. Tamura texture features, on the other hand, are defined from the perspective of human visual perception and comprise six texture properties: coarseness, contrast, directionality, line-likeness, regularity, and roughness. Among these, coarseness, contrast, and directionality are the most commonly used, describing respectively the granularity, light-dark contrast, and directional uniformity of textures. In this study, these descriptors were used to quantify the visual attributes of image patches as input features for concept induction, thereby revealing the association between neuron activation and low-to-mid-level visual features.
Contextual Bias Strength Is Not Positively Correlated with Neuron Interpretability
A counterintuitive finding is that stronger contextual bias does not imply higher neuron-feature validation rates. Among the five datasets, the Sugarcane dataset with moderate bias had the highest validation rate (40.5%), while the Cassava dataset with high bias had the lowest (19.2%).
This result suggests that behavioral-level contextual dependence and the semantic interpretability of hidden neurons may capture different aspects of CNN behavior. A model may exhibit strong contextual dependence on high-bias data, yet its internal neuron activations may be difficult to describe with simple visual features; conversely, moderately biased datasets may encourage the model to learn richer, more interpretable mid-level features. This dissociation implies that evaluating models solely based on classification accuracy or bias detection is insufficient; neuron-level interpretability analysis is also needed.
Concept Induction as a Complementary Tool for Bias Detection
The study's ultimate conclusion is that concept induction can complement patch-based bias detection methods. Traditional bias detection focuses on whether a model relies on context, while concept induction further reveals which visual attributes the model depends on. This combination helps diagnose model vulnerabilities more comprehensively and provides a basis for designing more robust agricultural vision systems.
For example, if a model is found to rely heavily on Gabor features at middle-layer neurons, and there are spurious correlations between background textures and class labels in the training data, targeted data augmentation or regularization can be applied to reduce the model's over-reliance on texture.
Of course, this study has limitations. The overall validation rate is not high (27.4%), indicating that many neuron activations still cannot be explained by existing descriptors; moreover, the study only covers agricultural datasets and four CNN architectures, and whether the conclusions generalize to other domains and more advanced models (such as Transformers) remains to be verified. Nevertheless, it opens a new window for understanding contextual bias in CNNs: moving from behavioral observation to statistical characterization of internal representations.
For AI researchers and engineers, the takeaway is: evaluating model bias cannot rely solely on outputs; it is also necessary to delve into hidden layers and use interpretability tools to reveal the signals the model truly depends on. Concept induction offers an automated, statistically verifiable pathway that is worth trying in more scenarios.
One-line Summary
Concept induction reveals interpretable visual attributes of CNN hidden neurons on contextual image patches, finding that middle depths are most interpretable and that bias strength is uncorrelated with interpretability.
Paragraph Summary
This study uses concept induction to derive logical expressions of interpretable visual attributes from hidden neuron activation patterns, validated across five agricultural datasets, four CNN architectures, three depths, and five image patch positions. Results show that 27.4% of neuron-feature pairs pass statistical testing, with validation rates at middle depths (40.2%) significantly higher than at deep layers (12.0%), and edge, Gabor, and Tamura texture features being most prominent. Image patch position has no dominant effect, and contextual bias strength is not positively correlated with neuron interpretability: the moderately biased Sugarcane dataset has the highest validation rate (40.5%), while the highly biased Cassava has the lowest (19.2%). This indicates that concept induction can serve as a complementary tool to traditional bias detection, helping diagnose the visual attributes a model depends on and providing a basis for designing more robust vision systems.
Related articles

Gemini 4 Neon Leak: A Potential Strong Contender Against Opus 5.5
A Hacker News thread suggests Gemini 4 Neon looks like an Opus 5.5 competitor. This article unpacks the product positioning behind this community signal, shifts in the top-model competitive landscape, and what it means for developers' model selection strategy.

Claude Fable 5.2's Self-Authored Short Film 'Fable' Sparks Discussion
A short film 'Fable,' credited to Claude Fable 5.2 for script, visuals, and directing, appears on Hacker News. This article analyzes the significance of full-pipeline AI creation, community reactions, and what it means for content creators.

Anthropic IPO Pricing: A Signal of AI Giants Going Public
Anthropic's IPO pricing draws attention. This article analyzes its potential impact on AI industry valuation, commercialization paths, and the competitive landscape, and highlights public-market expectations and challenges for AI companies.