Too Few Defect Samples in Industrial Inspection: Anomaly Detection vs. Supervised Learning — A Practical Guide

A Counterintuitive Challenge in Industrial Inspection

There's a paradox at the heart of AI deployment in industrial quality control: the more stable the production line, the harder it is to train a reliable model.

A computer vision engineer shared this exact bottleneck on Reddit: his line was running well, but out of hundreds of thousands of parts, only around 200 real defects had been recorded — spread across six or seven defect types, with some categories containing fewer than 20 samples. For a classical supervised segmentation model, that's essentially nothing to work with.

This isn't an edge case. Any mature manufacturing line faces the same problem: defects are rare events by definition, yet AI needs large volumes of examples to learn. Delivering a usable inspection system with almost no defect samples is a challenge that nearly every industrial AI project must eventually confront.

Four Common Approaches — and Their Pitfalls

When facing severe class imbalance, the industry typically turns to a handful of strategies. Each comes with real costs.

Path 1: Anomaly Detection Using Only Normal Samples

This is the most intuitive approach — train exclusively on normal samples so the model learns "what normal looks like," then flag anything that deviates as anomalous. Algorithms like PaDiM and PatchCore are classic examples; they require no defect samples at all.

The problem is false positives. Anomaly detection defines "anomalous" too broadly: a speck of dust on the lens, a part positioned at a slightly off angle — all of it gets flagged. Models that look great in the lab quickly overwhelm QA teams with false alarms on a real production line. The original poster put it bluntly: the false positive rate in real-world deployments is "rough."

Path 2: Synthetic Defect Data Augmentation

This means artificially painting cracks or scratches onto good-part images to generate defect samples. It sounds promising, but synthetic defects often look noticeably different from real ones. As the author pointed out directly: the model may end up learning "does this region look pasted on?" rather than "is there actually damage here?"

In other words, synthetic data introduces new artifacts, and the model learns to exploit them — actually hurting generalization.

Path 3: Buying or Scraping More Defect Data

This path is largely a dead end in industrial settings. Defects are highly specific to the part and the process — scratches on someone else's aluminum housing are simply not the same as scratches on your product. External data transfers poorly and offers limited value.

Path 4: Waiting to Accumulate More Data

This is the most honest answer, but also the least practical. At current defect rates, building a sufficient dataset might take 18 months — and most projects need to demonstrate value far sooner.

The Pragmatic Trade-off: Drop Sub-classification, Go Binary

A key question raised in the original post is worth sitting with: for defect categories with only 20 samples, is it even worth trying to model them individually?

The author leaned toward a more pragmatic solution — collapsing all defects into a single binary classification problem (defective / not defective), and delegating the actual categorization to human review after the fact.

The logic is straightforward:

• 20 samples simply cannot support a reliable multi-class model. Forcing fine-grained classification just introduces noise and overfitting.
• Binary classification reduces task complexity, allowing the limited defect samples to be used more effectively.
• While losing classification detail hurts reporting and analytics, for the core goal of catching defects on the line, "catch it first, classify it later" is often the only honest approach.

As the author put it: "Losing classification will hurt reporting, but with this little data, it might be the only honest choice."

This reflects a broader engineering principle: when data is scarce, don't ask the model to do more than it's capable of. Break the problem apart — let AI handle the coarse screening it's good at, and reserve the judgment calls and few-shot pattern recognition for humans.

Bringing Anomaly Detection False Positives to an Acceptable Level

If you go with anomaly detection, the central challenge becomes false positive control. Based on community experience, several approaches are worth exploring:

Tightly Control Input Consistency

Many false positives come from variables that have nothing to do with defects — lighting shifts, part orientation, lens contamination. Before images ever reach the anomaly detection model, use fixed lighting, mechanical positioning, and image registration to eliminate these sources of variation. Separating "environmental anomalies" from "product anomalies" first can dramatically reduce false alarm rates.

Apply Post-Processing Rules to Filter Noise

Models like PatchCore output anomaly heatmaps. Post-processing rules — such as minimum anomaly region area thresholds or connected-component constraints — can filter out sparse, unstructured noise responses and retain only spatially coherent signals that are likely to represent real defects.

Design a Human-in-the-Loop Threshold Strategy

Rather than chasing full automation, build a tiered system: high-confidence normal parts pass automatically, clear defects are rejected automatically, and borderline cases are routed to human review. This bounds missed-detection risk while keeping manual workload manageable.

Practical Recommendations for Similar Industrial Inspection Projects

Putting it all together, here's a decision framework for industrial inspection scenarios with almost no defect samples:

1. Prefer anomaly detection over supervised learning — you have an abundance of good parts, not defects.
2. Treat false positive reduction as the top engineering priority — and address it at the data collection level (lighting, positioning), not just in the algorithm.
3. Adopt binary classification pragmatically — don't force sub-classification on low-sample categories; delegate categorization to humans.
4. Be skeptical of synthetic data — unless you can verify that the model has actually learned real defect features rather than synthetic artifacts.
5. Design a human-AI collaboration workflow — AI handles coarse screening; humans handle nuanced judgment and classification reporting.

The real difficulty in deploying industrial AI rarely comes down to squeezing out the last fraction of a percent in model accuracy. It comes from delivering, under the triple constraints of data scarcity, cost, and time, a system that QA teams actually trust and production lines actually adopt. Sometimes, an honest binary classifier is worth far more than a flashy multi-class model that falls apart in the real world.