Which ML Subfields Are Fading Into Irrelevance? A Community Debate

Machine learning evolves at a breathtaking pace — and the flip side of rapid iteration is that once-celebrated research directions can fall into obscurity within just a few years. A recent discussion thread in the Reddit machine learning community put a sharp question on the table: which ML subfields are becoming irrelevant?

The thread struck a nerve, and in doing so revealed the complicated emotions researchers feel when confronting sunk costs. For anyone preparing to enter the field, this kind of honest reflection is well worth reading.

Neural Architecture Search: 3,000 Models, No Transformer

The first example the original poster raised was Neural Architecture Search (NAS). Citing a survey paper, they noted that over roughly five years, the NAS field produced more than 3,000 newly proposed model architectures, consuming computational resources on an astronomical scale.

The irony, of course, is that Transformer — the architecture that truly reshaped the entire field — was not discovered through NAS. It was designed by human researchers. After Transformer's rise, NAS quietly faded from the mainstream spotlight.

This observation points to a deeper question worth sitting with: when a field consumes enormous compute and still fails to produce a defining breakthrough, does that suggest a fundamental limitation in its research paradigm? NAS was predicated on using automated search to replace human intuition about architecture design — yet the most important advance came from human insight. That contrast is telling.

Neural Architecture Search emerged around 2017, with the core idea of algorithmically searching a predefined space for optimal neural network structures, replacing the labor-intensive process of manual architecture design. Early landmark work like Google Brain's NASNet required thousands of GPU-hours across hundreds of devices to find architectures that performed well on specific tasks. To reduce search costs, researchers later proposed weight-sharing methods like DARTS (Differentiable Architecture Search), compressing search time to days on a single GPU. Critics, however, pointed out that these approaches are highly sensitive to how the search space is defined — and that defining the search space still requires human judgment, which undermines much of the "automation" claim. The deeper issue is that NAS optimizes within a known vocabulary of architectural design choices, while Transformer's breakthrough came from introducing an entirely new inductive bias — the attention mechanism — a paradigm-level innovation that lies outside the coverage of any fixed search space.

Adversarial ML: 9,000 Papers and Got Nowhere

The second example the poster raised was even more pointed. They referenced a talk by Nicholas Carlini, one of the most prominent researchers in adversarial machine learning, which featured a slide titled: "9,000 papers and got nowhere."

The poster admitted they couldn't think of any concrete real-world applications for adversarial ML beyond "possibly making attackers smarter" — since all the attack techniques are now published in the open literature.

This critique touches on a genuine paradox in security research: by advancing theoretical understanding, attack-and-defense research simultaneously provides a toolkit for potential adversaries. Adversarial example research has continued for years, yet mainstream deep learning systems haven't seen robustness improvements that are anywhere near commensurate with the volume of papers published. When a field's output primarily consists of "finding new vulnerabilities" rather than "building reliable defenses," its practical value is legitimately open to question.

The concept of adversarial examples was formally introduced by Szegedy et al. in 2013: by adding imperceptible perturbations to input images, one could cause high-accuracy image classifiers to output wrong labels with high confidence. This discovery sparked widespread academic interest, spawning a large body of attack methods (FGSM, PGD attacks) and defenses (adversarial training, randomized smoothing) across vision, text, and speech models. Nicholas Carlini himself is a major contributor to the field; his C&W attack remains a standard benchmark for evaluating defenses. The fundamental difficulty lies in an asymmetry between offense and defense: an attacker only needs to find one effective perturbation, while a defender must be robust to all possible attacks — and adversarially trained models typically pay a cost in standard accuracy. This structural disadvantage, combined with limited industrial demand for adversarial robustness in real deployments, has created a visible gap between the field's academic output and its real-world impact.

ML Ethics and Fairness: A Question of Priorities

The poster's most controversial argument targeted ML ethics, bias, and fairness research. Their claim: in an era where people are seriously discussing "extinction risk" from AI, worrying about bias and fairness feels out of step with the moment.

They posed a pointed rhetorical question: when most people are being displaced from their jobs by AI, what does it even mean to enforce bias and fairness? From their perspective, rather than studying fairness, the field should be opening up a new subfield called "ML-induced extinction."

This is clearly a highly personal position, and an easy one to push back against. Ethics and fairness research is concerned with concrete harms being inflicted on real people right now — hiring discrimination, credit bias, inequitable risk assessments in the justice system. These problems don't disappear because a larger, longer-horizon risk also exists. Framing "near-term harm" and "long-term risk" as competing priorities is itself a contestable framing. That said, the poster's frustration does reflect a genuine debate within the research community about where resources should be directed.

"It'll Come Back Eventually" — Rational Judgment or Emotional Defense?

The final section of the post is arguably its most valuable contribution: a methodological observation about the nature of pushback. The poster noted that whenever someone suggests a research direction is becoming obsolete, the response is typically emotional rather than logical — people claim that any method will eventually "have its moment," pointing to SVMs, Linear Discriminant Analysis (LDA), and Markov chains as classics that might once again become the next big thing in ML.

The poster's response was a sharp analogy: he wasn't denying that vacuum tubes might one day come back into fashion — but perhaps we shouldn't be investing our efforts in researching vacuum tubes right now.

The analogy cuts to the core of the issue. Research resources are finite. The relevant question isn't whether a direction is "absolutely incapable of making a comeback" — it's whether, given current technology and current priorities, it represents the best use of available effort.

The SVMs and Markov chains the poster references are genuinely instructive cases, reflecting a recurring narrative in ML of "classic method revivals." SVMs were the dominant approach for classification tasks before deep learning's rise, and they still see stable use in small-sample, high-dimensional structured data domains like bioinformatics — but they're no longer considered a research frontier. Markov chains and their variants (HMMs, MCMC) remain active as foundational tools for probabilistic inference and have gained renewed theoretical attention in the analysis of diffusion models. The emotional appeal of the "comeback" narrative lies in what it offers researchers who have invested heavily in a particular direction: psychological protection. The sunk cost fallacy from behavioral economics applies directly here — prior learning costs cause people to overestimate the probability of future returns from that direction. The vacuum tube analogy is designed to break that cognitive bias: the possibility of revival exists, but it isn't sufficient justification for current resource allocation.

What This Debate Means for Newcomers

Setting aside the arguments about specific subfields, the real value of this discussion is the reminder it offers to anyone entering ML: don't treat a field's historical prestige as a guarantee of its future value.

A few takeaways worth carrying:

• Be alert to mismatches between compute and output. A direction that has produced a mountain of papers isn't necessarily solving real problems. The NAS and adversarial ML examples show that paper count and actual impact can be wildly disconnected.
• Focus on paradigms, not trends. Transformer came from human insight, not automated search — a reminder that genuine breakthroughs often come from rethinking the nature of the problem itself.
• Distinguish "outdated" from "foundational." Classic methods like SVMs and Markov chains may no longer be research frontiers, but they remain important foundational tools. "No longer cutting-edge" doesn't mean "without value."
• Think critically about ethics research. It may not be on the technical frontier toward AGI, but it addresses the direct real-world consequences of AI systems being deployed today. Dismissing it as irrelevant isn't as straightforward as it might seem.

It bears emphasizing: all of these perspectives originate from a single Reddit discussion thread, representing the subjective views of individual researchers rather than any industry consensus. Any claim that a particular subfield is "dead" should be weighed carefully against a broader body of evidence.