How to Find Research Gaps in ML/LLM/CV: A Systematic Methodology and Direction Guide

A Undergraduate's Research Dilemma

Recently, a B.Tech CSE (Computer Science and Engineering) undergraduate posted on Reddit seeking advice, sparking quite a bit of discussion. Their request was very specific: rather than looking for generic project ideas, they wanted to find limitations or failure cases empirically observed in recent papers, which could then be developed into a research paper worthy of an A/B-tier conference.

In computer science, academic conferences (rather than journals) are the primary venue for publishing results—a stark contrast to fields like physics or chemistry. A/B-tier conferences typically refer to the ranking system recommended by the China Computer Federation (CCF). A-tier conferences like NeurIPS, ICML, CVPR, and ACL have acceptance rates around 20%-25% and represent the highest level in their respective fields; B-tier conferences like AAAI, IJCAI, and ECCV also carry significant academic influence. A paper accepted at an A-tier conference often carries more weight in academic evaluation than many SCI Q1 journal papers. For undergraduates, publishing at a B-tier or higher conference is already an outstanding research achievement.

Behind this request were several key constraints: a 1.5-year Capstone project with a 4-person team, aiming to produce meaningful academic output. Capstone Projects are a widely adopted graduation project format in engineering universities worldwide, particularly common in CS programs in India (B.Tech degree system), the US, and Singapore. Unlike traditional individual thesis projects, Capstone emphasizes team collaboration, project management, and industry/research-grade output. They typically last 1-2 years, involve teams of 3-5 people, and are guided by academic advisors. This scenario is remarkably common across universities globally—students don't lack execution ability; what they lack is a methodology for finding real problems worth solving.

reddit source: Looking for research gap in Ml/LLM/ Computer vision

This article doesn't offer "standard answers." Instead, it systematically outlines how to scientifically identify research gaps in three major directions—Machine Learning (ML), Large Language Models (LLMs), and Computer Vision (CV)—while pointing out several directions worth exploring in depth based on the current state of the art.

Why "Finding Research Gaps" Is a Discipline in Itself

From "Reading Papers" to "Reading the Cracks in Papers"

Many beginners only focus on methods and results when reading papers, but real research opportunities are often hidden in three places:

The Limitations section: Nearly every top-conference paper honestly states at the end, "Our method fails under X conditions." These self-disclosed weaknesses are natural entry points for follow-up work.
The Future Work section: Items the authors explicitly list as "things we didn't get to" are often low-hanging fruit.
Anomalies in experimental results: A baseline performing unexpectedly under certain data distributions, or a metric collapsing on a specific subset—these details are often glossed over, yet they harbor deeper issues.

Using "Reproduction" to Verify Whether Problems Are Real

For a 1.5-year project, investing the first 3 months in reproducing recent papers is absolutely worthwhile. When you actually get someone else's code running, you'll discover that results that "look perfect" in the paper are often fragile—switch the dataset, tweak hyperparameters, change the input distribution, and performance drops dramatically. This "reproduction-driven discovery" is the most solid starting point for research and far more reliable than brainstorming a topic from scratch.

It's worth noting that the "Reproducibility Crisis" in machine learning is already an openly discussed topic in academia. Multiple studies have shown that a significant proportion of paper results cannot be fully reproduced—sometimes due to insufficiently reported hyperparameter sensitivity, sometimes due to subtle differences in data preprocessing, and sometimes because the choice of random seeds happened to produce favorable results. For this reason, reproduction itself is a valuable academic contribution, and some top conferences (such as NeurIPS) even have dedicated Reproducibility Tracks.

Potential Research Gaps in Three Major Directions

Large Language Models (LLMs)

LLMs are currently the hottest and most competitive direction, but numerous unsolved challenges remain, many of which are well-suited for team collaboration:

Detectability and Attribution of Hallucinations: Why do models hallucinate? Can we identify "uncertain" tokens during the generation process? "Hallucination" in LLMs refers to the model generating content that appears fluent and plausible but is actually incorrect or entirely fabricated. The root cause is closely tied to LLMs' training mechanism: the model is essentially performing next-token probability prediction, optimizing for statistical coherence of language rather than factual accuracy. Hallucinations can be broadly categorized into two types—factual hallucinations (fabricating non-existent facts) and faithfulness hallucinations (contradicting given context). Current mitigation strategies include Retrieval-Augmented Generation (RAG), Reinforcement Learning from Human Feedback (RLHF), and incorporating external knowledge verification during inference, but none fundamentally solve the problem. Since 2024, token-level uncertainty estimation and automatic hallucination attribution have become new research hotspots—for example, probing the model's internal hidden states to predict which tokens might be hallucination artifacts. There is still significant room for interpretability research in this area.
Actual Utilization Rate of Long Contexts: Many models claim to support ultra-long contexts, but in practice they exhibit the "Lost in the Middle" phenomenon. This finding originates from a 2023 paper of the same name by Stanford University (Liu et al.), which showed that when key information is placed in the middle of a long text, performance of nearly all mainstream LLMs drops significantly. Models tend to better utilize information at the beginning and end of text, while attention allocation to middle sections is severely insufficient. This phenomenon is related to inherent biases in the Transformer architecture's attention mechanism—although self-attention can theoretically attend to any position, positional biases in training data and extrapolation limitations of positional encoding create a gap between theoretical capability and actual performance. Even models claiming to support 128K or even million-token context windows often reveal weaknesses in mid-section retrieval during Needle-in-a-Haystack tests. Systematically measuring and improving how models utilize information in the middle of long texts is an empirically tractable research topic.
Reasoning Capability Boundaries of Small Models: In resource-constrained scenarios, where exactly do distilled or fine-tuned small models "break" in multi-step reasoning? Knowledge Distillation is a model compression technique formally proposed by Hinton et al. in 2015. The core idea is to use the output distribution of a large "teacher model" to guide the training of a small "student model." In the LLM domain, this technique is widely used to transfer capabilities from models like GPT-4 and Claude to models with orders-of-magnitude fewer parameters. However, the distillation process exhibits clear capability degradation, especially in multi-step reasoning, Chain-of-Thought, and complex logical inference tasks. Small models can often learn the surface format of reasoning, but starting around the 3rd-4th step in a reasoning chain, logical "fractures" appear—where causal relationships between consecutive steps no longer hold. Systematically characterizing the conditions under which these fractures occur (number of reasoning steps, problem complexity, domain specificity, etc.) is a direction highly suitable for empirical research.

Computer Vision (CV)

After years of development in CV, the space for purely chasing benchmarks is shrinking, but directions combining robustness and cross-modal approaches remain active:

Out-of-Distribution (OOD) Generalization: Vision models often experience cliff-like performance drops outside their training distribution. OOD Generalization is one of the most fundamental challenges in machine learning, directly determining whether models can transition from the lab to real-world deployment. Traditional supervised learning assumes that training and test data come from the same distribution (the i.i.d. assumption), but in reality, this assumption almost never holds—an autonomous driving model may encounter weather conditions never seen in its training set, and a medical imaging model may face images from different scanner models. OOD detection (determining whether input comes from the training distribution) and OOD generalization (maintaining performance under distribution shift) are two related but distinct research directions. In recent years, Domain Generalization, Invariant Risk Minimization (IRM), and causal inference-based methods have become mainstream research paradigms, but in the era of large-scale visual foundation models (such as CLIP and DINOv2), the effectiveness of these methods is being reexamined. Building more realistic failure case libraries and analyzing their root causes is a valuable research direction.
Adversarial Robustness and Real-World Perturbations: Beyond the adversarial examples commonly used in academia, model behavior under real perturbations like lighting changes, occlusion, and sensor noise deserves in-depth study. Adversarial attack methods commonly used in academia, such as PGD and FGSM, produce perturbations that are nearly imperceptible at the pixel level but can completely break a model. However, challenges in real deployment often come from more "natural" perturbations—rain blur, backlit overexposure, dirty lenses, partial object occlusion, etc. The mechanisms behind these two types of perturbations are fundamentally different, and existing models' resistance to them is inconsistent. Studying the relationship between the two and how to address them in a unified manner is a direction with both theoretical value and practical significance.
Fine-Grained Understanding in Vision-Language Models: Current multimodal models frequently err on fine-grained tasks such as counting, spatial relationships, and attribute binding. Vision-Language Models (VLMs) like GPT-4V, Gemini, and LLaVA, while impressive at image captioning and visual question answering, have been shown by benchmarks like Winoground and ARO (Attribute, Relation, Order) to suffer from three core issues: first, attribute binding errors, where the model cannot correctly associate attributes like color and size with specific objects (e.g., confusing "a red cup next to a blue ball" with "a blue cup next to a red ball"); second, weak spatial relationship understanding, with accuracy on "above/below, left/right, near/far" judgments far below human levels; third, counting deficiency, where models frequently err when determining the number of objects in an image. These deficiencies are believed to be related to the training objective of CLIP-style contrastive learning—the coarse-grained objective of image-text matching cannot drive the model to learn compositional semantics. Each of these specific failure modes can be individually decomposed into a paper.

Traditional Machine Learning (ML)

Even in the deep learning era, classical ML still has areas worth exploring:

Data-Efficient Learning: In scenarios with scarce labeled data, where are the actual benefit boundaries of active learning, semi-supervised, and self-supervised methods? Active Learning reduces annotation costs by intelligently selecting the most informative samples for labeling; Semi-supervised Learning leverages large amounts of unlabeled data to assist learning from limited labeled data; Self-supervised Learning relies entirely on the structure of the data itself to learn representations. Each of these three paradigms has its applicable scenarios, but in practice, their benefits are often highly dependent on data characteristics, task type, and domain features—blindly applying them may yield minimal gains. Systematically comparing the actual performance of these methods against theoretical expectations across different scenarios is itself a meaningful empirical study.
Model Calibration: Do the model's predicted confidence scores truly reflect its accuracy? Model calibration refers to the consistency between a model's output probabilities/confidence scores and its actual accuracy. A perfectly calibrated model, when it says "I'm 90% confident," should indeed be correct 90% of the time. However, deep neural networks (especially modern large models) commonly suffer from overconfidence—even when making incorrect predictions, the model may output extremely high confidence scores. This issue was systematically revealed by Guo et al. in their 2017 classic paper "On Calibration of Modern Neural Networks." In safety-critical domains like medical diagnosis, autonomous driving, and financial risk management, poorly calibrated models can lead to catastrophic consequences. Temperature Scaling, Platt Scaling, and Mixup are commonly used post-hoc calibration methods, but how to achieve intrinsic calibration during training remains an open question. The calibration problem is especially important in safety-critical domains and deserves deep investigation.

Practical Advice for Research Teams

Start Narrow, Go Deep Rather Than Broad

1.5 years may sound long, but for a 4-person team targeting a conference paper, it's not that generous. Avoid overly broad topics at all costs. Rather than thinking "I want to solve LLM hallucination," focus on something like "quantifying the trigger conditions for a specific type of hallucination in medical Q&A scenarios." The more specific the problem, the easier it is to produce publishable results within a limited timeframe.

Here's a practical litmus test for topic selection: if your research question can't be articulated in a single sentence, or if you can't design a first experiment within two weeks to preliminarily test your hypothesis, then the problem is likely not specific enough. Good research questions should be falsifiable, measurable, and have clear evaluation criteria.

Leverage Tools to Track the Research Frontier

Teams should subscribe to daily updates from relevant arXiv categories, use Papers with Code to track SOTA changes, and follow workshops at top conferences (NeurIPS, ICML, CVPR, ACL, etc.)—workshop papers often reveal emerging pain points beyond mainstream directions.

Papers with Code is an open platform maintained by Meta AI that links academic papers with their code implementations, datasets, and leaderboards. Researchers can use the platform to track SOTA trends on specific tasks, discovering which methods are being surpassed and which fields are stagnating. Beyond this, the research frontier tracking ecosystem includes: Semantic Scholar's Research Feeds (AI-based paper recommendations), Connected Papers (citation relationship visualization), arXiv Sanity (an arXiv paper filtering tool originally developed by Andrej Karpathy, now community-maintained), and Hugging Face's Daily Papers. For undergraduate teams, building a systematic literature tracking habit is far more effective than occasional intensive reading sessions.

Divide Work Wisely and Iterate Quickly

A 4-person team can adopt a role-rotation mechanism across "literature review–experimentation–writing." In the early phase, focus all firepower on scanning literature and reproducing baselines; in the middle phase, lock onto 1-2 specific problems to dig deep; in the final phase, concentrate on writing and supplementary experiments. Sync regularly with your advisor and adjust direction promptly to avoid spending too much time on the wrong path.

More specifically, a reference timeline would be: Months 1-3 for literature survey and direction screening, while simultaneously reproducing 2-3 core papers; Months 4-6 for finalizing the research question, designing the experimental plan, and completing preliminary experiments; Months 7-12 for in-depth experiments, result analysis, and iterative method improvement; the final 6 months for concentrated writing, supplementary comparison experiments and ablation studies, submission, and revision based on reviewer feedback. It's worth emphasizing that writing should not be left until the end—recording findings and thoughts alongside experiments will dramatically reduce writing pressure later on.

Conclusion

Finding research gaps is fundamentally a mental shift from "accepting established conclusions" to "questioning established conclusions." For the undergraduate on Reddit, and for all researchers just getting started, perhaps the best advice is: get your hands dirty with reproduction first, then look for problems in the cracks. Truly valuable topics are rarely conjured from thin air—they're "stumbled upon" through repeated engagement with code, data, and papers.

One final note: don't set an unreasonably high mental bar for "originality." A significant amount of valuable work in academic research is incremental—making measurable improvements to existing methods, validating established findings in new scenarios, providing deeper empirical analysis of known problems—these are all contribution types recognized by the academic community. What matters is whether your work moves the field's understanding of a problem forward by a step, even if it's just a small one.