NeurIPS Rejected, Now Targeting ICLR: How Much Should You Actually Change Based on Reviewer Feedback?

Every year after NeurIPS decisions drop, academia witnesses a massive "paper migration" — rejected authors pack their bags and head toward ICLR or the next upcoming deadline. A researcher recently sparked a discussion on Reddit's machine learning community, raising a question almost every researcher faces: When resubmitting, how much of the NeurIPS reviewer feedback do you actually incorporate?

The question sounds mundane, but it touches on paper revision strategy, the limitations of peer review, and academic mental health all at once. Drawing from the original thread, here are several dimensions worth thinking through carefully.

Full Adoption vs. Selective Incorporation

The core question the original poster raised was: when facing NeurIPS reviewer feedback, do people try to "address every single comment," or do they "selectively revise only the critiques they find reasonable and valuable"?

This is the classic dilemma of paper revision. Accepting everything might seem safe and signal good faith to reviewers, but it can also pull the paper in too many directions and dilute its original focus. Selective adoption, on the other hand, demands that authors have a clear sense of their own work — you need to know which criticisms are genuinely on target and which stem from reviewers misunderstanding the paper.

Notably, the original poster also raised a more contentious scenario: Has anyone deliberately ignored certain reviewer suggestions because you felt they would push the research in the wrong direction? This point is particularly important. Reviewers are not omniscient, and their suggestions can sometimes diverge from the author's true research intent. Blindly complying may compromise the paper's academic integrity, while standing firm requires a convincing argument — either in the rebuttal or in the revised version.

"Lack of Novelty" Is the Hardest Criticism to Address

The original poster specifically focused on papers that were questioned for novelty or significance, listing a range of typical reviewer phrasings:

• "The contribution is incremental"
• "Not sufficiently different from prior work"
• "The empirical gains don't justify the proposed method"
• "The problem itself isn't significant enough"
• "Theoretical contribution is limited"
• "Interesting idea, but unclear what the broader impact is"

These critiques are notoriously difficult to address because they can't simply be fixed by running more experiments. Judgments like "incremental" or "not sufficiently different" are fundamentally subjective assessments of a work's value. To truly respond, authors may need to restructure their paper's narrative, more clearly delineate differences from prior work, or re-establish the weight of their contribution through stronger experiments and deeper analysis.

In other words, novelty critiques don't test your ability to patch things up — they test your ability to retell a research story more compellingly. Often the problem isn't that the research itself is insufficient, but that the paper failed to communicate its value clearly.

At the structural level, top venues have no unified standard for judging "novelty," and there are significant differences in perception across research areas and individual reviewers. Top conferences like NeurIPS typically use double-blind review, where reviewers score without knowing the authors' identities. This reduces reputation bias to some degree, but also makes reviewers more reliant on the paper's own narrative to assess value. Research has shown that the same paper submitted to different reviewers often yields accept and reject decisions in roughly equal proportions — NeurIPS ran a famous consistency experiment in 2021 that revealed reviewer disagreement rates approaching randomness. This means that a significant portion of "insufficient novelty" judgments are noise rather than signal. Authors need to distinguish: is this a problem with how the work was expressed, or a mismatch in the reviewer's conceptual framework? The former can be improved by rewriting the related work comparison and contribution statements; the latter is more about submission venue selection and audience targeting.

Managing Pressure Under Tight Deadlines

The original post also touched on a very real pain point: ICLR and other upcoming conference deadlines often fall shortly after NeurIPS decisions, so how do you make meaningful revisions in such a short window?

This is the most easily overlooked yet most agonizing part of the resubmission process. The window available for authors to digest reviewer feedback, redesign experiments, and rewrite the paper is extremely limited. Under this kind of pressure, the "selective adoption" strategy mentioned earlier becomes even more pragmatic — invest limited time in revisions that most improve acceptance odds while staying aligned with the research direction, rather than trying to address everything.

Pressure management isn't just about time allocation; it's also a mindset issue. Post-rejection frustration and deadline anxiety can both degrade the quality of your judgment. The "😭" emoji the original poster used reflects a sentiment shared across the entire academic community.

For context: NeurIPS typically releases decisions in late November to early December, while ICLR submission deadlines historically fall in early October. This means that if authors want to resubmit a paper rejected from NeurIPS to ICLR in the same year, the window has already closed. The more common resubmission path is: submit to NeurIPS (May deadline) → get rejected → resubmit to ICLR in the fall of the same year (October deadline), giving roughly four to five months for revisions. While this is more comfortable than "rushing to a deadline the moment results drop," four months is still quite tight if you need to redesign experiments or substantially restructure the paper. Understanding this timeline helps authors more rationally prioritize revisions after receiving NeurIPS feedback, rather than falling into the anxious cycle of "wanting to fix everything but having time for nothing."

What Can We Take Away from This Discussion?

While this was an open-ended community question without a definitive conclusion, several emerging points are worth keeping in mind:

Reviewer feedback is a reference, not a mandate. Whether to adopt it should be based on whether the criticism is sound and whether it aligns with your research direction — not mechanical compliance.

Novelty critiques are often a narrative problem. Rather than panicking about "the work not being new enough," reflect on whether you've clearly communicated the contribution.

Under time constraints, focus on the main issues. When deadline pressure hits, prioritize the core criticisms most likely to affect acceptance.

For researchers caught in the "NeurIPS rejection → ICLR resubmission" cycle, perhaps the greatest value of this discussion is the reminder that you're not alone — and that responding to reviewer feedback rationally and strategically, rather than accepting or rejecting it wholesale, is the mark of a mature scientific approach.