Top Conference Paper Quality in Crisis? ICLR Reviewer Says Submissions Aren't Even Workshop-Worthy

A researcher who reviews for both ICLR and AAMAS took to Reddit to state bluntly that the eight submissions they handled were "shockingly bad" in quality, going so far as to suggest authors "don't submit if you're not ready" — sparking heated discussion in academic circles about the declining quality of papers at top conferences.

A Reviewer's Frustration: Not Even Meeting the Workshop Bar

The reviewer wrote that they had three ICLR papers and five AAMAS papers to review, but the quality was so poor that they "wouldn't even accept them into a workshop, let alone the main conference." They specifically noted that first-time submitters should definitely seek feedback from advisors or peers first, otherwise they're just wasting the time of a large number of reviewers.

While emotional, these remarks struck a chord with many academic workers. Top conference submissions have been swelling year after year, while the reviewer pool remains relatively fixed. The number of papers assigned to each reviewer keeps rising, and the frustration of encountering low-quality submissions is amplified accordingly.

The Vicious Cycle of Submission Volume and Review Quality

The scale of submissions to top AI conferences has continued to expand in recent years, with ICLR, NeurIPS, and others repeatedly hitting new highs. The surge in submissions brings two direct consequences: first, reviewers are severely overloaded, with each person handling far more papers than a reasonable workload; second, the average depth of reviews declines, and genuinely valuable feedback becomes rarer. When reviewers expend their energy on clearly immature manuscripts, authors who have carefully polished their papers may paradoxically receive more cursory evaluations.

The Reddit user's complaint is essentially an individual expression of this structural contradiction.

ICLR (International Conference on Learning Representations) is a top conference in deep learning, known for its open review process where all submissions and review comments are publicly visible; AAMAS (International Conference on Autonomous Agents and Multiagent Systems) is the flagship conference in the multi-agent systems field. The two differ considerably in review process and community culture, but both are experiencing rapid growth in submissions. The fact that one reviewer is simultaneously handling review duties for top conferences in two different fields itself reflects the current strain on the AI conference reviewer pool — conference organizers often need to recruit reviewers across fields, and reviewers themselves face the cognitive burden of cross-domain reviewing.

Workshops are typically affiliated with main conferences, with lower review standards and higher acceptance rates, often seen as a platform for showcasing early-stage work. The reviewer's use of "not even workshop-worthy" to describe manuscript quality implies that these submissions have obvious deficiencies in research completeness, experimental sufficiency, or writing standards — already below the minimum threshold for academic conferences.

NeurIPS (Conference on Neural Information Processing Systems) is one of the largest top conferences in machine learning, with submissions in recent years exceeding 20,000. ICLR's submissions have also grown from about 500 in 2017 to thousands in recent years. Behind this exponential growth is the rapid expansion of the number of AI researchers, rigid graduation publication requirements, and the popularity of the "submit first, revise later" strategy under preprint culture.

The reviewer pool's expansion speed lags far behind submission growth: qualified reviewers typically need publication experience in relevant fields, while senior researchers have limited time, forcing conferences to heavily recruit doctoral students and even junior researchers as reviewers. The direct consequence of reviewer overload is reduced review time and attention per paper, with reviews becoming increasingly templated and superficial. A more insidious effect is that after facing large numbers of low-quality manuscripts, reviewers easily develop "review fatigue," reducing their willingness to invest effort in subsequent papers — forming a negative spiral in quality perception.

What's the Problem: Inexperienced Newcomers or Misaligned Incentives?

The post didn't specify exactly what was wrong with the papers, but from the call to "seek advisor feedback first," it can be inferred that the problems likely center on fundamental aspects such as incomplete research design, insufficient experiments, and non-standard writing — rather than mere English expression issues.

The Plight of First-Time Submitters

For many junior graduate students, top conference submissions are often hard requirements for graduation or their advisor's projects. Without systematic guidance, students easily equate "completing a paper" with "ready to submit," overlooking that peer review itself is a test of research maturity.

In the AI field, top conference papers have become hard requirements for graduation in many labs, with some advisors even linking submission counts to project completion and student evaluations. Under this incentive structure, there's an inherent tension between the time pressure students face and paper maturity: a truly solid paper often requires months or even a year of iteration, while graduation windows and project deadlines won't wait.

Common mistakes first-time submitters make include: packaging course projects or preliminary experiments directly as complete papers, lacking ablation studies and baseline comparisons, related work surveys that merely list rather than critically analyze, and confusing contributions with implementation details in writing. The root of these problems is often not attitude but a lack of awareness of the implicit standards for "what counts as mature research." The reason advisor and senior peer pre-review is so critical is precisely that they can provide this standard calibration — pointing out which flaws are fatal in reviewers' eyes and which can be remedied during the rebuttal phase.

The Distortion of Submission Strategy

Another possibility is strategic submission: some authors, adopting a "cast a wide net and hope for luck" mentality, send clearly immature manuscripts to top conferences, hoping to pass through luck or chance. This behavior is especially damaging to the review ecosystem in conferences with massive submission volumes.

Community Reaction: Resonance and Rebuttal Coexist

To be clear, the specific comments above come from a single Reddit post, and no systematic community feedback data has been observed. But judging from discussions around review quality in the AI academic community in recent years, such complaints are not isolated.

A common rebuttal is that what reviewers see as "bad papers" may sometimes result from differences in field perspective or unfamiliar writing styles, not necessarily quality issues. Moreover, top conference review itself involves randomness — the same paper may receive vastly different evaluations from different reviewers. Therefore, placing all the blame on authors is not entirely fair.

Can Conferences Do More

Some conferences have attempted mechanisms like desk reject, mandatory format checks, and reviewer training to filter out clearly substandard submissions. But judging from this reviewer's experience, these measures' effectiveness remains limited.

Practical Advice for Submitters

Setting aside the emotion, the advice in this post is actually quite practical: before submitting, have experienced peers vet your work — confirm the research question is clear, the experimental design is complete, and the writing meets basic standards, then submit. For first-time submitters, one serious pre-review round of feedback may be more valuable than submitting to three more conferences.

Reviewing is a public resource of the academic community. Spending one extra week polishing before submission may save reviewers several hours — a net positive for the entire ecosystem.