ICLR Open Review Process Explained: When Papers Go Public and When Reviews Become Visible

How ICLR's Open Review Differs from Traditional Conferences

For researchers accustomed to the submission workflows of ICML, NeurIPS, or AAAI, submitting to ICLR (International Conference on Learning Representations) for the first time can be genuinely confusing — largely due to its distinctive OpenReview-based open peer review system. A Reddit user recently captured this common source of confusion perfectly: the submission portal states that papers will be made publicly visible to the community "from the start of the review process," yet reviews begin in October while the discussion period doesn't open until November. What exactly does that gap mean?

The open review model ICLR uses is built around a core principle: increasing transparency and accountability in academic peer review. Papers, reviewer comments, author responses, and the back-and-forth discussion throughout the review process all become visible to the broader research community at specific stages. This stands in stark contrast to the double-blind, closed-door review systems used at most other major venues — and it's precisely what makes first-time submitters uneasy, especially when their submission includes supplementary materials like code.

OpenReview is the open academic review platform ICLR uses, jointly developed by the University of Massachusetts Amherst and New York University with the goal of making peer review more transparent and accountable. Unlike traditional submission systems such as ScholarOne or CMT, OpenReview archives papers, reviews, author responses, and discussion threads in a searchable, publicly accessible format. Anyone can register an account and post a public comment on any submission. This design gives ICLR's review process complete retrospective traceability — researchers can look up the full review history of any past submission, including papers that were ultimately rejected. This radical transparency fundamentally conflicts with the "black box" review culture that academia has long relied on, and it remains one of the most persistently debated aspects of ICLR since its founding in 2013.

When Does a Submission Become Publicly Visible?

The first question on this submitter's mind: if reviews start in October and the discussion period begins in November, does the paper actually become publicly visible in October?

Based on ICLR's OpenReview practices in recent years, the answer is generally yes — papers become publicly visible to the community when the review process formally begins, which coincides roughly with when reviewers start their work. In other words, once the review phase kicks off, anyone can browse the submitted paper on the OpenReview platform. It's worth noting that while the paper is public at this point, it typically remains author-anonymous (double-blind) — community members can read the content but cannot see who wrote it.

This design has a practical implication worth internalizing: a paper's core ideas enter the public eye before the review process has concluded and before acceptance decisions have been made. For researchers concerned about being scooped, this is a feature of the ICLR mechanism that must be understood before submitting.

Are Supplementary Materials and Code Also Made Public?

The submitter's second concern is more specific: does supplementary material — especially code — become visible at the same time as the paper? They explicitly mentioned that their supplementary material contains code and worried about those resources being shared or redistributed before the review concludes.

This is a very practical concern. Under OpenReview's default mechanics, supplementary materials submitted alongside a paper are in principle made publicly visible at the same time as the paper, unless the platform or conference policy places specific access restrictions on certain materials. This means code and other resources could potentially be accessed by a broader audience during the review period.

For authors with this concern, it's worth carefully checking the specific ICLR policies for the submission year before submitting: certain supplementary materials may be restricted to reviewers and area chairs (ACs) only, rather than being visible to the entire community. If code involves unpublished work or raises intellectual property concerns, a prudent approach is to evaluate which materials truly need to be included in the supplement versus which can be released after acceptance — or to use anonymized code hosting to reduce exposure.

Anonymized code hosting is a common technical approach for managing this risk. Researchers typically upload their codebase to an anonymized GitHub mirror service (such as Anonymous GitHub) or create a temporary account with no personal information before submitting. This satisfies supplementary material requirements while preventing the code from being directly linked to the authors' real identities. Anonymous GitHub is a proxy service specifically designed for double-blind review, capable of automatically replacing links and content within a repository that might reveal author identity. It has been widely used by submitters to ICLR, NeurIPS, and several other venues. One important caveat: even with anonymization, distinctive coding style, specific dependency combinations, or institutional information in comments can inadvertently reveal authorship. A systematic de-identification review of the code before submission is advisable.

When Do Reviews Become Visible?

The third question gets at the heart of the transparency model: once a reviewer submits their review, can authors and the community see it immediately — or does it stay hidden until the discussion period opens?

In ICLR's open review process, the visibility of reviews is typically tied to the start of the rebuttal/discussion period. That is, even though reviewers submit their reviews on a rolling basis starting in October, those reviews are generally released to authors and the community all at once when the discussion period officially opens — so that author responses and multi-party discussion can begin. This explains the gap between the start of reviewing (October) and the start of the discussion period (November): the former is an internal phase for reviewers, while the latter is when public interaction begins.

That said, specific arrangements may differ slightly from year to year, and authors should always treat the official timeline and reviewer guidelines published for that cycle as the authoritative source.

Practical Advice for First-Time ICLR Submitters

Drawing on this community discussion, here are a few practical takeaways for researchers encountering ICLR for the first time:

• Understand the public-by-default nature upfront: Papers go public when the review phase begins — this is the single biggest difference from the closed review systems at NeurIPS and ICML. Prepare mentally and strategically before you submit.
• Handle supplementary materials with care: When code or sensitive resources are involved, confirm the visibility rules first. Adjust what you include in the supplement if necessary, or use anonymous hosting to reduce risk.
• Follow the official timeline closely: The exact timing and visibility rules for each phase — review, discussion, and decision — can be adjusted from year to year. Always rely on the OpenReview platform and official ICLR announcements as your authoritative source.
• Make the most of the discussion period: The discussion period after reviews go public is where ICLR's model delivers real value. A thorough, well-reasoned author response can meaningfully influence the final outcome.

ICLR's open review model enhances transparency, but it does so by moving "going public" to the review stage itself — well before decisions are made. Understanding these rules eliminates unnecessary anxiety and helps authors plan their submission strategy with greater confidence.