Paper reading is a vital skill in any field of research. In this class, we will experiment with a completely new approach to provide you the training to hone your skills: We ask you to act as a peer reviewer of this year’s ICLR submissions!

ICLR, known as the International Conference on Learning Representations, is one of the best venues for machine learning research (check out Google Scholar Metrics). It is also known for its pioneer effort in exploring an open peer review process. ICLR chooses to make all its submissions publically accessible, where everyone can comment on any submissions before and after they are accepted/rejected.

We will ask you to review this year’s ICLR submissions in the area of reinforcement learning, strictly following ICLR’s requirements. To avoid interference with ICLR’s review process, we ask you to post your review on our Piazza forum. But for high quality reviews, we will encourage you to post your comments on ICLR’s open review system. In this way, you can directly interact with the authors and contribute your effort in this community!

As the submission deadline for ICLR’2023 is September 28th, we will start our assignment right after it. By October 4th, the instructor will compile a list of ICLR’2023 submissions and post it here. You need to choose a paper from the list, write your review about it, and post on Piazza. The official review of ICLR’2023 becomes available on November 4th, and by then you should compare your review with those from the official reviewers, from which you can get some sense about how a domain expert reviews a paper and how your evaluation differs from an expert’s.

What to Cover in Your Review

A good scientific peer review should be thoughtful and constructive. ICLR requires a review to consist of the following parts:

1. Summarize what the paper claims to contribute. Be positive and generous.	
2. List strong and weak points of the paper. Be as comprehensive as possible.	
3. Clearly state your recommendation (accept or reject) with one or two key reasons for this choice.	
4. Provide supporting arguments for your recommendation.	
5. Ask questions you would like answered by the authors to help you clarify your understanding of the paper and provide the additional evidence you need to be confident in your assessment. 	
6. Provide additional feedback with the aim to improve the paper. Make it clear that these points are here to help, and not necessarily part of your decision assessment.

The complete ICLR review guide can be found here. You should carefully read the instruction before working on your review. And if this is your first time working on a scientific peer review, you are highly encouraged to read this Nature article about how to write good peer reviews.

This is How We Thank You

Once you post your ICLR peer review on Piazza, we ask everyone in this class to read and evaluate it. Those received most postive feedback from our class (top 10%) will receive bonus points (+5% points). The instructor and TA will choose the truly exceptional ones and encourage the students to post your review on openreview and participate in the open discussions with the authors and other reviewers. More bonus points will be given to such cases (+10%).

List of Selected ICLR’2023 Submissions

Both TAs and the instructor have picked five of their favorite submissions from ICLR’2023, and the list is provided in the following. Everyone of you only need to choose of the 15 papers to review and post your review under this Piazza thread before November 11, 2022.

  1. Universal embodied intelligence: learning from crowd, recognizing the world, and reinforced with experience
  2. Robust Policy Optimization in Deep Reinforcement Learning
  3. Planning With Uncertainty: Deep Exploration in Model-Based Reinforcement Learning
  4. Online Reinforcement Learning via Posterior Sampling of Policy
  5. I pick you choose’: Joint human-algorithm decision making in multi-armed bandits
  6. Continuous Monte Carlo Graph Search
  7. Jump-Start Reinforcement Learning
  8. Reinforcement Learning for Bandits with Continuous Actions and Large Context Spaces
  9. Extreme Q-Learning: MaxEnt RL without Entropy
  10. System Identification as a Reinforcement Learning Problem
  11. Deep Reinforcement Learning for Cryptocurrency Trading: Practical Approach to Address Backtest Overfitting
  12. Fairness-Aware Model-Based Multi-Agent Reinforcement Learning for Traffic Signal Control
  13. Skill-Based Reinforcement Learning with Intrinsic Reward Matching
  14. Human-level Atari 200x faster
  15. Reward Design with Language Models

Review Template

The following is the official review template required by ICLR’2023, and we will follow it to construct our reviews.

Summary Of The Paper

Please provide a brief summary of the paper and its contributions.

Strength And Weaknesses

Please list both the strengths and weaknesses of the paper. When discussing weaknesses, please provide concrete, actionable feedback on the paper.

Clarity, Quality, Novelty And Reproducibility

Can you provide an evaluation of the quality, clarity and originality of the work?

Summary Of The Review

Please provide a short summary justifying your recommendation of the paper.

Correctness

When evaluating correctness, we urge reviewers to focus on the technical content of the paper instead of language. This is especially important to support researchers from diverse geographical backgrounds. (choose one rating)

  1. The main claims of the paper are incorrect or not at all supported by theory or empirical results.
  2. Several of the papers claims are incorrect or not well-supported.
  3. Some of the papers claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
  4. All of the claims and statements are well-supported and correct.

Technical Novelty And Significance

For this question, contributions are technical in nature, including new models, techniques, or theoretical insights. (choose one rating)

  1. The contributions are neither significant nor novel.
  2. The contributions are only marginally significant or novel.
  3. The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
  4. The contributions are significant, and do not exist in prior works.

Empirical Novelty And Significance

For this question, contributions include new insights supported by empirical results including those arising from new benchmarks or datasets. (choose one rating)

  1. Not applicable
  2. The contributions are neither significant nor novel.
  3. The contributions are only marginally significant or novel.
  4. The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
  5. The contributions are significant, and do not exist in prior works.

Flag For Ethics Review

Choose from one or more entries.

  • NO.
  • Yes, Discrimination / bias / fairness concerns
  • Yes, Privacy, security and safety
  • Yes, Legal compliance (e.g., GDPR, copyright, terms of use)
  • Yes, Potentially harmful insights, methodologies and applications
  • Yes, Responsible research practice (e.g., human subjects, data release)
  • Yes, Research integrity issues (e.g., plagiarism, dual submission)
  • Yes, Unprofessional behaviors (e.g., unprofessional exchange between authors and reviewers)
  • Yes, Other reasons (please specify below)

Recommendation

(choose one rating)

  1. strong reject
  2. reject, not good enough
  3. marginally below the acceptance threshold
  4. marginally above the acceptance threshold
  5. accept, good paper
  6. strong accept, should be highlighted at the conference

Confidence

(choose one rating)

  1. You are unable to assess this paper and have alerted the ACs to seek an opinion from different reviewers.
  2. You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
  3. You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
  4. You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
  5. You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.