What to Read: Text Book

Reinforcement Learning: An Introduction, Second Edition, By Richard S. Sutton and Andrew G. Barto, MIT Press

What to Read: Research Papers

The following list and comments only represent the instructor’s personal opinion.

  • NeurIPS: One of the most important and influential conference in machine learning field, many interesting and seminal papers in reinforcement learning research.
  • ICML: Another most important and influential conference in machine learning field, another good place to find interesting and seminal papers in reinforcement learning research.
  • ICLR: A new but fast raising conference in the field of machine learning, especially deep learning. A good place to find the most recent deep reinforcement learning research.
  • AAAI: One of the most traditional venue for artificial intelligence research. You can also find many good papers in reinforcement learning research.
  • JMLR: A very good journal for machine learning research. If you are looking for longer version of the conference papers, here is the good place to go.
  • If you are interested in rankings or indices of those conferences and journals, you may take a look at Google Scholar’s Metrics. Reinforcement learning is under this category.

What to Read: Courses Offered in Other Institutes

What to Read: Background in Machine Learning and Convex Optimization

What to Watch: Great Tutorials and Short Lectures on RL

Where to Test Your RL Algorithms

Most of open frameworks for reinforcement learning focus on deep reinforcement learning. But they can definitely be used as test beds for more traditional reinforcement learning research, e.g., tabular MDP algorithms. Moreover, they are mostly for games and simulation-based studies, as such environments are easier to create. Among all choices openly available, OpenAI’s Gym should be the most popular one. It is designed for developing and comparing reinforcement learning algorithms.

If you are a fan of Microsoft’s Minecraft game, you might consider to test your RL algorithm with their Minecraft game engine using Azure Machine Learning.

Where to Run Your Deep RL Algorithms

Thanks to the great success of deep reinforcement learning, which has become a reference solution for reinforcement learning problems. But the power and flexibility come with the high dependence on computational resources, i.e., the GPUs. We understand the difficulty of getting access to (free) GPUs for an ordinary student, and we have compiled a list of online public resources where you can find such access. You are definitely more than welcome to share other possible resources you found.

How to Entertain Your Audiences

The importance of giving a good and impressive presentation about your research can never be underestimated. We would like to provide you the platform to improve/hone your presentation skills in this course.

Here are some general tips for giving a good research presentation.

I personally also like to watch TED talks, which are not only inspiring in the ideas they deliver but also the way they deliver.

How to Write Your Scientific Story

General Advice on Computer Science Research

Use LaTex!

  • We require you to use LaTeX for your reports in this course, as LaTeX is a skill you should learn if you haven’t already!
  • We provide you the official ACM templates to structure your reports.
  • Additionally,