Read the original document by opening this link in a new tab.
Table of Contents
1 Introduction
2 Background
2.1 Artificial Intelligence
2.2 Machine Learning
2.3 Deep Learning
2.4 Reinforcement Learning
2.4.1 Problem Setup
2.4.2 Value Function
2.4.3 Exploration vs. Exploitation
2.4.4 Dynamic Programming
2.4.5 Monte Carlo
2.4.6 Temporal Difference Learning
2.4.7 Multi-step Bootstrapping
2.4.8 Model-based RL
2.4.9 Function Approximation
2.4.10 Policy Optimization
2.4.11 Deep RL
2.4.12 Brief Summary
2.5 Resources
Part I: Core Elements
3 Value Function
3.1 Deep Q-Learning
3.2 Distributional Value Function
3.3 General Value Function
4 Policy
4.1 Policy Gradient
4.2 Actor-Critic
4.3 Trust Region Methods
4.4 Policy Gradient with Off-Policy Learning
4.5 Benchmark Results
5 Reward
6 Model
7 Exploration vs. Exploitation
8 Representation
8.1 Classics
8.2 Neural Networks
8.3 Knowledge and Reasoning
Part II: Important Mechanisms
9 Attention and Memory
10 Unsupervised Learning
11 Hierarchical RL
12 Multi-Agent RL
13 Relational RL
14 Learning to Learn
14.1 Few/One/Zero-Shot Learning
14.2 Transfer/Multi-task Learning
14.3 Learning to Optimize
14.4 Learning Reinforcement Learn
14.5 Learning Combinatorial Optimization
14.6 AutoML
Part III: Applications
15 Games
15.1 Board Games
15.2 Card Games
15.3 Video Games
16 Robotics
16.1 Sim-to-Real
16.2 Imitation Learning
16.3 Value-based Learning
16.4 Policy-based Learning
16.5 Model-based Learning
16.6 Autonomous Driving Vehicles
17 Natural Language Processing
17.1 Sequence Generation
17.2 Machine Translation
17.3 Dialogue Systems
18 Computer Vision
18.1 Recognition
18.2 Motion Analysis
18.3 Scene Understanding
18.4 Integration with NLP
18.5 Visual Control
18.6 Interactive Perception
19 Finance and Business Management
19.1 Option Pricing
19.2 Portfolio Optimization
19.3 Business Management
20 More Applications
20.1 Healthcare
20.2 Education
20.3 Energy
20.4 Transportation
20.5 Computer Systems
20.6 Science, Engineering and Art
20.6.1 Chemistry
20.6.2 Mathematics
20.6.3 Music
21 Discussions
21.1 Brief Summary
21.2 Challenges and Opportunities
21.3 Epilogue
Summary
We discuss deep reinforcement learning in an overview style. We draw a big picture, filled with details. We discuss six core elements, six important mechanisms, and twelve applications, focusing on contemporary work, and in historical contexts. We start with background of artificial intelligence, machine learning, deep learning, and reinforcement learning (RL), with resources. Next we discuss RL core elements, including value function, policy, reward, model, exploration vs. exploitation, and representation. Then we discuss important mechanisms for RL, including attention and memory, unsupervised learning, hierarchical RL, multi-agent RL, relational RL, and learning to learn. After that, we discuss RL applications, including games, robotics, natural language processing (NLP), computer vision, finance, business management, healthcare, education, energy, transportation, computer systems, and, science, engineering, and art. Finally we summarize briefly, discuss challenges and opportunities, and close with an epilogue.