Deep reinforcement learning is a branch of machine learning that enables you to implement controllers and decision-making systems for complex systems such as robots and autonomous systems. Deep reinforcement learning lets you implement deep neural networks that can learn complex behaviors by training them with data generated dynamically from simulation models. It does not require a predefined training dataset, labeled or unlabeled, all you need is a simulation model that represents the environment you are interacting with and trying to control.
MATLAB® and Simulink® support the complete workflow for designing and deploying a reinforcement learning based controller. You can:
- Get started with reinforcement learning using examples for simple control systems, autonomous systems, and robotics
- Quickly switch, evaluate, and compare popular reinforcement learning algorithms with only minor code changes
- Use deep neural networks to define complex reinforcement learning policies based on image, video, and sensor data
- Train policies faster by running multiple simulations in parallel using local cores or the cloud
- Deploy reinforcement learning controllers to embedded devices
Reinforcement Learning Agents
Reinforcement learning agents are comprised of a policy that performs a mapping from an input state to an output action and an algorithm responsible for updating this policy. Deep Q-networks, actor-critic, and deep deterministic policy gradients are popular examples of algorithms. The algorithm updates the policy such that it maximizes the long-term reward signal provided by the environment.
Policies can be represented by deep neural networks, polynomials, and lookup tables. You can then implement built-in and custom agents as MATLAB objects or Simulink blocks.
Environment Modeling in MATLAB and Simulink
Training with reinforcement learning algorithms is a dynamic process as the agent interacts with the environment around it. For applications such as robotics and autonomous systems, performing this training in the real world with actual hardware can be expensive and dangerous. This is why virtual models of the environment that generate data through simulations are greatly preferred for reinforcement learning.
You can build a model of your environment in MATLAB and Simulink that describes the system dynamics, how they are affected by actions taken by the agent, and a reward that evaluates the goodness of the action performed. These models can be continuous or discrete in nature and can represent your system at varying levels of fidelity. In addition, you can parallelize simulations to accelerate training. In some cases, you may be able to reuse existing MATLAB and Simulink models of your system for reinforcement learning with minimal modifications.
Examples and Reference Applications
Get started with reinforcement learning by implementing controllers for problems such as balancing an inverted pendulum, navigating a grid-world problem, and balancing a cart-pole system. You can also design systems for adaptive cruise control and lane-keeping assist for autonomous vehicles. Reinforcement learning can also be used for robotics applications, such as trajectory planning, and teaching behaviors, such as locomotion.