getActionInfo

Obtain action data specifications from reinforcement learning environment, agent, or experience buffer

Syntax

actInfo = getActionInfo(env)

actInfo = getActionInfo(agent)

actInfo = getActionInfo(buffer)

Description

actInfo = getActionInfo(env) extracts action information from reinforcement learning environment env.

actInfo = getActionInfo(agent) extracts action information from reinforcement learning agent agent.

actInfo = getActionInfo(buffer) extracts action information from experience buffer buffer.

Examples

Extract Action and Observation Specifications from Reinforcement Learning Environment

The reinforcement learning environment for this example is a longitudinal dynamics model comprising two cars, a leader and a follower. The vehicle model is also used in the Adaptive Cruise Control System Using Model Predictive Control (Model Predictive Control Toolbox) example.

Open the model.

mdl = "rlACCMdl";
open_system(mdl);

Specify path to the agent block in the model.

agentblk = mdl + "/RL Agent";

Create the observation and action specifications.

% Observation specifications
obsInfo = rlNumericSpec([3 1],LowerLimit=-inf*ones(3,1),UpperLimit=inf*ones(3,1));
obsInfo.Name = "observations";
obsInfo.Description = "information on velocity error and ego velocity";

% Action specifications
actInfo = rlNumericSpec([1 1],LowerLimit=-3,UpperLimit=2);
actInfo.Name = "acceleration";

Define environment interface.

env = rlSimulinkEnv(mdl,agentblk,obsInfo,actInfo)

env = 
SimulinkEnvWithAgent with properties:

           Model : rlACCMdl
      AgentBlock : rlACCMdl/RL Agent
        ResetFcn : []
  UseFastRestart : on

The reinforcement learning environment env is a SimulinkEnvWithAgent object.

Extract the action and observation specifications from env.

actInfoExt = getActionInfo(env)

actInfoExt = 
  rlNumericSpec with properties:

     LowerLimit: -3
     UpperLimit: 2
           Name: "acceleration"
    Description: [0x0 string]
      Dimension: [1 1]
       DataType: "double"

obsInfoExt = getObservationInfo(env)

obsInfoExt = 
  rlNumericSpec with properties:

     LowerLimit: [3x1 double]
     UpperLimit: [3x1 double]
           Name: "observations"
    Description: "information on velocity error and ego velocity"
      Dimension: [3 1]
       DataType: "double"

The action information contains acceleration values while the observation information contains the velocity and velocity error values of the ego vehicle.

Input Arguments

env — Reinforcement learning environment

rlFunctionEnv object | SimulinkEnvWithAgent object | rlNeuralNetworkEnvironment object | predefined MATLAB environment object

Reinforcement learning environment from which to extract the action information, specified as one of the following:

• MATLAB^® environment represented as one of the following objects.

  • rlFunctionEnv

  • rlNeuralNetworkEnvironment

  • Predefined MATLAB environment created using rlPredefinedEnv

• Simulink^® environment represented as a SimulinkEnvWithAgent object.

For more information on reinforcement learning environments, see Reinforcement Learning Environments and Create Custom Simulink Environments.

agent — Reinforcement learning agent

rlQAgent object | rlSARSAAgent object | rlDQNAgent object | rlPGAgent object | rlDDPGAgent object | rlTD3Agent object | rlACAgent object | rlPPOAgent object | rlTRPOAgent object | rlSACAgent object | rlMBPOAgent object

Reinforcement learning agent from which to extract the action information, specified as one of the following objects.

• rlQAgent

• rlSARSAAgent

• rlDQNAgent

• rlDDPGAgent

• rlTD3Agent

• rlPGAgent

• rlACAgent

• rlPPOAgent

• rlTRPOAgent

• rlSACAgent

• rlMBPOAgent

For more information on reinforcement learning agents, see Reinforcement Learning Agents.

buffer — Experience buffer

rlReplayMemory object | rlPrioritizedReplayMemory object | rlHindsightReplayMemory object | rlHindsightPrioritizedReplayMemory object

Experience buffer, specified as one of the following replay memory objects.

• rlReplayMemory

• rlPrioritizedReplayMemory

• rlHindsightReplayMemory

• rlHindsightPrioritizedReplayMemory

Output Arguments

actInfo — Action data specifications

array of rlNumericSpec objects | array of rlFiniteSetSpec objects

Action data specifications extracted from the reinforcement learning environment, returned as an array of one of the following:

• rlNumericSpec objects

• rlFiniteSetSpec objects

• A mix of rlNumericSpec and rlFiniteSetSpec objects

Version History

Introduced in R2019a