getLearnableParameters

Obtain learnable parameter values from agent, function approximator, or policy object

Since R2019a

Syntax

pars = getLearnableParameters(agent)

pars = getLearnableParameters(fcnAppx)

pars = getLearnableParameters(policy)

Description

Agent

pars = getLearnableParameters(agent) returns the learnable parameter values from the agent object agent.

Actor or Critic

pars = getLearnableParameters(fcnAppx) returns the learnable parameter values from the actor or critic function approximator object fcnAppx.

Policy

pars = getLearnableParameters(policy) returns the learnable parameters values from the policy object policy.

Examples

Modify Critic Parameter Values

Assume that you have an existing trained reinforcement learning agent. For this example, load the trained agent from Train DDPG Agent to Control Double Integrator System.

load("DoubleIntegDDPG.mat","agent") 

Obtain the critic function approximator from the agent.

critic = getCritic(agent);

Obtain the learnable parameters from the critic.

params = getLearnableParameters(critic)

params=2×1 cell array
    {[-4.9869 -1.5577 -0.3351 -0.1097 -0.0444 7.5240e-04]}
    {[                                                 0]}

Modify the parameter values. For this example, simply multiply all of the parameters by 2.

modifiedParams = cellfun(@(x) x*2,params,"UniformOutput",false);

Set the parameter values of the critic to the new modified values.

critic = setLearnableParameters(critic,modifiedParams);

Set the critic in the agent to the new modified critic.

setCritic(agent,critic);

Display the new parameter values.

getLearnableParameters(getCritic(agent))

ans=2×1 cell array
    {[-9.9737 -3.1153 -0.6702 -0.2194 -0.0888 0.0015]}
    {[                                             0]}

Modify Actor Parameter Values

Assume that you have an existing trained reinforcement learning agent. For this example, load the trained agent from Train DDPG Agent to Control Double Integrator System.

load("DoubleIntegDDPG.mat","agent") 

Obtain the actor function approximator from the agent.

actor = getActor(agent);

Obtain the learnable parameters from the actor.

params = getLearnableParameters(actor)

params=2×1 cell array
    {[-15.4601 -7.2076]}
    {[               0]}

Modify the parameter values. For this example, simply multiply all of the parameters by 2.

modifiedParams = cellfun(@(x) x*2,params,"UniformOutput",false);

Set the parameter values of the actor to the new modified values.

actor = setLearnableParameters(actor,modifiedParams);

Set the actor in the agent to the new modified actor.

setActor(agent,actor);

Display the new parameter values.

getLearnableParameters(getActor(agent))

ans=2×1 cell array
    {[-30.9201 -14.4153]}
    {[                0]}

Input Arguments

agent — Reinforcement learning agent

reinforcement learning agent object

Reinforcement learning agent, specified as one of the following objects:

• rlQAgent

• rlSARSAAgent

• rlDQNAgent

• rlPGAgent

• rlDDPGAgent

• rlTD3Agent

• rlACAgent

• rlSACAgent

• rlPPOAgent

• rlTRPOAgent

• Custom agent — For more information, see Create Custom Reinforcement Learning Agents.

fcnAppx — Actor or critic function object

reinforcement learning function approximator object

Function approximator object, specified as one of the following:

• rlValueFunction object — Value function critic

• rlQValueFunction object — Q-value function critic

• rlVectorQValueFunction object — Multi-output Q-value function critic with a discrete action space

• rlContinuousDeterministicActor object — Deterministic policy actor with a continuous action space

• rlDiscreteCategoricalActor object — Stochastic policy actor with a discrete action space

• rlContinuousGaussianActor object — Stochastic policy actor with a continuous action space

• rlContinuousDeterministicTransitionFunction object — Continuous deterministic transition function for neural network environments

• rlContinuousGaussianTransitionFunction object — Continuous Gaussian transition function for neural network environments

• rlContinuousDeterministicRewardFunction object — Continuous deterministic reward function for neural network environments

• rlContinuousGaussianRewardFunction object — Continuous Gaussian reward function for neural network environments

• rlIsDoneFunction object — IsDone function for neural network environments

To create an actor or critic function object, use one of the following methods.

• Create a function object directly.

• Obtain the existing critic from an agent using getCritic.

• Obtain the existing actor from an agent using getActor.

policy — Reinforcement learning policy

rlMaxQPolicy | rlEpsilonGreedyPolicy | rlDeterministicActorPolicy | rlAdditiveNoisePolicy | rlStochasticActorPolicy

Reinforcement learning policy, specified as one of the following objects:

• rlMaxQPolicy

• rlEpsilonGreedyPolicy

• rlDeterministicActorPolicy

• rlAdditiveNoisePolicy

• rlStochasticActorPolicy

Output Arguments

pars — Learnable parameters

rlValueFunction object | rlQValueFunction object | rlVectorQValueFunction object | rlContinuousDeterministicActor object | rlDiscreteCategoricalActor object | rlContinuousGaussianActor object

Learnable parameter values for the function object, returned as a cell array. You can modify these parameter values and set them in the original agent or a different agent using the setLearnableParameters function.

Version History

Introduced in R2019a

R2022a: getLearnableParameters now uses approximator objects instead of representation objects

Using representation objects to create actors and critics for reinforcement learning agents is no longer recommended. Therefore, getLearnableParameters now uses function approximator objects instead.

R2020a: getLearnableParameterValues is now getLearnableParameters

getLearnableParameterValues is now getLearnableParameters. To update your code, change the function name from getLearnableParameterValues to getLearnableParameters. The syntaxes are equivalent.