How to set block parameters in a Matlab RL reset function

13 ビュー (過去 30 日間)
Carson Cooper
Carson Cooper 2022 年 4 月 13 日
回答済み: Steve Miller 2022 年 11 月 28 日
I am using a reset function as a part of the rlSimulinkEnv and calling it in the main script. In the reset function, when I try to reset a constant block, this works as expected. However, when I try to reset variable found in a simulink block like a revolute joint, the value will be reset at the beginning of the first episode, but then never again.
agentBlk = [mdl '/RL Agent'];
observationInfo = rlNumericSpec([23 1]);
actionInfo = rlNumericSpec([6 1]);
observationInfo.Name = 'observation';
actionInfo.Name = 'action';
env = rlSimulinkEnv(mdl,agentBlk, observationInfo, actionInfo);
env.ResetFcn = @(in)handResetFcn(in,sep,yL_end,l);
Here is the reset function. In this case, the 'goal_x' and 'goal_y' variables correspond to constant blocks and reset every episode. The joint angles will not reset every time. The will be set to the first episode's random value for every episode.
function in = handResetFcn(in,sep,yL_end)
x = sep+.1*(rand-.5);
y = yL_end+.05*rand;
in = in.setVariable('goal_x',x);
in = in.setVariable('goal_y',y);
tl1_0 = pi/2 + .1*(rand-.5);
tl2_0 = pi/2 + .1*(rand-.5);
in = setBlockParameter(in,'FullHand_ChangeInitCondModel/Environment/Revolute Joint','PositionTargetValue',num2str(tl1_0));
in = setBlockParameter(in,'FullHand_ChangeInitCondModel/Environment/Revolute Joint1','PositionTargetValue',num2str(tl2_0));
end
  1 件のコメント
ajshank
ajshank 2022 年 6 月 2 日
編集済み: ajshank 2022 年 6 月 2 日
I can confirm this is the case for Matlab R2021b. Other tricks such as assigning a workspace variable myVar for the block/mask parameter, and calling assignin("base", "myVar", val) in the reset function don't work either.

サインインしてコメントする。

回答 (1 件)

Steve Miller
Steve Miller 2022 年 11 月 28 日
Hi Carson,
I suspect the reason this did not work is because you have not configured the target position to be a run-time parameter. If you do not configure this to be a run-time parameter, your sweep may not recognize that the value has changed from run to run. You can see in example Quadruped Robot Locomotion Using DDPG Agent that the joint targets are configured as run-time parameters.
Below is a screenshot showing the setting you need to change on the blocks.
--Steve

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by