Using environment reset function: Access Simulink workspace for if then tests

I'm attempting to code a PPO RL agent in which a glider is released and attempts to navigate to a target within "R997" distance. I've generated the model within Simulink including outputing the last distance value to the target ("D2Tgt") with a workspace block. For navigating to the target within a R997 value, it gets a reward. It gets other reward and penalties along the way as well. Some publications have mentioned such training cases can be speed up by holding the target location constant instead of randomizing it each training episode until the it starts getting close where you start randomizing it. This is done by starting with a large acceptable distance and decreasing it over the training time. This is what I've attempted to code below. The issue I'm having is pulling the "ans.D2Tgt" from the workspace during RL training even though I have a simulink workspace out block for this value. Any advice???
ans.D2Tgt=random initalization for now
env = rlSimulinkEnv('SimulinkModel','SimulinkModel/RL Agent',obsInfo,actInfo);
env.ResetFcn = @(in) localResetFcn(in, R997, ans);
function in = localResetFcn(in, R997, ans)
persistent n
if isempty(n)
n=0
end
ans.D2Tgt=ans.D2Tgt
DropHeight = randi([deleted values were here],1,1);
blk = 'SimulinkModel/6DOF Wind (Wind Angles)';
in = setBlockParameter(in,blk,'xme_0',mat2str([0 0 -DropHeight]));
ReleaseVelocity=randi([deleted values were here],1,1);
blk = 'SimulinkModel/6DOF Wind (Wind Angles)';
in = setBlockParameter(in,blk,'Vm_0',mat2str([ReleaseVelocity 0 0]));
if (ans.D2Tgt <= R997 * 5) && (n==0)
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 4));
elseif (ans.D2Tgt <= R997 * 4) && (n==1)
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 3));
elseif (ans.D2Tgt <= R997 * 3) && n==2
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 2));
elseif (ans.D2Tgt <= R997 * 2) && n==3
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 1));
elseif (ans.D2Tgt <= R997 * 1) && (n>=4)
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
else
end
end

1 件のコメント

Emmanouil Tzorakoleftherakis
Emmanouil Tzorakoleftherakis 2023 年 12 月 21 日
Apologies, I am not following the question. Are you able to use D2Tgt inside the reset function? If yes, where is the problem exactly?

サインインしてコメントする。

回答 (0 件)

カテゴリ

ヘルプ センター および File ExchangeReinforcement Learning Toolbox についてさらに検索

製品

リリース

R2023b

質問済み:

2023 年 12 月 16 日

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by