RL DDPG Actions have high oscillation
12 ビュー (過去 30 日間)
古いコメントを表示
Hello, I am using the DDPG Reinforcement learning toolbox in matlab to train a 3DOF robotic arm to move. the actions are joint torques, and although the actions reach the target, they are highly oscillating and noisy.
Can anyone help explane where this comes from ? ie: the algorithm itself, noise options ....
I am using the walking robot example to build noise options:
%% DDPG Agent Options
agentOptions = rlDDPGAgentOptions;
agentOptions.SampleTime = 0.025;
agentOptions.DiscountFactor = 0.99;
agentOptions.MiniBatchSize = 128;
agentOptions.ExperienceBufferLength = 5e5;
agentOptions.TargetSmoothFactor = 1e-3;
agentOptions.NoiseOptions.MeanAttractionConstant = 0.5;
agentOptions.NoiseOptions.Variance = 0.3;
agentOptions.NoiseOptions.VarianceDecayRate = 1e-5;
i think it might have something to do with MeanattractionConstant, varience, or varience decay. (by the way, the joint limits are between -3,3).
the actions i get look like this :
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/1534032/image.bmp)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/1534037/image.bmp)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/1534042/image.bmp)
0 件のコメント
回答 (1 件)
Emmanouil Tzorakoleftherakis
2023 年 11 月 9 日
Hi,
The noise options you are mentioning are only used during training and are essential for exploration. If the plots you are showing above are from training, you may consider reducing the noise variance a bit.
If the plots you are showing are from the trained agent, you can consider penalizing large action changes in your reward signal. That would help reduce the oscillatory content.
Hope this helps
8 件のコメント
Sourabh
2023 年 12 月 15 日
Actually i have a signal and i want to sample that signal at interval of 4 sec to make a array and then feed that array to my observation. Can i do it using rate transition block
参考
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!