Reinforcement Learning Noise Model Mean Attraction Constant
3 ビュー (過去 30 日間)
古いコメントを表示
Tech Logg Ding
2020 年 12 月 4 日
What does the mean attraction constant do? How can I tune it properly to promote exploration and learning? I can't seem to get the logic behind it.
With a sample time of 2, when I set it to 1 I get very noisy outputs. In the following graphs, rpm and valve%opening are the agents outputs and they are already scaled by a scaling layer.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/444468/image.png)
When I set it to 0.05, then it seems like the noise model is not doing much explorations.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/444473/image.png)
I also noticed that by applying the abs(1 - MeanAttractionConstant.*SampleTime) formula,
When sample time is 2 and the MAC is 1, the formula gives 1.
When sample time is 2 and the MAC is 0.05, the formula gives 0.9.
How does this relate to how fast the noise converge to the mean?
Thank you very much.
0 件のコメント
採用された回答
Emmanouil Tzorakoleftherakis
2020 年 12 月 4 日
Assuming you are using DDPG, there is some information on the noise model here. I wouldn't worry too much about the mean attraction constant. The value of variance, variancedecayrate and variancemin play a much bigger role on 1) how much noise is added to the agent output and 2) for how long. If you want less noise to be added, reduce the variance value. If you want to explore for longer time, reduce the decay rate and set variancemin to a larger value.
4 件のコメント
Sourabh
2024 年 1 月 9 日
i am using DDPG and i need to set my sample time to 800 sec and then i got error as
abs(1 - mean attrc const.*sample time) <= 1
so i made mean att cont.(mac) to 0.0001 but still i am getting the same error
my question is i have to change the mac in noise options of agent or is it some different mean attarc const.
その他の回答 (0 件)
参考
カテゴリ
Help Center および File Exchange で Training and Simulation についてさらに検索
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!