How to save an rl agent after every 1000 episodes?

23 ビュー (過去 30 日間)
Guru Bhargava Khandavalli
Guru Bhargava Khandavalli 2021 年 3 月 7 日
編集済み: Lance 2023 年 6 月 29 日
I am training a DDPG agent where the training runs over 1000 episodes. To see how it evolves, I would like to save the agents after every 1000 episodes. As i see the options available in rlTrainingOptions, it is only possible to save every agent after a critical value. This slows down the training process significantly because saving every agent consumes a lot of time. Is there an efficient way to save the agents only after every 1000 episodes?
Thank you.
  1 件のコメント
Heesu Kim
Heesu Kim 2021 年 3 月 12 日
編集済み: Heesu Kim 2021 年 3 月 19 日
I agree with this. I don't understand why it does not have the most useful option. And I'm disappointed that this question still doesn't have any answer.

サインインしてコメントする。

採用された回答

Madhav Thakker
Madhav Thakker 2021 年 3 月 19 日
Hi Guru,
I understand currently in rlTrainingOptions, there is no option to save the agent after specific number of episodes. I have raised an enhancement request for the same and this might be considered in future releases.
Hope this helps.
  2 件のコメント
Heesu Kim
Heesu Kim 2021 年 3 月 19 日
Yay!
Dmitriy Ogureckiy
Dmitriy Ogureckiy 2023 年 1 月 20 日
I am the same opinion. Add this, please. Ball on your side.

サインインしてコメントする。

その他の回答 (2 件)

Manuel Sebastian Rios Beltran
Manuel Sebastian Rios Beltran 2022 年 6 月 2 日
@Madhav Thakker But they did not do it :( a year later
  1 件のコメント
Matteo D'Ambrosio
Matteo D'Ambrosio 2023 年 5 月 28 日
2 years later! Agreed that this enhancement is much needed...

サインインしてコメントする。


Lance
Lance 2023 年 6 月 23 日
編集済み: Lance 2023 年 6 月 29 日
From what I understand, the only other work around would be to write another training command. You would have to predfine this for every "checkpoint" ie. 10,20,30 episodes. The training-progress graph will continue to be actively updated. (Note I am using R2022a)
% Define all agents, observations, actions, environment, etc....
maxepisodes=500;
trainingOpts=rlMultiAgentTrainingOptions;
trainingOpts.SaveAgentCriteria="EpisodeCount";
trainingOpts.SaveAgentValue=maxepisodes
trainingStats=train([agent1,agent2],environment,trainingOpts); % Will train to max episodes and save agent
% Edit Trainingoptions to increase maxepisodes and save agent value
trainingStats(1,1).TrainingOptions.MaxEpisodes=1000;
trainingStats(1,1).TrainingOptions.SaveAgentValue=[1000,1000];
trainnigStats(1,1).TrainingOptions.StopTrainingValue=[1000,1000];
trainingStats(1,2).TrainingOptions.MaxEpisodes=1000;
trainingStats(1,2).TrainingOptions.SaveAgentValue=[1000,1000];
trainnigStats(1,2).TrainingOptions.StopTrainingValue=[1000,1000];
% Resume training -- Will train to 1000 episodes and save agent again
trainingStats2=train([agent1,agent2],environment,trainingStats) %Note you use trainingStats here not trainingOpts
Let me know if this helps!

カテゴリ

Help Center および File ExchangeTraining and Simulation についてさらに検索

製品


リリース

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by