Saving Trained RL Agent after Training

87 ビュー (過去 30 日間)
PB75
PB75 2021 年 4 月 29 日
コメント済み: Zaid Jaber 2023 年 11 月 14 日
Hi All,
I trained a RL agent, the environment output was acceptable, my plan was to initially validate the agent in the simulation after training finished with the following code.
As i was concerned that I would restart training on the agent when I ran the script to run the 'sim' function, my IsDone flag in the simulation was manually set to 1 (previously 0 to permit training) and additionally commented out the 'training' function.
%trainingStats = train(agentSS,env,trainingOpts)
rng(0)
simOptions = rlSimulationOptions('MaxSteps',maxsteps);
experience = sim(env,agentSS,simOptions);
There was no ouput from the simulation, with no warnings, I then reset the IsDone flag back to 0, and reran the script, now the ouput was 0 on all scopes.
Did I lose the trained agent data when I set the IsDone flag to 1 after training?.
My next step was to try to save the trained agent with adding the following code found in the documentation, but still joy. My thoughts are I have overwritten and lost the trained data!
save("initialAgent.mat","agentSS")
load('initialAgent.mat')
rng(0)
simOptions = rlSimulationOptions('MaxSteps',maxsteps);
experience = sim(env,agentSS,simOptions);
How can I add code to ensure the trained agent data is saved automatically via 'RLTrainingOptions' after training has been completed, such as when maxepisodes are reached? Do not want to make the same mistake.
Is this correct?
trainingOpts = rlTrainingOptions(...
'MaxEpisodes',maxepisodes, ...
'MaxStepsPerEpisode',maxsteps, ...
'StopTrainingCriteria','AverageReward',...
'StopTrainingValue',-100,...
'ScoreAveragingWindowLength',100,...
'SaveAgentCriteria',"EpisodeCount",...
'SaveAgentValue',maxepisodes,...
'SaveAgentDirectory',"savedAgents")
Thanks
Patrick

採用された回答

Emmanouil Tzorakoleftherakis
Emmanouil Tzorakoleftherakis 2021 年 4 月 29 日
編集済み: Emmanouil Tzorakoleftherakis 2021 年 4 月 29 日
Setting the IsDone flag to 1 does not erase the trained agent - it actually makes sense that the sim was not showing anything because it was immediately stopped by the IsDone flag.
To save the final agent, simply add the save command you have right after when you call 'train'.
My guess is that when you reran the whole script, you created a new agent from scratch and saved it again to a mat file, which replaced the already trained agent. This is why it's good practive to always have sections in your (live) script, so that you can pick exactly what lines you want to run.
  3 件のコメント
Apoorv Pandey
Apoorv Pandey 2023 年 2 月 27 日
How to save multiple agents like in this example
https://in.mathworks.com/help/reinforcement-learning/ug/train-agents-for-path-following.html
Zaid Jaber
Zaid Jaber 2023 年 11 月 14 日
Hi
How i can use that file to resume training with more number of episodes ?

サインインしてコメントする。

その他の回答 (0 件)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by