Reinforcement learning and Paralle computation
3 ビュー (過去 30 日間)
古いコメントを表示
I am condcting reinforcement learning with Sac agent.
I tried to use GPU and Parallel computation, but in case of using paralle computaion, the training result changed.
At all, learning is much worse than without parallel processing.
Do you know what caused it?
%% AGENT setting
agentOptions = rlSACAgentOptions;
agentOptions.SampleTime = Ts;
agentOptions.DiscountFactor = 0.90;
agentOptions.TargetSmoothFactor = 1e-3;
agentOptions.ExperienceBufferLength = 500;
agentOptions.MiniBatchSize = 256;
agentOptions.EntropyWeightOptions.TargetEntropy = -2;
agentOptions.NumStepsToLookAhead = 1;
agentOptions.ResetExperienceBufferBeforeTraining = false;
agent = rlSACAgent(actor,[critic1 critic2],agentOptions);
%% Learning setting
maxepisodes = 10000;
maxsteps = 1e6;
trainingOptions = rlTrainingOptions(...
'MaxEpisodes',maxepisodes,...
'MaxStepsPerEpisode',maxsteps,...
'StopOnError','on',...
'Verbose',true,...
'Plots','training-progress',...
'StopTrainingCriteria','AverageReward',...
'StopTrainingValue',Inf,...
'ScoreAveragingWindowLength',10);
trainingOptions.UseParallel = true;
trainingOptions.ParallelizationOptions.Mode = 'async';
trainingOptions.ParallelizationOptions.StepsUntilDataIsSent = 32;
trainingOptions.ParallelizationOptions.DataToSendFromWorkers = 'Experiences';
1 件のコメント
Takeshi Takahashi
2022 年 4 月 18 日
agentOptions.ExperienceBufferLength seems too short, which may indirectly affect the parallel training. Can you increase ExperienceBufferLength to 1e6 or more?
回答 (0 件)
参考
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!