Deep Reinforcement Learning Toolbox with LSTM RNN

8 ビュー (過去 30 日間)
PB75
PB75 2021 年 7 月 15 日
Hi,
Currently I am using a LSTM RNN network in my TD3 agent model.
I am using a LSTM network architecture based on the DDPG TD3 the online help, as shown below
%Critic Network
rng(0) % fix the random seed
statePath1 = [
sequenceInputLayer(numObservations,'Normalization','none','Name','observation')
fullyConnectedLayer(400,'Name','CriticStateFC1')
reluLayer('Name','CriticStateRelu1')
fullyConnectedLayer(300,'Name','CriticStateFC2')
];
actionPath1 = [
sequenceInputLayer(numActions,'Normalization','none','Name','action')
fullyConnectedLayer(300,'Name','CriticActionFC1')
];
commonPath1 = [
additionLayer(2,'Name','add')
reluLayer('Name','CriticCommonRelu1')
lstmLayer(16,'OutputMode','sequence','Name','CriticLSTM');
fullyConnectedLayer(1,'Name','CriticOutput')
];
criticNet = layerGraph(statePath1);
criticNet = addLayers(criticNet,actionPath1);
criticNet = addLayers(criticNet,commonPath1);
criticNet = connectLayers(criticNet,'CriticStateFC2','add/in1');
criticNet = connectLayers(criticNet,'CriticActionFC1','add/in2');
%Actor Network
actorNet = [
sequenceInputLayer(numObservations,'Normalization','none','Name','observation')
fullyConnectedLayer(400,'Name','ActorFC1')
lstmLayer(8,'OutputMode','sequence','Name','ActorLSTM')
reluLayer('Name','ActorRelu1')
fullyConnectedLayer(300,'Name','ActorFC2')
reluLayer('Name','ActorRelu2')
fullyConnectedLayer(numActions,'Name','ActorFC3')
tanhLayer('Name','ActorTanh1')
];
My Mini Batch size is set to 200.
When my training begins the model runs very quickly, during the initial episodes, which is the same as the mini batch size of 200, then from episode 201 onward the model runs at a normal training pace and seems to start learning, this can be seen in the episode manager plot below.
My question is:
Is this initial training behavour normal for a LSTM RNN?
Cheers
Patrick

回答 (0 件)

カテゴリ

Find more on Deep Learning with Time Series and Sequence Data in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by