how to create a critic with recurent neuronal network when the INPUT data(obsinfo) is matrix dimension(5x1320000)

obsInfo = rl.util.rlFiniteSetSpec({TRAINDATA});

ActInfo = rl.util.rlFiniteSetSpec([0, 1]);

env = rlFunctionEnv(obsInfo, ActInfo,'myStepFunction','myResetFunction');

type myStepFunction.m

type myResetFunction.m

dnn = [

featureInputLayer(obsInfo.Dimension(5x1320000),'Normalization','none','Name','obsInfo ')

fullyConnectedLayer(24,'Name','CriticStateFC1')

reluLayer('Name','CriticRelu1')

fullyConnectedLayer(24, 'Name','CriticStateFC2')

reluLayer('Name','CriticCommonRelu')

fullyConnectedLayer(length(actInfo.Elements),'Name','output')];

critic = rlQValueRepresentation(dnn,obsInfo,actInfo,'Observation',{'state'},criticOpts);

criticOpts = rlRepresentationOptions('LearnRate', 0.001, 'GradientThreshold', 1);

Answer 1

[praguna manvi]

Hi,

To build a critic with RNN consider using the below code snippet that uses “sequenceInputLayer” connected to “lstmLayer” which is then connected to "fullyConnectedLayer" and “relu” as shown below :

obsInfo = rlNumericSpec([5 1320000]);
actInfo = rlFiniteSetSpec([0, 1]);

% Create the environment
env = rlFunctionEnv(obsInfo, actInfo, @myStepFunction, @myResetFunction);

% Define the critic network with an LSTM layer
dnn = [
    sequenceInputLayer([5 1320000],'Normalization','none','Name','obsInfo')
    lstmLayer(24,'OutputMode','sequence','Name','CriticLSTM')
    fullyConnectedLayer(24,'Name','CriticStateFC1')
    reluLayer('Name','CriticRelu1')
    fullyConnectedLayer(24, 'Name','CriticStateFC2')
    reluLayer('Name','CriticCommonRelu')
    fullyConnectedLayer(length(actInfo.Elements),'Name','output')];

% Define critic options
criticOpts = rlRepresentationOptions('LearnRate', 0.001, 'GradientThreshold', 1);

% Create the critic representation
critic = rlQValueRepresentation(dnn, obsInfo, actInfo, 'Observation', {'obsInfo'}, criticOpts);

Please refer to the following documentation on how to build a network on sequential input:

https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.sequenceinputlayer.html?searchHighlight=sequenceInputLayer&s_tid=srchtitle_support_results_1_sequenceInputLayer