Is it possible to train LSTM Network without a Dataset?

In the following paper, they utilize Reinforcement Learning and within it, also use an LSTM network. On page 3, they say that they use some kind of loss function that allows the training of the LSTM network without a dataset. I was wondering how that could be possible? If someone could explain, I would greatly appreciate it.

Answer 1

[Emmanouil Tzorakoleftherakis]

In the paper they mention "Although a readily available dataset is required to train an LSTM network, we devised an efficient way to tackle this challenge utilizing the experiences stored in the replay memory of the Q-network".

This is how training works with experience buffers in RL - you don't have data at the beginning, then you run simulations and store the data you collect in the experience buffer, which you are then using to train the policy. So the data is not "readily available" but you are still sing your experience buffer.

[Huzaifah Shamim]

So is there a way in MATLAB for me to take the data collected in the experience buffer and input it into the LSTM network as I would like to replicate that aspect of the paper? Or would I have to add a lstmlayer somewhere in the following code to take care of that?

%% Setting Up DQN
hiddenLayerSize1 = 128;
hiddenLayerSize2 = 64;
DQNetwork = [
    imageInputLayer([N_cols 1 1],'Normalization','none','Name','Binary Vector')
    fullyConnectedLayer(hiddenLayerSize1,'Name','fc1')
%     reluLayer('Name','CriticReLu1')
    fullyConnectedLayer(hiddenLayerSize2,'Name','fc2')
%     reluLayer('Name','CriticReLu2')
    fullyConnectedLayer(2, 'Name', 'Val Corresponding to Action')
    ];