Parallel training is not supported for agent with state

When trying to use the parallel option when training my RL agent with a simulink model i get a pop up window saing ;

"Parallel training is not supported for agent with state"

Can anyone help explain what this means?

[Tanjulee Siddique]

Hi Will, I'm currently facing the same issue, did you find a workaround it?

[will aston]

No fix for this issue yet. I'll keep you updated if I resolve it

[Timothy Lawrence]

When you use an agent with the LSTM option the agent has internal state because RNNs implement memory. Note the second sentence on this page.

"Note that parallel training and simulation of agents using recurrent neural networks, or agents within multi-agent environments, is not supported."

There is a way to paralelize training of RNNs but MATLAB has not yet implemeneted it. I hope they do this in an upcoming release soon.

[Timothy Lawrence]

Here is another research paper on how to train RNNs in paralell. I hope MathWorks implements this soon.

[Timothy Lawrence]

It is obvious that MathWorks knows how to solve this because the Experiment Manager App supports paralell processes using LSTM networks in this example.I tested it using paralell batches and it works fine. I will need to code my own primitive paralell experiment manager type app for RL using parfor instead of using the native app, which would have been much preferred.