Bipedal walking robot TD3 training example bad convergence

Hi all,

I have attempted to run the bipedal walking robot example training myself and it converged to an suboptimal solution. I used the TD3 agent training and also used gpu to host my actor and critic.

The final simulation shows that the robot learnt to fall at the start of the simulation. Why does my training produce significantly different results compared to the example? Did hosting the networks on the gpu caused this?

Here's the training plot. Note that the maximum reward was only 35 compared to the 250 shown in the example.

Thank you :)