How can we reduce the overshoot in a controller trained with reinforcement learning while it is tracking a square wave.

We tried to make a controller for a rotary flexible link utilising reinforcement learning. We based our system on the example given on the matlab websiteof the water tank system. But in our system there is a very high initial overshoot which we have not been able to reduce. Can someone please help us with our problem. We have also attached a screenshot of our obtained performance.

Answer 1

[Emmanouil Tzorakoleftherakis]

Hello,

It's all about the reward signal in RL. It's not like with PIDs where you can play with gains and you know from theory what outcome to expect. Here I would probably add a term in the reward that penalizes the agent more when the error is >some threshold.