Grid World on Simulink

Hello everyone!

I'd like to apply some contrained optimization on the Grid World environment (https://nl.mathworks.com/help/reinforcement-learning/ug/train-q-learning-agent-to-solve-basic-grid-world.html?searchHighlight=grid%20world&s_tid=srchtitle_grid%20world_5). I would like to have the enviroment and the agent on Simulink.

Does anyone have it already developed or could give me some advice on how to go about it?

Thanks!

Answer 1

[Ayush Aniket]

Hi GCats,

As per my understanding the grid World Environment is not already available in Simulink. However, since this environment consists of a countable number of states (grids) and the transition structure is also specified it is equivalent to a finite state machine. Hence, you can leverage Stateflow chart block in Simulink to model this. You can refer to the following link to learn more about it:

https://in.mathworks.com/help/stateflow/gs/stateflow-charts.html

The grids of the environment (states) can be modeled by Stateflow states and the actions for that grid can also be defined therein. Once the environment structure is modeled, you can use the ‘createIntegratedEnv’ function to create an ‘env’ variable from your Simulink model. You can read more about this in the following documentation page:

https://www.mathworks.com/help/reinforcement-learning/ref/createintegratedenv.html

Regarding the agent, you can use the RL agent block to model an RL agent in Simulink. Any type of agent can be defined in the base MATLAB workspace and then associated with the RL Agent block. All the input signals required by this block such as ‘observation’, ‘reward’, and ‘isDone’ will be fed from the Stateflow chart Grid World model. For more information about RL agent block you can refer to the following documentation page:

https://in.mathworks.com/help/reinforcement-learning/ref/rlagent.html

Once the agent and the environment have been modeled, the training can follow similar steps as in the following example link:

https://in.mathworks.com/help/reinforcement-learning/ug/create-simulink-environment-and-train-agent.html

Hope it helps.