How to implement multi-agent RL training with custom MATLAB environment function?

Hello everyone,

I have implemented one custom RL environment with MATLAB template environment class. I want to introduce multi-agents to the environment. I find all three examples provided for multi-agent RL are based on Simulink. My question is if it is at all possible to do the same with a MATLAB function? Or do I need to implement my custom RL environment in simulink to work with multi-agent RL?

Thanks.

[Arman Ali]

Need help on this?

Answer 1

[Emmanouil Tzorakoleftherakis]

Hello,

Currently multi-agent training is only supported in Simulink. If you have an environment created in MATLAB you could copy and paste the core parts like the reward and step function into a MATLAB Fcn block in Simulink as a workaround.

Edit:

Multi-agent training is also available in MATLAB as of R2023b release. For an example see here.

Please take a look at the release notes for a list of new features per release.

[laha_M]

Thanks Emmanouil.

I was wondering if we could do something like this. Lets say my single agent has two actions [A; B]. Now to make it sort of like multiagent (lets say 3 agents), I write my actions as [A1, A2, A3; B1, B2, B3] i.e. passing them as a vector of actions where A1, B1 are the actions of the first agent and so on.

I know its not a multi-agent. Rather its like single agent with higher action spaces. Does it make sense? Will it work?

" If you have an environment created in MATLAB you could copy and paste the core parts like the reward and step function into a MATLAB Fcn block in Simulink"-- I am not comfortable with simulink. Can you please share some similar examples/documentations on how to covert a MATLB functions to simulink blocks?

Thanks.

[Emmanouil Tzorakoleftherakis]

Yes that would likely be another way to set up the problem.

There are a lot of examples in the doc for MATLAB Fcn block - maybe you can start with this one.

[laha_M]

Thanks.

[Maha Mosalam]

yes please share the latest update in this is a custom Fn to multi agent availble or any one got a solutions for that

[Arman Ali]

I am trying using single agent with multiple actions where each action represents individual agent output, I have also compared it with multiagent system and it seems like both results are similar. The benefit of using single agent with multiple space/action combination is that it make the system fast using simulink. I don't know if it is a better approach but somehow it's working and I will be using it until multiagent is available in Matlab, because simulink version of multiagent is too slow.

Answer 2

[Arman Ali]

anyone can guide on this further?

[Maha Mosalam]

yes please share the latest update in this is a custom Fn to multi agent availble or any one got a solutions for that