multi-agent deep reinforcement learning

I designed the deep reinforcement learning multi-agent system with three DDPG agents. Each agent does an independent task. I prepared a counter to calculate the total rewards of each agent in each episode in the Simulink. The calculated total rewards in each episode for each agent are different from the calculated rewards of each agent in the Matlab training-progress of Reinforcement Learning Episode Manager. But for a single agent in the reinforcement learning system, these rewards were the same.

1) Are the rewards calculated by the algorithm in the multi-agent reinforcement system influenced by the calculations of other agent's rewards?

2) Is the update of the weights of the network of each agent not independent of the other agents?

Answer 1

[Ari Biswas]

Assuming you are training multiple agents in Simulink using the Reinforcement Learning Toolbox in R2020b:

1. The rewards are calculated by the environment, not the agent algorithm so they should not be affected unless the environment is changing them. When you compare rewards between single and multi-agents please ensure that the state-action pairs are the same. Rewards depend on states and actions and you may get different results for different state-action pairs.
2. In R2020b, the agent neural networks are updated independently.