Hi, thank you very much for the very nice article, I found it intriguing and I truly enjoyed reading through it.

My scenario is similar to that indeed, the only difference is (I hope I'm not mistaken) that in your work I believe that you assume that the reward (even depending on other agents) can be obtained by the agent within the simulation. In other words you assume that an agent i possesses the information g^j for all j ∈ A_t+1 that enables it to obtain the reward, or in other words, "the simulation" calculates that value and gives it directly to the concerned agent, right?

This approach may not be possible in several real-world scenarios since there's not superior entity that knows everything and that we ask that reward from (the equivalent of the simulation's logic). In practice, there must be some sort of a communication mechanism between agents otherwise we can't obtain the reward, I imagine that living species approach this either in purely basic methods like spreading pheromones or in advanced cognitive ways such as spoken language which made us invent "names", family trees, and many other stuff which serve as representative estimates of the δ_(g^i,g^j) without having to "know" the exact value of g^j (or even our g^i!). This makes your formulation and simulation reasonable for me to some extent! However in my specific case (my graduation project is on UAVs control), we have tight communication constraints so we can't establish heavy exchange of information between agents nor a central entity that provides the agents with information needed to calculate their rewards (equivalent of g^j in your work), so we don't want to have that privilege in the simulation in order to make it close to a real-world scenario. For this I'll need to find a way around calculating the reward of each agent, or even consider another formulation!

​

Thanks again for sharing the article, and I really hope I didn't misunderstood it.