Sharing "common knowledge" with __common__ key in info dictionary

How severe does this issue affect your experience of using Ray?

  • Low: It annoys or frustrates me for a moment.

Hello, in older versions of ray/RLlib, there was an option to store arbitrary data such as “common knowledge” information in the info dict under keys of a user’s choice, including the key common which is still shown in the codebase on these two lines here: ray/ at releases/2.2.0 · ray-project/ray · GitHub

However, in the current ray 2.2.0 MultiAgentEnvWrapper, there is an assert that specifically prevents using the __common__ key in the info dict. Specifically, if there is a __common__ key set in the dictionary, this code checks that the key is unrecognized in the obs dict, and thus throws the assert: ray/ at 7c58114bdf7836eaef27f3d30056c1a7a4b700c3 · ray-project/ray · GitHub

Also for some reason with 0 num_workers, this assert is never reached. But with num_workers greater than or equal to 1, the assert is hit… which seems like unintended behavior…

I am blocked here because I cannot replace the MultiAgentEnvWrapper with my own custom wrapper (to remove the assert), and I need to use __common__ in my info dict.

Could the RLlib team remove the assert? And/or is there a better way to maintain common info…?

Thank you!

An issue highlighting this request was made in the ray GitHub repo: [RLLib] __common__ key not supported in info dict of vectorized MultiAgentEnv · Issue #32267 · ray-project/ray · GitHub

Hi @wcode,

I am not sure if you saw this but I think I replied to a similar question on slack. Just my thoughts on the current limitation.

Hi @mannyv your solution is an OK workaround for the meantime. However, it necessitates copying redundant data per agent, which scales poorly and could have a nontrivial impact on performance depending on the number of agents and how much common knowledge is stored.