This is a known issue: when using DQN with the new API stack and MultiAgentReplayBuffer, the buffer’s add() method expects a SampleBatch, MultiAgentBatch, or dict, but receives a list of MultiAgentEpisodes, which does not have the .as_multi_agent() method, resulting in AttributeError. This is confirmed in recent user reports and is not resolved in Ray 2.53.0. There is currently no official workaround in the documentation, and the problem has persisted across several Ray versions. Switching to the old API stack or using a different replay buffer type may avoid the error, but the new API stack with MultiAgentReplayBuffer is not functioning as expected for multi-agent DQN training at this time. See detailed discussion and error trace in the linked sources.
Would you like more detail on possible workarounds or alternative configurations?
Sources:
- Multi-agent replay buffer in DQN fails to run
- Multi-agent prioritized replay buffer giving me trouble in DQN
Hint: Mention @RunLLM in the post for followups.