How to Handle Agent Death In MultiAgent Scenarios

RLlib
1

How severe does this issue affect your experience of using Ray?

  • None: Just asking a question out of curiosity

In multi-agent environments, using RLlibs MultiAgentEnv class, if you have 1 agent which is set to terminated or truncated and another is not, what special handling must be done? for example

terminated: {'agent_0': False, 'agent_1': False, '__all__': False}
truncated: {'agent_0': False, 'agent_1': False, '__all__': False}

if something in the env sets it to

terminated: {'agent_0': True, 'agent_1': False, '__all__': False}
truncated: {'agent_0': False, 'agent_1': False, '__all__': False}

or

terminated: {'agent_0': False, 'agent_1': False, '__all__': False}
truncated: {'agent_0': True, 'agent_1': False, '__all__': False}

Do I need to remove “agent_0” from all dicts passed? or is there another variable I need to set so training continues for that episode, until time limit is reached or all is true for either truncated or terminated? Such that at the next env step the dicts will look like?

terminated: { ‘agent_1’: False, ‘all’: False}
truncated: { ‘agent_1’: False, ‘all’: False}

Does this need done to observations as well? I assume no action is passed for truncated or terminated agent.

2

Hi @VisionZUS29 ,

I believe once you have reported the decease of one of your agents via terminated or truncated, you should not report it again in the next env.step(). So you might want to delete the key of that agent in all the dicts you return.

As mentioned in these answers: 1 and 2.

Regards,

JL