Error when setting done=true: eval_data[i].env_id yields IndexError: list index out of range

[sven1977]

Hey @nathanlct , yeah, there was a similar bug in RLlib that was fixed here:

[RLlib] Issue #14022: Trajectory View API fails in MA-env where a new agent terminates right away (done=True right after initial obs).

I think this should fix your problem as well. Yes, it happened when a new(!) agent enters the episode and at the same time step, the episode terminates, such that this agent has an initial obs, but no action had to be calculated.