I am guessing you are referring to the first empty list in the result. All the others look to be the same. This is expected behavior.
The reason is because the default config has a key called “create_env_on_driver” that is False by default. The behavior of foreach_env checks the driver and if it does not have an env it returns an empty list.
In the code snippet below local_worker/local_results refers to the driver.
just want to make sure if shuffle_sequences has an impact.
I haven’t set this “env.action_space.seed(RANDOM_SEED)” in the environment. I did this in the past while using SB3 (when we instantiate an object of the class and that too when we use random actions), can try here as well and confirm…