Following the first example of RLlib: Industry-Grade Reinforcement Learning — Ray 2.0.0 running PPO on Taxi-v3 with pytorch. However the code doesn’t work and output is hard to interpret why it is happening. Is the example code on the doc just pseudocode that needs additional customization?
2022-09-22 17:08:21,878 WARNING algorithm.py:2100 -- You have specified 1 evaluation workers, but your `evaluation_interval` is None! Therefore, evaluation will not occur automatically with each call to `Trainer.train()`. Instead, you will have to call `Trainer.evaluate()` manually in order to trigger an evaluation run.
(pid=27557) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(pid=27557) 2022-09-22 17:08:23,403 WARNING compression.py:18 -- lz4 not available, disabling sample compression. This will significantly impact RLlib performance. To install lz4, run `pip install lz4`.
(pid=27557) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(pid=27558) 2022-09-22 17:08:23,416 WARNING compression.py:18 -- lz4 not available, disabling sample compression. This will significantly impact RLlib performance. To install lz4, run `pip install lz4`.
(RolloutWorker pid=27557) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(RolloutWorker pid=27557) 2022-09-22 17:08:23,945 WARNING env.py:155 -- Your env reset() method appears to take 'seed' or 'return_info' arguments. Note that these are not yet supported in RLlib. Seeding will take place using 'env.seed()' and the info dict will not be returned from reset.
(RolloutWorker pid=27557) 2022-09-22 17:08:23,948 ERROR worker.py:756 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=27557, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f2c558e6990>)
(RolloutWorker pid=27557) ValueError: The two structures don't have the same nested structure.
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) First structure: type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) Second structure: type=int str=408
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) More specifically: Substructure "type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=408" is not
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) During handling of the above exception, another exception occurred:
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) ray::RolloutWorker.__init__() (pid=27557, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f2c558e6990>)
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 72, in check_env
(RolloutWorker pid=27557) check_gym_environments(env)
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 186, in check_gym_environments
(RolloutWorker pid=27557) reset_obs, sampled_observation
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/spaces/space_utils.py", line 359, in convert_element_to_space_type
(RolloutWorker pid=27557) return tree.map_structure(map_, element, sampled_element, check_types=False)
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 428, in map_structure
(RolloutWorker pid=27557) assert_same_structure(structures[0], other, check_types=check_types)
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 287, in assert_same_structure
(RolloutWorker pid=27557) % (e, str1, str2))
(RolloutWorker pid=27557) ValueError: The two structures don't have the same nested structure.
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) First structure: type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) Second structure: type=int str=408
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) More specifically: Substructure "type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=408" is not
(RolloutWorker pid=27557) Entire first structure:
(RolloutWorker pid=27557) (., {'prob': ., 'action_mask': .})
(RolloutWorker pid=27557) Entire second structure:
(RolloutWorker pid=27557) .
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) During handling of the above exception, another exception occurred:
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) ray::RolloutWorker.__init__() (pid=27557, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f2c558e6990>)
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 495, in __init__
(RolloutWorker pid=27557) check_env(self.env)
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 84, in check_env
(RolloutWorker pid=27557) f"{actual_error}\n"
(RolloutWorker pid=27557) ValueError: Traceback (most recent call last):
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 280, in assert_same_structure
(RolloutWorker pid=27557) _tree.assert_same_structure(a, b, check_types)
(RolloutWorker pid=27557) ValueError: The two structures don't have the same nested structure.
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) First structure: type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) Second structure: type=int str=408
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) More specifically: Substructure "type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=408" is not
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) During handling of the above exception, another exception occurred:
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) ray::RolloutWorker.__init__() (pid=27557, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f2c558e6990>)
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 72, in check_env
(RolloutWorker pid=27557) check_gym_environments(env)
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 186, in check_gym_environments
(RolloutWorker pid=27557) reset_obs, sampled_observation
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/spaces/space_utils.py", line 359, in convert_element_to_space_type
(RolloutWorker pid=27557) return tree.map_structure(map_, element, sampled_element, check_types=False)
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 428, in map_structure
(RolloutWorker pid=27557) assert_same_structure(structures[0], other, check_types=check_types)
(RolloutWorker pid=27557) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 287, in assert_same_structure
(RolloutWorker pid=27557) % (e, str1, str2))
(RolloutWorker pid=27557) ValueError: The two structures don't have the same nested structure.
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) First structure: type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) Second structure: type=int str=408
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) More specifically: Substructure "type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=408" is not
(RolloutWorker pid=27557) Entire first structure:
(RolloutWorker pid=27557) (., {'prob': ., 'action_mask': .})
(RolloutWorker pid=27557) Entire second structure:
(RolloutWorker pid=27557) .
(RolloutWorker pid=27557)
(RolloutWorker pid=27557) The above error has been found in your environment! We've added a module for checking your custom environments. It may cause your experiment to fail if your environment is not set up correctly. You can disable this behavior by setting `disable_env_checking=True` in your environment config dictionary. You can run the environment checking module standalone by calling ray.rllib.utils.check_env([env]).
(RolloutWorker pid=27558) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(RolloutWorker pid=27558) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(RolloutWorker pid=27558) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(RolloutWorker pid=27558) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(RolloutWorker pid=27558) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(RolloutWorker pid=27558) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(RolloutWorker pid=27558) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(RolloutWorker pid=27558) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
(RolloutWorker pid=27558) RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
---------------------------------------------------------------------------
RayActorError Traceback (most recent call last)
~/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/algorithms/algorithm.py in setup(self, config)
424 local_worker=True,
--> 425 logdir=self.logdir,
426 )
~/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/evaluation/worker_set.py in __init__(self, env_creator, validate_env, policy_class, trainer_config, num_workers, local_worker, logdir, _setup)
126 num_workers,
--> 127 validate=trainer_config.get("validate_workers_after_construction"),
128 )
~/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/evaluation/worker_set.py in add_workers(self, num_workers, validate)
268 if validate:
--> 269 self.foreach_worker(lambda w: w.assert_healthy())
270
~/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/evaluation/worker_set.py in foreach_worker(self, func)
390 local_result = [func(self.local_worker())]
--> 391 remote_results = ray.get([w.apply.remote(func) for w in self.remote_workers()])
392 return local_result + remote_results
~/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/_private/client_mode_hook.py in wrapper(*args, **kwargs)
104 return getattr(ray, func.__name__)(*args, **kwargs)
--> 105 return func(*args, **kwargs)
106
~/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/_private/worker.py in get(object_refs, timeout)
2276 else:
-> 2277 raise value
2278
RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=27557, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f2c558e6990>)
ValueError: The two structures don't have the same nested structure.
First structure: type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})
Second structure: type=int str=408
More specifically: Substructure "type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=408" is not
During handling of the above exception, another exception occurred:
ray::RolloutWorker.__init__() (pid=27557, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f2c558e6990>)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 72, in check_env
check_gym_environments(env)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 186, in check_gym_environments
reset_obs, sampled_observation
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/spaces/space_utils.py", line 359, in convert_element_to_space_type
return tree.map_structure(map_, element, sampled_element, check_types=False)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 428, in map_structure
assert_same_structure(structures[0], other, check_types=check_types)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 287, in assert_same_structure
% (e, str1, str2))
ValueError: The two structures don't have the same nested structure.
First structure: type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})
Second structure: type=int str=408
More specifically: Substructure "type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=408" is not
Entire first structure:
(., {'prob': ., 'action_mask': .})
Entire second structure:
.
During handling of the above exception, another exception occurred:
ray::RolloutWorker.__init__() (pid=27557, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f2c558e6990>)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 495, in __init__
check_env(self.env)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 84, in check_env
f"{actual_error}\n"
ValueError: Traceback (most recent call last):
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 280, in assert_same_structure
_tree.assert_same_structure(a, b, check_types)
ValueError: The two structures don't have the same nested structure.
First structure: type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})
Second structure: type=int str=408
More specifically: Substructure "type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=408" is not
During handling of the above exception, another exception occurred:
ray::RolloutWorker.__init__() (pid=27557, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f2c558e6990>)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 72, in check_env
check_gym_environments(env)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 186, in check_gym_environments
reset_obs, sampled_observation
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/spaces/space_utils.py", line 359, in convert_element_to_space_type
return tree.map_structure(map_, element, sampled_element, check_types=False)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 428, in map_structure
assert_same_structure(structures[0], other, check_types=check_types)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 287, in assert_same_structure
% (e, str1, str2))
ValueError: The two structures don't have the same nested structure.
First structure: type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})
Second structure: type=int str=408
More specifically: Substructure "type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=408" is not
Entire first structure:
(., {'prob': ., 'action_mask': .})
Entire second structure:
.
The above error has been found in your environment! We've added a module for checking your custom environments. It may cause your experiment to fail if your environment is not set up correctly. You can disable this behavior by setting `disable_env_checking=True` in your environment config dictionary. You can run the environment checking module standalone by calling ray.rllib.utils.check_env([env]).
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
<ipython-input-5-6ae8e1a61a63> in <module>
29
30 # Create our RLlib Trainer.
---> 31 algo = PPO(config=config)
32
33 # Run it for n training iterations. A training iteration includes
~/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/algorithms/algorithm.py in __init__(self, config, env, logger_creator, **kwargs)
306 }
307
--> 308 super().__init__(config=config, logger_creator=logger_creator, **kwargs)
309
310 # Check, whether `training_iteration` is still a tune.Trainable property
~/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/tune/trainable/trainable.py in __init__(self, config, logger_creator, remote_checkpoint_dir, custom_syncer)
155 start_time = time.time()
156 self._local_ip = self.get_current_ip()
--> 157 self.setup(copy.deepcopy(self.config))
158 setup_time = time.time() - start_time
159 if setup_time > SETUP_TIME_THRESHOLD:
~/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/algorithms/algorithm.py in setup(self, config)
441 # - e.args[0].args[2]: The original Exception (e.g. a ValueError due
442 # to a config mismatch) thrown inside the actor.
--> 443 raise e.args[0].args[2]
444 # In any other case, raise the RayActorError as-is.
445 else:
ValueError: Traceback (most recent call last):
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 280, in assert_same_structure
_tree.assert_same_structure(a, b, check_types)
ValueError: The two structures don't have the same nested structure.
First structure: type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})
Second structure: type=int str=408
More specifically: Substructure "type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=408" is not
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 72, in check_env
check_gym_environments(env)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 186, in check_gym_environments
reset_obs, sampled_observation
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/spaces/space_utils.py", line 359, in convert_element_to_space_type
return tree.map_structure(map_, element, sampled_element, check_types=False)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 428, in map_structure
assert_same_structure(structures[0], other, check_types=check_types)
File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 287, in assert_same_structure
% (e, str1, str2))
ValueError: The two structures don't have the same nested structure.
First structure: type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})
Second structure: type=int str=408
More specifically: Substructure "type=tuple str=(402, {'prob': 1.0, 'action_mask': array([0, 1, 0, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=408" is not
Entire first structure:
(., {'prob': ., 'action_mask': .})
Entire second structure:
.
The above error has been found in your environment! We've added a module for checking your custom environments. It may cause your experiment to fail if your environment is not set up correctly. You can disable this behavior by setting `disable_env_checking=True` in your environment config dictionary. You can run the environment checking module standalone by calling ray.rllib.utils.check_env([env]).
(RolloutWorker pid=27558) 2022-09-22 17:08:23,954 ERROR worker.py:756 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=27558, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f52e0d3d810>)
(RolloutWorker pid=27558) ValueError: The two structures don't have the same nested structure.
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) First structure: type=tuple str=(332, {'prob': 1.0, 'action_mask': array([1, 1, 1, 0, 0, 0], dtype=int8)})
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) Second structure: type=int str=136
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) More specifically: Substructure "type=tuple str=(332, {'prob': 1.0, 'action_mask': array([1, 1, 1, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=136" is not
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) During handling of the above exception, another exception occurred:
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) ray::RolloutWorker.__init__() (pid=27558, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f52e0d3d810>)
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 72, in check_env
(RolloutWorker pid=27558) check_gym_environments(env)
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 186, in check_gym_environments
(RolloutWorker pid=27558) reset_obs, sampled_observation
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/spaces/space_utils.py", line 359, in convert_element_to_space_type
(RolloutWorker pid=27558) return tree.map_structure(map_, element, sampled_element, check_types=False)
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 428, in map_structure
(RolloutWorker pid=27558) assert_same_structure(structures[0], other, check_types=check_types)
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 287, in assert_same_structure
(RolloutWorker pid=27558) % (e, str1, str2))
(RolloutWorker pid=27558) ValueError: The two structures don't have the same nested structure.
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) First structure: type=tuple str=(332, {'prob': 1.0, 'action_mask': array([1, 1, 1, 0, 0, 0], dtype=int8)})
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) Second structure: type=int str=136
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) More specifically: Substructure "type=tuple str=(332, {'prob': 1.0, 'action_mask': array([1, 1, 1, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=136" is not
(RolloutWorker pid=27558) Entire first structure:
(RolloutWorker pid=27558) (., {'prob': ., 'action_mask': .})
(RolloutWorker pid=27558) Entire second structure:
(RolloutWorker pid=27558) .
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) During handling of the above exception, another exception occurred:
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) ray::RolloutWorker.__init__() (pid=27558, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f52e0d3d810>)
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 495, in __init__
(RolloutWorker pid=27558) check_env(self.env)
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 84, in check_env
(RolloutWorker pid=27558) f"{actual_error}\n"
(RolloutWorker pid=27558) ValueError: Traceback (most recent call last):
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 280, in assert_same_structure
(RolloutWorker pid=27558) _tree.assert_same_structure(a, b, check_types)
(RolloutWorker pid=27558) ValueError: The two structures don't have the same nested structure.
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) First structure: type=tuple str=(332, {'prob': 1.0, 'action_mask': array([1, 1, 1, 0, 0, 0], dtype=int8)})
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) Second structure: type=int str=136
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) More specifically: Substructure "type=tuple str=(332, {'prob': 1.0, 'action_mask': array([1, 1, 1, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=136" is not
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) During handling of the above exception, another exception occurred:
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) ray::RolloutWorker.__init__() (pid=27558, ip=10.215.145.218, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f52e0d3d810>)
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 72, in check_env
(RolloutWorker pid=27558) check_gym_environments(env)
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/pre_checks/env.py", line 186, in check_gym_environments
(RolloutWorker pid=27558) reset_obs, sampled_observation
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/ray/rllib/utils/spaces/space_utils.py", line 359, in convert_element_to_space_type
(RolloutWorker pid=27558) return tree.map_structure(map_, element, sampled_element, check_types=False)
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 428, in map_structure
(RolloutWorker pid=27558) assert_same_structure(structures[0], other, check_types=check_types)
(RolloutWorker pid=27558) File "/home/ec2-user/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/tree/__init__.py", line 287, in assert_same_structure
(RolloutWorker pid=27558) % (e, str1, str2))
(RolloutWorker pid=27558) ValueError: The two structures don't have the same nested structure.
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) First structure: type=tuple str=(332, {'prob': 1.0, 'action_mask': array([1, 1, 1, 0, 0, 0], dtype=int8)})
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) Second structure: type=int str=136
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) More specifically: Substructure "type=tuple str=(332, {'prob': 1.0, 'action_mask': array([1, 1, 1, 0, 0, 0], dtype=int8)})" is a sequence, while substructure "type=int str=136" is not
(RolloutWorker pid=27558) Entire first structure:
(RolloutWorker pid=27558) (., {'prob': ., 'action_mask': .})
(RolloutWorker pid=27558) Entire second structure:
(RolloutWorker pid=27558) .
(RolloutWorker pid=27558)
(RolloutWorker pid=27558) The above error has been found in your environment! We've added a module for checking your custom environments. It may cause your experiment to fail if your environment is not set up correctly. You can disable this behavior by setting `disable_env_checking=True` in your environment config dictionary. You can run the environment checking module standalone by calling ray.rllib.utils.check_env([env]).
(raylet) [2022-09-22 17:08:24,254 E 4166 4192] (raylet) file_system_monitor.cc:105: /tmp/ray/session_2022-09-22_17-01-02_294657_1716 is over 95% full, available space: 4476575744; capacity: 116158152704. Object creation will fail if spilling is required.
(raylet) [2022-09-22 17:08:34,261 E 4166 4192] (raylet) file_system_monitor.cc:105: /tmp/ray/session_2022-09-22_17-01-02_294657_1716 is over 95% full, available space: 4476575744; capacity: 116158152704. Object creation will fail if spilling is required.