I am fairly new to RLlib and trying to set up a multi-agent RL script. I have successfully been able to build my algorithm. However, when training I am receiving a stranger GIL error that I am finding very little information about online. I am not sure if this is a problem with RLlib, or how I designed my custom environment. If anyone has helpful advice it would be greatly appreciated. Here is the output of running the code, I have highlighted the error I am getting in bold:
2024-07-25 14:58:23,901 INFO worker.py:1779 – Started a local Ray instance. View the dashboard at 127.0.0.1:8265
2024-07-25 14:58:24,857 INFO packaging.py:530 – Creating a file package for local directory ‘C:\Users\Cameron Mehlman\Documents\magpie_rl’.
2024-07-25 14:58:25,122 INFO packaging.py:358 – Pushing file package ‘gcs://_ray_pkg_bd73014ad01c10f7.zip’ (1.59MiB) to Ray cluster…
2024-07-25 14:58:25,122 INFO packaging.py:371 – Successfully pushed file package ‘gcs://_ray_pkg_bd73014ad01c10f7.zip’.
[2024-07-25 14:58:27,269][learning.train_ray][INFO] - Save directory found …
current directory: C:\Users\Cameron Mehlman\Documents\magpie_rl/logs/2d_test/2024-07-25
[2024-07-25 14:58:27,285][space_sim.sim][INFO] - Initializing simulation…
[2024-07-25 14:58:27,285][space_sim.sim][INFO] - Simulation initialized
[2024-07-25 14:58:27,285][learning.make_env][INFO] - Setting up evasion environment
[2024-07-25 14:58:27,292][learning.make_env][INFO] - Initializing adversarial agents
[2024-07-25 14:58:27,297][envs.evade_pursuit_env][INFO] - Initializing env …
c:\Users\Cameron Mehlman\Documents\magpie_rl.venv\lib\site-packages\ray\rllib\algorithms\algorithm.py:516: RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases. You could suppress this warning by setting env variable PYTHONWARNINGS=“ignore::DeprecationWarning”
UnifiedLogger will be removed in Ray 2.7.
return UnifiedLogger(config, logdir, loggers=None)
c:\Users\Cameron Mehlman\Documents\magpie_rl.venv\lib\site-packages\ray\tune\logger\unified.py:53: RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases. You could suppress this warning by setting env variable PYTHONWARNINGS=“ignore::DeprecationWarning”
The JsonLogger interface is deprecated in favor of the ray.tune.json.JsonLoggerCallbackinterface and will be removed in Ray 2.7. self._loggers.append(cls(self.config, self.logdir, self.trial)) c:\Users\Cameron Mehlman\Documents\magpie_rl\.venv\lib\site-packages\ray\tune\logger\unified.py:53: RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases. You could suppress this warning by setting env variable PYTHONWARNINGS="ignore::DeprecationWarning" TheCSVLogger interface is deprecated in favor of the ray.tune.csv.CSVLoggerCallback interface and will be removed in Ray 2.7.
self._loggers.append(cls(self.config, self.logdir, self.trial))
c:\Users\Cameron Mehlman\Documents\magpie_rl.venv\lib\site-packages\ray\tune\logger\unified.py:53: RayDeprecationWarning: This API is deprecated and may be removed in future Ray releases. You could suppress this warning by setting env variable PYTHONWARNINGS=“ignore::DeprecationWarning”
The TBXLogger interface is deprecated in favor of the ray.tune.tensorboardx.TBXLoggerCallback` interface and will be removed in Ray 2.7.
self._loggers.append(cls(self.config, self.logdir, self.trial))
2024-07-25 14:58:41,074 INFO trainable.py:161 – Trainable.setup took 13.722 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.
2024-07-25 14:58:41,074 WARNING util.py:61 – Install gputil for GPU system monitoring.
(RolloutWorker pid=25404) Warning: cannot generate point cloud data from simplified mesh, for point cloud augmentation use pyvista. Using corner points
ALL DONE
(RolloutWorker pid=25404) Fatal Python error: PyThreadState_Get: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
(RolloutWorker pid=25404) Python runtime state: initialized
(RolloutWorker pid=25404)
(RolloutWorker pid=25404) Thread 0x00001e68 (most recent call first):
(RolloutWorker pid=25404) File “c:\Users\Cameron Mehlman\Documents\magpie_rl.venv\lib\site-packages\ray_private\worker.py”, line 887 in main_loop
(RolloutWorker pid=25404) File “c:\Users\Cameron Mehlman\Documents\magpie_rl.venv\lib\site-packages\ray_private\workers\default_worker.py”, line 289 in
(RolloutWorker pid=27440)
(RolloutWorker pid=27440) Warning: cannot generate point cloud data from simplified mesh, for point cloud augmentation use pyvista. Using corner points
(RolloutWorker pid=27440) Fatal Python error: PyThreadState_Get: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
(RolloutWorker pid=27440) Python runtime state: initialized
(RolloutWorker pid=27440) Thread 0x00006a1c (most recent call first):
(RolloutWorker pid=27440) File “c:\Users\Cameron Mehlman\Documents\magpie_rl.venv\lib\site-packages\ray_private\worker.py”, line 887 in main_loop
(RolloutWorker pid=27440) File “c:\Users\Cameron Mehlman\Documents\magpie_rl.venv\lib\site-packages\ray_private\workers\default_worker.py”, line 289 in
my algorithm attempts to train 2 SAC policies competing in a zero-sum game, if any more information is needed please let me know. Thanks!