How severe does this issue affect your experience of using Ray?
- High: It blocks me to complete my task.
Hi, I am new to ray, and I have a physcial engine which are implemented by MPh (a python interface of COMSOL), and I need to run it on muti-node cluster. But it raise error as follow
Traceback (most recent call last):
File “python/ray/_raylet.pyx”, line 905, in ray._raylet.prepare_args_internal
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/ray/_private/serialization.py”, line 502, in serialize
return self._serialize_to_msgpack(value)
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/ray/_private/serialization.py”, line 480, in _serialize_to_msgpack
pickle5_serialized_object = self._serialize_to_pickle5(
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/ray/_private/serialization.py”, line 433, in _serialize_to_pickle5
raise e
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/ray/_private/serialization.py”, line 428, in _serialize_to_pickle5
inband = pickle.dumps(
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/ray/cloudpickle/cloudpickle.py”, line 1479, in dumps
cp.dump(obj)
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/ray/cloudpickle/cloudpickle.py”, line 1245, in dump
return super().dump(obj)
TypeError: cannot pickle ‘_jpype._JField’ objectThe above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “/public/home/acwq22s5az/my_job/rl-project-ray/main.py”, line 379, in
main(get_args())
File “/public/home/acwq22s5az/my_job/rl-project-ray/main.py”, line 221, in main
train_env = XVectorEnv(train_env_, wait_num=args.wait_num if args.asyn else None)
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/tianshou/env/venvs.py”, line 508, in init
super().init(env_fns, RayEnvWorker, **kwargs)
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/tianshou/env/venvs.py”, line 153, in init
self.workers = [worker_fn(_patch_env_generator(fn)) for fn in env_fns]
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/tianshou/env/venvs.py”, line 153, in
self.workers = [worker_fn(_patch_env_generator(fn)) for fn in env_fns]
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/tianshou/env/worker/ray.py”, line 28, in init
self.env = ray.remote(_SetAttrWrapper).options( # type: ignore
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/ray/actor.py”, line 830, in remote
return actor_cls._remote(args=args, kwargs=kwargs, **updated_options)
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/ray/_private/auto_init_hook.py”, line 21, in auto_init_wrapper
return fn(*args, **kwargs)
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/ray/util/tracing/tracing_helper.py”, line 388, in _invocation_actor_class_remote_span
return method(self, args, kwargs, *_args, **_kwargs)
File “/public/home/acwq22s5az/softwares/anaconda3/lib/python3.9/site-packages/ray/actor.py”, line 1161, in _remote
actor_id = worker.core_worker.create_actor(
File “python/ray/_raylet.pyx”, line 4062, in ray._raylet.CoreWorker.create_actor
File “python/ray/_raylet.pyx”, line 4067, in ray._raylet.CoreWorker.create_actor
File “python/ray/_raylet.pyx”, line 867, in ray._raylet.prepare_args_and_increment_put_refs
File “python/ray/_raylet.pyx”, line 858, in ray._raylet.prepare_args_and_increment_put_refs
File “python/ray/_raylet.pyx”, line 914, in ray._raylet.prepare_args_internal
TypeError: Could not serialize the argument <environs.XEnviron object at 0x2b80a7f55670> for a task or actor tianshou.env.worker.ray._SetAttrWrapper.init:
===============================================
Checking Serializability of
===============================================
!!! FAIL serialization: cannot pickle ‘_jpype._JField’ object
Serializing ‘_is_protocol’ False…
Serializing ‘_np_random’ Generator(PCG64)…
Serializing ‘act2operate’ <bound method XEnviron.act2operate of <environs.XEnviron object at 0x2b80a7f55670>>…
!!! FAIL serialization: cannot pickle ‘_jpype._JField’ object
Serializing ‘func’ <function XEnviron.act2operate at 0x2b8037441dc0>…
WARNING: Did not find non-serializable object in <bound method XEnviron.act2operate of <environs.XEnviron object at 0x2b80a7f55670>>. This may be an oversight.
===============================================
Variable:FailTuple(act2operate [obj=<bound method XEnviron.act2operate of <environs.XEnviron object at 0x2b80a7f55670>>, parent=])
was found to be non-serializable. There may be multiple other undetected variables that were non-serializable.
Consider either removing the instantiation/imports of these variables or moving the instantiation into the scope of the function/class.
===============================================
Check Serialization — Ray 3.0.0.dev0 for more information.
If you have any suggestions on how to improve this error message, please reach out to the Ray developers on Issues · ray-project/ray · GitHub
===============================================