Checkpoint Backend creation

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

I was trying creating ray serve for a custom environment with loading the saved checkpoint
Code:
serve.create_backend(“ppo”, ServePPOModel,checkpoint_path)

I got the following error that I couldn’t understand
Error:
pid=5892) 2021-06-25 16:53:14,590 ERROR worker.py:409 – Exception raised in creation task: The actor died because of an error raised in its creation task, ray::eUUVZf:SERVE_CONTROLLER_ACTOR:ppo#BQRHCK:RayServeWrappedReplica.init (pid=5892, ip=192.168.1.6)
(pid=5892) File “python\ray_raylet.pyx”, line 501, in ray._raylet.execute_task
(pid=5892) File “python\ray_raylet.pyx”, line 448, in ray._raylet.execute_task.function_executor
(pid=5892) File “python\ray_raylet.pyx”, line 1664, in ray._raylet.CoreWorker.run_async_func_in_event_loop
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\concurrent\futures_base.py”, line 432, in result
(pid=5892) return self.__get_result()
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\concurrent\futures_base.py”, line 388, in __get_result
(pid=5892) raise self._exception
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\serve\backend_worker.py”, line 74, in init
(pid=5892) await sync_to_async(_callable.init)(*init_args)
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray_private\async_compat.py”, line 29, in wrapper
(pid=5892) return func(*args, **kwargs)
(pid=5892) File “”, line 5, in init
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\agents\trainer_template.py”, line 123, in init
(pid=5892) Trainer.init(self, config, env, logger_creator)
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\agents\trainer.py”, line 548, in init
(pid=5892) super().init(config, logger_creator)
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\tune\trainable.py”, line 98, in init
(pid=5892) self.setup(copy.deepcopy(self.config))
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\agents\trainer.py”, line 709, in setup
(pid=5892) self._init(self.config, self.env_creator)
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\agents\trainer_template.py”, line 150, in _init
(pid=5892) self.workers = self._make_workers(
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\agents\trainer.py”, line 791, in _make_workers
(pid=5892) return WorkerSet(
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\evaluation\worker_set.py”, line 92, in init
(pid=5892) self._local_worker = self._make_worker(
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\evaluation\worker_set.py”, line 367, in _make_worker
(pid=5892) worker = cls(
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\evaluation\rollout_worker.py”, line 534, in init
(pid=5892) self._build_policy_map(policy_dict, policy_config)
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\evaluation\rollout_worker.py”, line 1193, in build_policy_map
(pid=5892) policy_map[name] = cls(obs_space, act_space, merged_conf)
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\policy\tf_policy_template.py”, line 237, in init
(pid=5892) DynamicTFPolicy.init(
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\policy\dynamic_tf_policy.py”, line 158, in init
(pid=5892) dist_class, logit_dim = ModelCatalog.get_action_dist(
(pid=5892) File “C:\Users\Wavicledata\anaconda3\lib\site-packages\ray\rllib\models\catalog.py”, line 245, in get_action_dist
(pid=5892) assert np.all(action_space.high == high)
(pid=5892) AssertionError
(pid=13360) 2021-06-25 16:53:14,694 ERROR controller.py:121 – Exception updating backend state: Failed to look up actor with name ‘eUUVZf:SERVE_CONTROLLER_ACTOR:ppo#BQRHCK’. You are either trying to look up a named actor you didn’t create, the named actor died, or the actor hasn’t been created because named actor creation is asynchronous.

Could you help to know what this error is

2

It looks like there was an error initializing the class. I wonder, if you try to instantiate the class ServePPOModel normally (without using Ray Serve), do you get the same RLlib error?

Are you running this tutorial RLlib Tutorial — Ray v2.0.0.dev0 or have you made some changes to it? cc @simon-mo

3

Yeah, I have used that tutorial as the reference, it is working for another environment, it is not working for this particular environment.
Even when trying to instantiate the class ServePPOModel normally, I am getting the same assertion error
assert np.all(action_space.high == high_)
in the code file ray\rllib\models\catalog.py

4

Also, the code works only with ray 1.0.0, so saving the checkpoint I do with this version and deploying I do with ray 1.4.0.
If I change saving the checkpoint to ray 1.0.0, I get the following error
if dtype not in (dtypes.float16, dtypes.bfloat16, dtypes.float32,
dtypes.float64, dtypes.int32, dtypes.int64):
raise ValueError(“Invalid dtype %r” % dtype)

ValueError: Invalid dtype tf.int16