# After trainer.train(), there is a bug coming out **URL:** \ **Category:** RLlib\ **Created:** [July 7, 2022, 4:58pm UTC](https://discuss.ray.io/t/after-trainer-train-there-is-a-bug-coming-out/6754 "2022-07-07T16:58:43Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![qinmoelei](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/qinmoelei/32/2284_2.png) [@qinmoelei](https://discuss.ray.io/u/qinmoelei)\ **Post date:** [July 7, 2022, 4:58pm UTC](https://discuss.ray.io/t/after-trainer-train-there-is-a-bug-coming-out/6754/1 "2022-07-07T16:58:44Z")
File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/ray/util/tracing/tracing\_helper.py”, line 461, in \_resume\_span TypeError: ‘NoneType’ object is not callable I think there are some None exist in the checkpoint ---
**Author:** ![xwjiang2010](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/xwjiang2010/32/1476_2.png) [@xwjiang2010](https://discuss.ray.io/u/xwjiang2010)\ **Post date:** [July 7, 2022, 5:37pm UTC](https://discuss.ray.io/t/after-trainer-train-there-is-a-bug-coming-out/6754/2 "2022-07-07T17:37:53Z")
Could you provide a repro script and paste the full stacktrace? Thanks! ---
**Author:** ![qinmoelei](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/qinmoelei/32/2284_2.png) [@qinmoelei](https://discuss.ray.io/u/qinmoelei)\ **Post date:** [July 8, 2022, 5:43am UTC](https://discuss.ray.io/t/after-trainer-train-there-is-a-bug-coming-out/6754/4 "2022-07-08T05:43:46Z")
Thanks for helping me 🙂 ```auto ray.init(ignore_reinit_error=True) trainer = A2CTrainer register_env(env_name, lambda config: env_creator(self.env_name) (config)) model_config["env"] = env_name model_config["env_config"] =train_env_config sharpes = [] checkpoints = [] trainer = trainer(env=env_name, config=model_config) for i in range(2): trainer.train() valid_env_instance = env_creator(env_name)( valid_env_config) state = valid_env_instance.reset() done = False while not done: action = trainer.compute_single_action(state) state, reward, done, information = valid_env_instance.step( action) sharpes.append(information["sharpe_ratio"]) checkpoint = self.trainer.save() trainer.restore(checkpoint) checkpoints.append(checkpoint) loc = sharpes.index(max(sharpes)) trainer.restore(checkpoints[loc]) trainer.save(trained_model_dict) ray.shutdown() ``` # and here is the problem: 2022-07-08 13:41:42,119 INFO services.py:1470 – View the Ray dashboard at [http://127.0.0.1:8265](http://127.0.0.1:8265) 2022-07-08 13:41:44,078 INFO trainer.py:903 – Current log\_level is WARN. For more information, set ‘log\_level’: ‘INFO’ / ‘DEBUG’ or use the -v and -vv flags. (RolloutWorker pid=1692368) 2022-07-08 13:41:48,494 WARNING env.py:135 – Your env doesn’t have a .spec.max\_episode\_steps attribute. This is fine if you have set ‘horizon’ in your config dictionary, or `soft_horizon`. However, if you haven’t, ‘horizon’ will default to infinity, and your environment will not be reset. 2022-07-08 13:41:48,670 WARNING util.py:65 – Install gputil for GPU system monitoring. (RolloutWorker pid=1692368) ================================= (RolloutWorker pid=1692368) the profit margin is 64.1782399823528 % (RolloutWorker pid=1692368) the sharpe ratio is 1.4976463038091303 (RolloutWorker pid=1692368) the Volatility is 0.008453999055043442 (RolloutWorker pid=1692368) the max drawdown is 0.40600774513409027 (RolloutWorker pid=1692368) the Calmar Ratio is 1.3984378526411951 (RolloutWorker pid=1692368) the Sortino Ratio is 2.042888761004623 (RolloutWorker pid=1692368) ================================= # the profit margin is 7.941804289270293 % the sharpe ratio is 0.39396025059480905 the Volatility is 0.022334853455848163 the max drawdown is 0.35554847692589864 the Calmar Ratio is 0.3936379399741129 the Sortino Ratio is 0.4875073433055513 /home/sunshuo/ray\_results/A2CTrainer\_portfolio\_2022-07-08\_13-41-44rkwv\_e2u/checkpoint\_000001/checkpoint-1 \ False \ 0.9 \ 0.999 \ False \ 1e-08 \ None Traceback (most recent call last): File “/home/sunshuo/qml/TradeMaster\_reframe/agent/SOTA/SOTA.py”, line 215, in a.train\_with\_valid() File “/home/sunshuo/qml/TradeMaster\_reframe/agent/SOTA/SOTA.py”, line 187, in train\_with\_valid self.trainer.restore(checkpoint) File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/ray/tune/trainable.py”, line 583, in restore self.load\_checkpoint(checkpoint\_path) File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/ray/rllib/agents/trainer.py”, line 2070, in load\_checkpoint self. **setstate** (extra\_data) File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/ray/rllib/agents/trainer.py”, line 2693, in **setstate** self.workers.local\_worker().restore(state[“worker”]) File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 1542, in restore self.policy\_map[pid].set\_state(state) File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/ray/rllib/policy/torch\_policy.py”, line 766, in set\_state optim\_state\_dict = convert\_to\_torch\_tensor(s, device=self.device) File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/ray/rllib/utils/torch\_utils.py”, line 163, in convert\_to\_torch\_tensor return tree.map\_structure(mapping, x) File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/tree/ **init**.py”, line 430, in map\_structure [func(\*args) for args in zip(\*map(flatten, structures))]) File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/tree/ **init**.py”, line 430, in [func(\*args) for args in zip(\*map(flatten, structures))]) File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/ray/rllib/utils/torch\_utils.py”, line 157, in mapping tensor = torch.from\_numpy(np.asarray(item)) TypeError: can’t convert np.ndarray of type numpy.object\_. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint8, and bool. Exception ignored in: \ Traceback (most recent call last): File “/home/sunshuo/miniconda3/envs/TradeMaster/lib/python3.8/site-packages/ray/util/tracing/tracing\_helper.py”, line 461, in \_resume\_span TypeError: ‘NoneType’ object is not callable ---
**Author:** ![qinmoelei](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/qinmoelei/32/2284_2.png) [@qinmoelei](https://discuss.ray.io/u/qinmoelei)\ **Post date:** [July 8, 2022, 2:57pm UTC](https://discuss.ray.io/t/after-trainer-train-there-is-a-bug-coming-out/6754/5 "2022-07-08T14:57:59Z")
Thanks for the help. I look up in the issue of Ray and find a very similiar problem which will happen in python 3.8, I downgrad my python to 3.7 and now my code works perfectly. ---
**Author:** ![xwjiang2010](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/xwjiang2010/32/1476_2.png) [@xwjiang2010](https://discuss.ray.io/u/xwjiang2010)\ **Post date:** [July 11, 2022, 3:45am UTC](https://discuss.ray.io/t/after-trainer-train-there-is-a-bug-coming-out/6754/6 "2022-07-11T03:45:52Z")
Thanks! Just by looking at the stacktrace - seems very similar to this [issue](https://github.com/ray-project/ray/issues/22976). It may be worthwhile to document there what you have tried and discovered.