I am writing to ask a question about the max_episode_steps attribute.
I have created a custom environment in RLlib. Whenever I start the trainin g of the agent I get bellow warnings. Could you help me how to solve this issue?
It should be noted that I have added max_steps in horizon or soft_horizon of the config dictionary, but it shows these warnings again:
Your env doesn’t have a .spec.max_episode_steps attribute. This is fine if you have set ‘horizon’ in your config dictionary, or soft_horizon. However, if you haven’t, ‘horizon’ will default to infinity, and your environment will not be reset.
– DeprecationWarning: _get_slice_indices has been deprecated. This will raise an error in the future!
so this warning means that RLlib doesn’t know when to reset your environment unless your environment has some done signal that it produces. If your environment has an attribute called .spec.max_episode_steps then it will not show that warning.
If you’ve set horizon config parameter then you should be fine and you can ignore the warning.
The second warning is a warning from tensorflow that they need to resolve on their end AFAICT.
Hi @avnishn and @AI360, how did you set the horizon parameter in config?
I also get that warning regarding the _max_steps, and I think this is one of the root causes why my tune trials never end
