Missing 'grad_gnorm' key in some `input_trees` after some training time

RLlib
22

@Halman I had a look into your repo. For example in this file here:

You can see that the reset function is defined like this:

def reset(self):
For the new gym API that RLLib uses in the newer versions the reset function has to look something like this:
def reset(self, *, seed=None, options=None):
However you can look into the linkt hat avnishn provided to the gymnasium docs above. There seemes to be a way to wrap “old” environments.

1 Like
23

(post deleted by author)

24

Thank you for your reply. I apologize for asking such a simple question as a beginner.

From your answer, it seems that there are two possible solutions.

  1. Adding seed and option to the reset function of my own environment

Regarding this, I tried adding seed and option to the reset function of BaseAviary.py and inputting the calculation, but I got the following error. This multi_agent_env.py implements the MultiAgentEnv class, and the BaseMultiagentAviary.py for the unique environment imports them. Also, this BaseMultiagentAviary is a subclass of BaseAviary. Therefore, I thought that the change of the reset function in the BaseAviary.py that I applied was applied, but it seems not to be working as desired.

~/rllib/env/multi_agent_env.py", line 896, in reset obs_and_infos = self.env.reset(seed=seed, options=options) TypeError: reset() got an unexpected keyword argument ‘seed’

Is there any possible cause? If you can think of anything, please let me know in the comments.

  1. Using the method of wrapping the old environment that avnishn commented on

Regarding this, I think that I can use the description in the link below. But I don’t understand how to use it because of my lack of knowledge. If you could teach me some specific usage (code), it would be helpful, even if it’s only a part of it.

## Step API Compatibility

If environments implement the (old) done step API, Gymnasium provides both functions (gymnasium.utils.step_api_compatibility.convert_to_terminated_truncated_step_api()) and wrappers (gymnasium.wrappers.StepAPICompatibility) that will convert an environment with the old step API (using done) to the new step API (using termination and truncation).

If you would prefer to start another topic instead of this one, please let me know.

25

@Halman Regarding 1: Your reset function should look like this:

def reset(self, *, seed=None, options=None):

Take a look at the examples in github under rlllib/env. Your error implies that you didn’t change the reset function correctly. If you want someone to check you should post a code snippet here. However changing the arguments for the reset and step functions is not enough since you would also have to change the returns since they are now obs, rew, terminated, truncated, info and not only obs, rew, done, info as before.
Regarding 2: I never used this option so i’m not able to provide info on that. However there seems to be a small example under this link: