The issues is that the parameter name for the create_env function is env_config. You are using the name gymconfig now perhaps there is a variable with that name in a scope accessible from env_creator in which case it will work. But, you would be using a free variable that may or may not exist. If you want to use gymconfig then change the function to be: def env_creator(gymconfig):
1 Like
if I have “done” setup in my custum openai gym environment , do i still need to have “horizon = 10” in the config ?
when i remove it , my rewards in the trials are nan .