Cannot use wrapper around custom env: "invalid env specifier" ValueError

RLlib
1

Hi,

I’m trying to set up a custom environment that needs to be auto-reset and needs to have its duration of episodes limited. After a bit of fiddling, I’ve learned from ray’s feedback in error messages that I’m supposed to do this via wrappers instead of using e.g. “horizon”.
However, using wrappers does not work with a custom environment - I get the error given below - and I’m at a loss why this would be the case. Maybe I’m understanding something wrong in the use of wrappers.

Steps to reproduce:

  1. Take custom_env.py from https://github.com/ray-project/ray/blob/master/rllib/examples/custom_env.py
  2. Add an AutoResetWrapper around SimpleCorridor (line 106):
        .environment(AutoResetWrapper(SimpleCorridor), env_config={"corridor_length": 5})
  1. Executing this gives me:
...
  File "/home/k/k202064/.conda/envs/ray/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py", line 2360, in _get_env_id_and_creator
    raise ValueError(
ValueError: <AutoResetWrapper<class '__main__.SimpleCorridor'>> is an invalid env specifier. You can specify a custom env as either a class (e.g., YourEnvCls) or a registered env id (e.g., "your_env").
Trial status: 1 ERROR
...

Environment (conda list excerpt):
gymnasium 0.29.0 pypi_0 pypi
python 3.10.12 hd12c33a_0_cpython conda-forge
ray 2.6.2 pypi_0 pypi

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.