Error for custom Atari environment

RLlib
1

Hi,

I’m just getting started with RLlib and becoming more and more desperate using a modified Atari environment. I want to utilize the gym.ObserverationWrapper for domain adaptation, hence manipulating the observation.

It works like a charm for Pong and Breakout with observation type == ‘ram’, thus not the actual images.
As soon as I change the observation type to == ‘image’, I obtain the following error:

  File "/home/chris/.pyenv/versions/test_env/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 404, in __init__
    self.env: EnvType = wrap(self.env)
  File "/home/chris/.pyenv/versions/test_env/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 391, in wrap
    framestack=model_config.get("framestack"))
  File "/home/chris/.pyenv/versions/test_env/lib/python3.6/site-packages/ray/rllib/env/atari_wrappers.py", line 281, in wrap_deepmind
    if "NoFrameskip" in env.spec.id:
AttributeError: 'NoneType' object has no attribute 'id'

I have simplified the code to the following and I am still obtaining the same error:

import ray
from ray import tune
import ray.rllib.agents.impala as impala
from gym.envs.atari import AtariEnv


class AugmentedPong(AtariEnv):
    def __init__(self, config):
        super(AugmentedPong, self).__init__(game='pong', obs_type='image')


def env_creator(env_config):
    return AtariEnv(game='pong', obs_type='image', frameskip=1)


config = impala.DEFAULT_CONFIG.copy()

ray.init()
trainer = impala.ImpalaTrainer(config=config, env=AugmentedPong)

Same error for:

tune.registry.register_env('test_env', env_creator)
trainer = impala.ImpalaTrainer(config=config, env='test_env')

I’m using

  • Python 3.6.12
  • Ray 1.0.1.post1
  • Gym 0.17.3

What the heck am I missing here? :confused:
Thanks up front!

Cheers,
Chris

2

Apparently this issue is caused by the standard atari wrapper from deepmind.

It works with the rllib preprocessing:

config['preprocessor_pref'] = 'rllib'

Cheers,
Chris

3

Thanks for raising this issue @Chris!
This looks like bad error message handling on our side. Will provide a fix with a more meaningful error or just a warning + automatic change of config. …

1 Like
4

Here is the PR that fixes the issue:

1 Like