Unable to register openspiel environments

RLlib
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hi, I’m relatively new to RLLib, and am trying to use the open spiel games in here.

import ray
from ray import tune
from ray.rllib.env.wrappers.pettingzoo_env import PettingZooEnv
from ray.rllib.algorithms.ppo import PPOConfig
from ray.tune.registry import register_env
import pyspiel  # OpenSpiel environments
from shimmy import OpenSpielCompatibilityV0
from open_spiel.python import rl_environment

def env_creator(env_config):
    return PettingZooEnv(OpenSpielCompatibilityV0(game_name="battleship"))

register_env("battleship", env_creator)

ppo_config = (
    PPOConfig()
    .environment(
        env="battleship",
        env_config={"game_name": "battleship"}
    )
    .framework("torch")
    .rollouts(num_rollout_workers=2) 
    .training(
        use_critic=True,
        use_gae=True,
        lambda_=0.95,
        vf_loss_coeff=0.5,
        entropy_coeff=0.01,
        clip_param=0.2,
        vf_clip_param=10.0,
        grad_clip=0.5
    )
    .multi_agent(
        policies={"agent_1": (None, None, None, {}), "agent_2": (None, None, None, {})},
        policy_mapping_fn=lambda agent_id, **kwargs: "agent_1" if agent_id % 2 == 0 else "agent_2"
    )
)

ray.shutdown()
ray.init(ignore_reinit_error=True)

ppo_config.environment(env=game, env_config={"game_name": game})
tune.run(
        "PPO",
        config=ppo_config.to_dict(),
        stop={"training_iteration": 100},
        checkpoint_freq=10
)

ray.shutdown()

However, in this case, it seems as if the environment does not get registered.

Getting the following error:

(RolloutWorker pid=436255) Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=436255, ip=192.168.0.18, actor_id=a72790a297c690e479ecd64301000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7a67b7a060d0>)
(RolloutWorker pid=436255)   File "/home/***/anaconda3/envs/***/lib/python3.9/site-packages/gymnasium/envs/registration.py", line 687, in make
(RolloutWorker pid=436255)     env_spec = _find_spec(id)
(RolloutWorker pid=436255)   File "/home/***/anaconda3/envs/***/lib/python3.9/site-packages/gymnasium/envs/registration.py", line 531, in _find_spec
(RolloutWorker pid=436255)     _check_version_exists(ns, name, version)
(RolloutWorker pid=436255)   File "/home/***/anaconda3/envs/***/lib/python3.9/site-packages/gymnasium/envs/registration.py", line 397, in _check_version_exists
(RolloutWorker pid=436255)     _check_name_exists(ns, name)
(RolloutWorker pid=436255)   File "/home/***/anaconda3/envs/***/lib/python3.9/site-packages/gymnasium/envs/registration.py", line 374, in _check_name_exists
(RolloutWorker pid=436255)     raise error.NameNotFound(
(RolloutWorker pid=436255) gymnasium.error.NameNotFound: Environment `battleship` doesn't exist.
(RolloutWorker pid=436255) 

During handling of the above exception, another exception occurred:

ray::RolloutWorker.__init__() (pid=436255, ip=192.168.0.18, actor_id=a72790a297c690e479ecd64301000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7a67b7a060d0>)
  File "/home/***/anaconda3/envs/***/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 397, in __init__
    self.env = env_creator(copy.deepcopy(self.env_context))
  File "/home/***/anaconda3/envs/***/lib/python3.9/site-packages/ray/rllib/env/utils/__init__.py", line 115, in _gym_env_creator
    raise EnvError(ERR_MSG_INVALID_ENV_DESCRIPTOR.format(env_descriptor))
ray.rllib.utils.error.EnvError: The env string you provided ('battleship') is:
a) Not a supported or -installed environment.
b) Not a tune-registered environment creator.
c) Not a valid env class string.

Try one of the following:
a) For Atari support: `pip install gym[atari] autorom[accept-rom-license]`.
   For PyBullet support: `pip install pybullet`.
b) To register your custom env, do `from ray import tune;
   tune.register('[name]', lambda cfg: [return env obj from here using cfg])`.
   Then in your config, do `config['env'] = [name]`.
c) Make sure you provide a fully qualified classpath, e.g.:
   `ray.rllib.examples.envs.classes.repeat_after_me_env.RepeatAfterMeEnv`

Also, when I run the code independent of RLLib, for the environment, it seems to run, i.e.:

env = PettingZooEnv(OpenSpielCompatibilityV0(game_name="battleship"))
env.reset()

This runs.