Setting observation space in Custom Env from config

RLlib
1
  • High: It blocks me to complete my task.

Hello Everyone,
I have a custom environment for which I want to set the observation space from a custom config dictionary.
My custom config looks like this

env :
    low : 0
    high : 1
    dim1 : 2
    dim2 : 5

Note: This is a hydra conf, I am correctly passing conf.env and for other operations the configs are set correctly.

And the CustomEnv.observation_space is set as

self.observation_space = Box(
            low= config.get("low"),
            high = config.get("high"),
            shape=(config.get("dim1")*config.get("dim2"),),
        )

However, when PolicySpec tries to set the obs and action spaces, the config dict is empty. As a result it throws this error :

ValueError: observation_space not provided in PolicySpec for agent0 and env does not have an observation space OR no spaces received from other workers’ env(s) OR no observation_space specified in config!

My question is, does RLLib support setting obs/action spaces from configs. If not, what would a workaround look like.

2

This is how my configs and env registration looks like :

    config = {
        "env": "custom_env",
        "env_config": OmegaConf.to_container(cfg.env),
        "num_workers": 2,
        "framework": "torch",
        "multiagent": {
            "policies": {
                "agent0": PolicySpec(
                    observation_space=env_creator(cfg.env).observation_space,
                    action_space=env_creator(cfg.env).action_space,
                    config=A2CConfig.overrides(framework_str="torch"),
                ),
                "agent1": PolicySpec(
                    observation_space=env_creator(cfg.env).observation_space,
                    action_space=env_creator(cfg.env).action_space,
                    config=A2CConfig.overrides(framework_str="torch"),
                ),
            },
            "policy_mapping_fn": lambda agent_id, *args, **kwargs: agent_id,
        },
    }
    env_creator(env_config=OmegaConf.to_container(cfg.env))
    register_env("custom_env", env_creator)

And the trainer looks like this :

    trainer = RLTrainer(
        run_config=RunConfig(
            stop={"training_iteration": 2},
        ),
        scaling_config=ScalingConfig(num_workers=1, use_gpu=False),
        algorithm="A2C",
        config=config,
    )
    result = trainer.fit()

Any insight would be valuable.
Thank you.

3

Managed to send the conf in this way.

spaces = {
        "observation_space" : Box(low = cfg.env.low, high=cfg.env.high, shape=(cfg.env.dim1*cfg.env.dim2,)),
        "action_space" : Box(low = cfg.env.low, high=cfg.env.high,shape=(1,)) 
    }

    config = {
        "env": "custom_env",
        "env_config": spaces | OmegaConf.to_container(cfg.env),