Issues with MultiAgentEnv

RLlib
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I keep receiving the following error message, which is odd, as my environment is an extension of MultiAgentEnv. Is there anything else I need to do in order for my environment to pass the check successfully?

ValueError: Have multiple policies <PolicyMap lru-caching-capacity=100 policy-IDs=[‘shared_policy’]>, but the env <OrderEnforcing<PassiveEnvChecker<RoutingEnv>>> is not a subclass of BaseEnv, MultiAgentEnv, ActorHandle, or ExternalMultiAgentEnv!

class RoutingEnv(MultiAgentEnv):
metadata = {
        "render_modes": ["human"]
    }

    def __init__(self, render_mode="human", **kwargs):
        super().__init__()


gym.envs.register(
    id="MyEnv-v0",
    entry_point='routing_rl.envs:RoutingEnv',
    kwargs={"config": param_config}
)
env_name = "MyEnv-v0"
train_steps = 200000
learning_rate = 1e-3
save_dir = "saved_models"

def register(config):

    env = gym.make("MyEnv-v0")
    return env

# register the predefined scenario with RLlib
register_env("MultiEnv", register)


config = (
    PPOConfig()
    .training(lr=0.001, _enable_learner_api=False)
    .environment(env="MultiEnv")
    .environment(disable_env_checking=True)
    # RLlib needs +1 CPU than configured below (for the driver/trainer?)
    .resources(num_cpus_per_worker=1)
    .rollouts(num_rollout_workers=0)
    .multi_agent(
        policies={"shared_policy": PolicySpec()},
        policy_mapping_fn=lambda agent_id, episode, worker, **kwargs: "shared_policy",
    )
2

@TessMills it looks like you the code is not reproducable. Did you setup the RoutingEnv as a MultiAgentEnv with outputs for all agents?