Issues with MultiAgentEnv

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I keep receiving the following error message, which is odd, as my environment is an extension of MultiAgentEnv. Is there anything else I need to do in order for my environment to pass the check successfully?

ValueError: Have multiple policies <PolicyMap lru-caching-capacity=100 policy-IDs=[‘shared_policy’]>, but the env <OrderEnforcing<PassiveEnvChecker<RoutingEnv>>> is not a subclass of BaseEnv, MultiAgentEnv, ActorHandle, or ExternalMultiAgentEnv!

class RoutingEnv(MultiAgentEnv):
metadata = {
        "render_modes": ["human"]

    def __init__(self, render_mode="human", **kwargs):

    kwargs={"config": param_config}
env_name = "MyEnv-v0"
train_steps = 200000
learning_rate = 1e-3
save_dir = "saved_models"

def register(config):

    env = gym.make("MyEnv-v0")
    return env

# register the predefined scenario with RLlib
register_env("MultiEnv", register)

config = (
    .training(lr=0.001, _enable_learner_api=False)
    # RLlib needs +1 CPU than configured below (for the driver/trainer?)
        policies={"shared_policy": PolicySpec()},
        policy_mapping_fn=lambda agent_id, episode, worker, **kwargs: "shared_policy",

@TessMills it looks like you the code is not reproducable. Did you setup the RoutingEnv as a MultiAgentEnv with outputs for all agents?