MARL mapping policy examples not working

Username1 · January 23, 2023, 8:06pm

How severe does this issue affect your experience of using Ray?*

High

I am searching for a MARL example where 5 agents share the same policy and the same action and observation space. However, both are made of tuples.

Like this:

  act_space = Tuple([Discrete(10), Discrete(2)]) 
  obs_space = Tuple([Discrete(10), Discrete(2)]) 
  self.action_space = {agent: act_space for agent in range(self.num_agents)}
  self.observation_space = {agent: obs_space for agent in range(self.num_agents)}

I am getting the following error:

raise ValueError(
ValueError: `observation_space` not provided in PolicySpec for default_policy and env does not have an observation space OR no spaces received from other workers' env(s) OR no `observation_space` specified in config!

Then, I’ve tried to run the official example scripts and got the same error:

when I try to run this example:

github.com

ray-project/ray/blob/master/rllib/examples/coin_game_env.py

##########
# Contribution by the Center on Long-Term Risk:
# https://github.com/longtermrisk/marltoolbox
##########
import argparse
import os

import ray
from ray import air, tune
from ray.rllib.algorithms.ppo import PPO
from ray.rllib.examples.env.coin_game_non_vectorized_env import CoinGame, AsymCoinGame

parser = argparse.ArgumentParser()
parser.add_argument("--tf", action="store_true")
parser.add_argument("--stop-iters", type=int, default=2000)


def main(debug, stop_iters=2000, tf=False, asymmetric_env=False):
    train_n_replicates = 1 if debug else 1
    seeds = list(range(train_n_replicates))

This file has been truncated. show original

I get the following error:

AttributeError: 'AsymCoinGame' object has no attribute 'observation_space'

I have also tried to run this script getting the same error:

github.com

ray-project/ray/blob/master/rllib/examples/multi_agent_different_spaces_for_agents.py

"""
Example showing how one can create a multi-agent env, in which the different agents
have different observation and action spaces.
These spaces do NOT necessarily have to be specified manually by the user. Instead,
RLlib will try to automatically infer them from the env provided spaces dicts
(agentID -> obs/act space) and the policy mapping fn (mapping agent IDs to policy IDs).

---
Run this example with defaults (using Tune):

  $ python multi_agent_different_spaces_for_agents.py
"""

import argparse
import gymnasium as gym
import os

import ray
from ray import air, tune
from ray.rllib.env.multi_agent_env import MultiAgentEnv

This file has been truncated. show original

after fixing for typos: Line 61- an extra {} and Line 72 - misspeling of ‘truncated’

arturn · March 8, 2023, 11:55pm

Hi @Username1 ,

I have opened a fixing PR: [RLlib] Make coingame a tests again by ArturNiederfahrenhorst · Pull Request #33156 · ray-project/ray · GitHub
You can see the “solution” there or run on master soon to retrieve the correct code.

Username1 · April 5, 2023, 5:25pm

Thank you very much !

Topic		Replies	Views
AlgorithmConfig for multi-agent env with different observation/action spaces RLlib	0	250	July 13, 2023
Example of multi-agent environment RLlib	4	862	September 23, 2022
New observation and action spaces in Ray 2.0 RLlib	3	308	October 27, 2022
ray.rllib.utils.error.UnsupportedSpaceException RLlib	0	299	July 14, 2023
RLlib and gym.space RLlib	4	636	November 14, 2021

MARL mapping policy examples not working

Related topics