MARL mapping policy examples not working

Username1 · January 23, 2023, 8:06pm

How severe does this issue affect your experience of using Ray?*

High

I am searching for a MARL example where 5 agents share the same policy and the same action and observation space. However, both are made of tuples.

Like this:

  act_space = Tuple([Discrete(10), Discrete(2)]) 
  obs_space = Tuple([Discrete(10), Discrete(2)]) 
  self.action_space = {agent: act_space for agent in range(self.num_agents)}
  self.observation_space = {agent: obs_space for agent in range(self.num_agents)}

I am getting the following error:

raise ValueError(
ValueError: `observation_space` not provided in PolicySpec for default_policy and env does not have an observation space OR no spaces received from other workers' env(s) OR no `observation_space` specified in config!

Then, I’ve tried to run the official example scripts and got the same error:

when I try to run this example:

github.com

ray-project/ray/blob/master/rllib/examples/coin_game_env.py

##########
# Contribution by the Center on Long-Term Risk:
# https://github.com/longtermrisk/marltoolbox
##########
import argparse
import os

import ray
from ray import air, tune
from ray.rllib.algorithms.ppo import PPO
from ray.rllib.examples.env.coin_game_non_vectorized_env import CoinGame, AsymCoinGame

parser = argparse.ArgumentParser()
parser.add_argument("--tf", action="store_true")
parser.add_argument("--stop-iters", type=int, default=2000)


def main(debug, stop_iters=2000, tf=False, asymmetric_env=False):
    train_n_replicates = 1 if debug else 1
    seeds = list(range(train_n_replicates))

This file has been truncated. show original

I get the following error:

AttributeError: 'AsymCoinGame' object has no attribute 'observation_space'

I have also tried to run this script getting the same error:

github.com

ray-project/ray/blob/master/rllib/examples/multi_agent_different_spaces_for_agents.py

"""
Example showing how one can create a multi-agent env, in which the different agents
have different observation and action spaces.
These spaces do NOT necessarily have to be specified manually by the user. Instead,
RLlib will try to automatically infer them from the env provided spaces dicts
(agentID -> obs/act space) and the policy mapping fn (mapping agent IDs to policy IDs).

---
Run this example with defaults (using Tune):

  $ python multi_agent_different_spaces_for_agents.py
"""

import argparse
import gymnasium as gym
import os

import ray
from ray import air, tune
from ray.rllib.env.multi_agent_env import MultiAgentEnv

This file has been truncated. show original

after fixing for typos: Line 61- an extra {} and Line 72 - misspeling of ‘truncated’

arturn · March 8, 2023, 11:55pm

Hi @Username1 ,

I have opened a fixing PR: [RLlib] Make coingame a tests again by ArturNiederfahrenhorst · Pull Request #33156 · ray-project/ray · GitHub
You can see the “solution” there or run on master soon to retrieve the correct code.

Username1 · April 5, 2023, 5:25pm

Thank you very much !

Topic		Replies	Views
Example of multi-agent environment RLlib	4	513	September 23, 2022
New observation and action spaces in Ray 2.0 RLlib	3	247	October 27, 2022
ray.rllib.utils.error.UnsupportedSpaceException RLlib	0	233	July 14, 2023
Migration to Ray 2.2.0 RLlib	7	875	February 27, 2023
Rllib dont support tuple action spaces with unity mlagent environment RLlib	0	111	December 10, 2023

MARL mapping policy examples not working

Related Topics