Different observation space in MultiAgentEnv

RLlib
1

Hi, I am new to rllib and recently working on custom multi-agent env, not sure how MultiAgentEnv deal with observation space mapping. I found most examples expect all agents have equal observation space which just need ob_space = gym.space.
If I set observation space as a Dict like below will lead to ValueError because observation is just from a single agent while observation space is a Dict.
I am dealing with a homogeneous problem which this will not be a problem, just wondering if observation space mapping is available through MultiAgentEnv with independent learner setting.

self.observation_space = spaces.Dict({
            'agent_0': spaces.Tuple(
                [
                 spaces.Box(-np.inf, np.inf, shape=(9,), dtype=np.int64),
                 ]),   
            'agent_1': spaces.Tuple(
                [
                 spaces.Box(-np.inf, np.inf, shape=(9,), dtype=np.int64),
                 ]),
            'agent_2': spaces.Tuple(
                [
                 spaces.Box(-np.inf, np.inf, shape=(9,), dtype=np.int64),
                 ]),
        })
2

Hi @kyle-playground, from my experience (if not correct, please fix me :P), actually you don’t need to define the observation/action space in the environment unless you want to check the obs/action is in the action space.

In the multi-agent RL, you need to assign policy/learner for agents, where you need to define the observation and action space for each policy.

In all, the observation and action space is actually used for the policy.

1 Like
3

@UserRLlib Thanks for your reply! Now it makes sense why multiagent config requires to define observation space and action space in policy config dict.