Handling spaces.Dict in Multi-Agent Environment without .shape Attribute Error

RLlib
1

Hi everyone,

I am developing a multi-agent environment using RLlib, where each agent’s observation space is defined using spaces.Dict . My environment setup works well until integration with RLlib’s training loop, where I encounter a recurring issue related to the handling of these complex spaces.

I define my agents’ observation spaces using spaces.Dict which includes various subspaces such as spaces.Box and spaces.MultiBinary . I had several definitions since I was trying to find a solution to the spaces errors. Here is the definition of the observation and action spaces:

        self._observation_spaces = spaces.Dict(
            observer_satellites = spaces.Box(low=-np.inf, high=np.inf, shape=(self.num_observers, len(self.orbital_params_order))),
            target_satellites = spaces.Box(low=-np.inf, high=np.inf, shape=(self.num_targets, len(self.orbital_params_order_targets))),
            availability = spaces.MultiBinary(1),
            battery = spaces.Box(low=0, high=1, shape=(self.num_observers, 1)),
            storage =spaces.Box(low=0, high=1, shape=(self.num_observers, 1)),
            observation_status = spaces.Box(low=0, high=3, shape=(self.num_targets, )),
            pointing_accuracy = spaces.Box(low=-np.inf, high=np.inf, shape=(self.num_observers, self.num_targets)),
            communication_status = spaces.Box(low=0, high=1, shape=(self.num_observers, ), dtype=np.int8),
            communication_ability = spaces.MultiBinary(self.num_observers)
            )
        

        self._action_spaces = spaces.Discrete(2 + self.num_targets)
        self.infos = {agent: {} for agent in self.possible_agents}

    @property
    def observation_spaces(self):
        return {agent: self.observation_space(agent) for agent in self.possible_agents}

    @property
    def action_spaces(self):
        return {agent: self.action_space(agent) for agent in self.possible_agents}
    
    @functools.lru_cache(maxsize=None)
    def observation_space(self, agent):
        print(f"Returning observation space for {agent}: {self._observation_spaces}")
        return self._observation_spaces

    @functools.lru_cache(maxsize=None)
    def action_space(self, agent):
        print(f"Returning action space for {agent}: {self._action_spaces}")
        return self._action_spaces

During the training setup, when RLlib tries to handle these observation spaces, I encounter an error indicating that a 'dict' object has no attribute 'shape' . This seems to stem from somewhere within RLlib expecting a direct .shape attribute on the observation space, which does not exist for spaces.Dict .

The error typically points to the internals of policy setup or preprocessing layers where a .shape attribute is queried on the observation space:

File "/path/to/rllib/models/preprocessors.py", line 447, in _legacy_patch_shapes
    return space.shape
AttributeError: 'dict' object has no attribute 'shape'

Any guidance or workarounds to properly integrate complex observation spaces with RLlib would be highly appreciated!

Thank you!

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.