QMIX problem with obs space. Tuple is define in environment, but I can not to retunr a Tuple in reset o step methods

RLlib Configure Algorithm, Training, Evaluation, Scaling
1

Hi, I have this problem blocking me.
I have a custom MultiAgentEnv and I would like to implement the QMix algorithm.

I define first the grouping agent as:

grouping = {
    "group_1": ['agent_dsp', 'agent_nw', 'agent_sw', 'agent_nwb'],
}

Then I defined the obs and act spaces as:

obs_space = Tuple(
    [
        Box(float("-inf"), float("inf"), (41,)),
        Box(float("-inf"), float("inf"), (41,)),
        Box(float("-inf"), float("inf"), (41,)),
        Box(float("-inf"), float("inf"), (41,))
    ]
)
act_space = Tuple(
    [
        Discrete(2),
        Discrete(2),
        Discrete(2),
        Discrete(2)
    ]
)

When I build the environment with QMixConfig() and .with_grouping_agent() function all perfermance well, but when I train() the environment it is returned a error:

ValueError: Observation ([array([  0.       ,  27.       ,  23.167332 ,   0.6333333, 216.33333  ,      
        68.16808  ,  22.470259 ,   0.       ,   0.       ,   0.       ,
         0.       ,   0.       ,   0.       ,   0.       ,  24.35     ,
        23.533333 ,  23.116667 ,  39.7      ,  22.4      ,  39.458332 ,
         0.       ,   2.0208333,   4.7      ,   0.5833333,  32.7      ,
        21.1      ,  52.666668 ,   0.       ,   1.9583334,   3.7      ,
         4.9583335,  37.1      ,  18.5      ,  42.083332 ,   0.       ,
         1.9416667,   5.7      ,   0.875    ,  -0.33     ,   0.       ],
      dtype=float32), array([  0.       ,  27.       ,  23.167332 ,   0.6333333, 216.33333  ,
        68.16808  ,  22.470259 ,   0.       ,   0.       ,   0.       ,
         0.       ,   0.       ,   0.       ,   0.       ,  24.35     ,
        23.533333 ,  23.116667 ,  39.7      ,  22.4      ,  39.458332 ,
         0.       ,   2.0208333,   4.7      ,   0.5833333,  32.7      ,
        21.1      ,  52.666668 ,   0.       ,   1.9583334,   3.7      ,
         4.9583335,  37.1      ,  18.5      ,  42.083332 ,   0.       ,
         1.9416667,   5.7      ,   0.875    ,  -0.33     ,   0.       ],
      dtype=float32), array([  0.       ,  27.       ,  23.167332 ,   0.6333333, 216.33333  ,
        68.16808  ,  22.470259 ,   0.       ,   0.       ,   0.       ,
         0.       ,   0.       ,   0.       ,   0.       ,  24.35     ,
        23.533333 ,  23.116667 ,  39.7      ,  22.4      ,  39.458332 ,
         0.       ,   2.0208333,   4.7      ,   0.5833333,  32.7      ,
        21.1      ,  52.666668 ,   0.       ,   1.9583334,   3.7      ,
         4.9583335,  37.1      ,  18.5      ,  42.083332 ,   0.       ,
         1.9416667,   5.7      ,   0.875    ,  -0.33     ,   0.       ],
      dtype=float32), array([  0.       ,  27.       ,  23.167332 ,   0.6333333, 216.33333  ,
        68.16808  ,  22.470259 ,   0.       ,   0.       ,   0.       ,
         0.       ,   0.       ,   0.       ,   0.       ,  24.35     ,
        23.533333 ,  23.116667 ,  39.7      ,  22.4      ,  39.458332 ,
         0.       ,   2.0208333,   4.7      ,   0.5833333,  32.7      ,
        21.1      ,  52.666668 ,   0.       ,   1.9583334,   3.7      ,
         4.9583335,  37.1      ,  18.5      ,  42.083332 ,   0.       ,
         1.9416667,   5.7      ,   0.875    ,  -0.33     ,   0.       ],
      dtype=float32)] dtype=None) outside given space (Tuple(Box(-inf, inf, (41,), float32), Box(-inf, inf, (41,), float32), Box(-inf, inf, (41,), float32), Box(-inf, inf, (41,), float32)))!

If in the environment I define the return observation with the form of:

self.next_obs_dict = (
                {'agent_dsp': self.next_obs},
                {'agent_nw': self.next_obs},
                {'agent_sw': self.next_obs},
                {'agent_nwb': self.next_obs}
            )

an AttributeError appears:

AttributeError: 'tuple' object has no attribute 'items'

There are any clue to fix this? Thanks so much in advance.

2

I think that the problem is in the with_agent_groups() method. After call it I expect that reset the environment start with the reset() method in GroupAgentsWrapper class, but instead the non grouping environment reset() method is called. I don’t know why…