Runung experiments with Tune using the A2C algorithm and the minigrid-Lava-Crissing environment

RLlib
1

Hello dear members of the RAY Team.

I am using Ray for conducting RL experiments using algorithms such as PPO, DQN, A2C, etc. I wanted to run these algorithms on the minigrid environment and more specifically on the Lava Crossing environment. In order to use minigrid I am using a custom env_creator function to initialize it there and use the wrapper ImgObsWrapper(gym. Make(‘MiniGrid-LavaCrossingS9N1-v0’)) to convert its observations to images. Then I am regestering it to RAY using the function register_env(). However when running experiments with tune. run, using A2C as the DRL algorithm of choice I am having the following Error:

ValueError: The two structures don’t have the same nested structure.

First structure: type=tuple str=(array([[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],

   [[0, 0, 0],
    [0, 0, 0],
    [0, 0, 0],
    [0, 0, 0],
    [0, 0, 0],
    [0, 0, 0],
    [0, 0, 0]],

   [[2, 5, 0],
    [2, 5, 0],
    [2, 5, 0],
    [2, 5, 0],
    [2, 5, 0],
    [2, 5, 0],
    [2, 5, 0]],

   [[1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0]],

   [[1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0]],

   [[1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0]],

   [[9, 0, 0],
    [9, 0, 0],
    [1, 0, 0],
    [9, 0, 0],
    [9, 0, 0],
    [9, 0, 0],
    [9, 0, 0]]], dtype=uint8), {})

Second structure: type=ndarray str=[[[ 57 29 88]
[ 5 195 95]
[107 171 126]
[193 203 48]
[ 62 105 215]
[ 57 192 233]
[ 83 3 114]]

[[166 176 4]
[102 200 237]
[135 114 221]
[116 255 104]
[ 29 69 68]
[223 35 33]
[217 154 68]]

[[151 105 49]
[ 3 156 10]
[212 43 181]
[ 53 228 19]
[165 28 20]
[ 24 38 171]
[160 105 163]]

[[ 27 27 20]
[230 56 228]
[ 56 159 65]
[159 91 53]
[ 69 207 70]
[ 61 33 48]
[162 181 199]]

[[229 74 251]
[ 28 211 95]
[247 111 49]
[ 47 59 68]
[112 193 50]
[176 217 236]
[141 202 24]]

[[ 99 254 25]
[ 50 120 107]
[ 52 184 100]
[ 60 186 169]
[ 18 218 251]
[125 54 13]
[150 153 142]]

[[220 115 1]
[247 239 99]
[ 26 171 249]
[165 45 44]
[ 33 205 121]
[ 22 132 133]
[124 19 200]]]

More specifically: Substructure "type=tuple str=(array([[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],

   [[0, 0, 0],
    [0, 0, 0],
    [0, 0, 0],
    [0, 0, 0],
    [0, 0, 0],
    [0, 0, 0],
    [0, 0, 0]],

   [[2, 5, 0],
    [2, 5, 0],
    [2, 5, 0],
    [2, 5, 0],
    [2, 5, 0],
    [2, 5, 0],
    [2, 5, 0]],

   [[1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0]],

   [[1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0]],

   [[1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0],
    [1, 0, 0]],

   [[9, 0, 0],
    [9, 0, 0],
    [1, 0, 0],
    [9, 0, 0],
    [9, 0, 0],
    [9, 0, 0],
    [9, 0, 0]]], dtype=uint8), {})" is a sequence, while substructure "type=ndarray str=[[[ 57  29  88]

[ 5 195 95]
[107 171 126]
[193 203 48]
[ 62 105 215]
[ 57 192 233]
[ 83 3 114]]

[[166 176 4]
[102 200 237]
[135 114 221]
[116 255 104]
[ 29 69 68]
[223 35 33]
[217 154 68]]

[[151 105 49]
[ 3 156 10]
[212 43 181]
[ 53 228 19]
[165 28 20]
[ 24 38 171]
[160 105 163]]

[[ 27 27 20]
[230 56 228]
[ 56 159 65]
[159 91 53]
[ 69 207 70]
[ 61 33 48]
[162 181 199]]

[[229 74 251]
[ 28 211 95]
[247 111 49]
[ 47 59 68]
[112 193 50]
[176 217 236]
[141 202 24]]

[[ 99 254 25]
[ 50 120 107]
[ 52 184 100]
[ 60 186 169]
[ 18 218 251]
[125 54 13]
[150 153 142]]

[[220 115 1]
[247 239 99]
[ 26 171 249]
[165 45 44]
[ 33 205 121]
[ 22 132 133]
[124 19 200]]]" is not

During handling of the above exception, another exception occurred:

Could you please offer me some help on how I can overcome this issue?
Thank you very much in advance !!!

Christos Peridis

2

Hi, the first substructure is a tuple (<some array>, {}) and the second one is only , without the empty dict and tuple around it. Where does the error occur?
Have a look at this file for a test of ours that wraps with ImgObsWrapper! :slight_smile: