I’m a newbie for both Ray and RL. The troublemaker is a member of my observation space(a dict). It’s a long MultiDiscrete space(length=52). The error occurred when Ray tried to convert that space to a tensor(I believe this is not a backend(torch) bug because tensorflow also meets the same problem).
I dived into the error point – part of the package tree. It seems that the error was raised by numpy.ndarray instantiation with raw data (ndarray(data)), but the correct way should be numpy.array(data). The array solution also has another defect that the nested data type will lose. In my situation, data is a list of tensors, but numpy.array(data) returns a common ndarray.
Environment:
Archlinux x86_64, Install all package with mamba(conda-forge) package:
pytorch 2.1.0
Ray 2.7.1
tree 0.1.8
StackTrace:
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/ray/rllib/utils/torch_utils.py”, line 262, in convert_to_torch_tensor
return tree.map_structure(mapping, x)
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py”, line 434, in map_structure
return unflatten_as(structures[0],
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py”, line 380, in unflatten_as
_, packed = _packed_nest_with_indices(structure, flat_sequence, 0)
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py”, line 314, in _packed_nest_with_indices
for s in _yield_value(structure):
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py”, line 132, in _yield_value
for _, v in _yield_sorted_items(iterable):
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py”, line 158, in _yield_sorted_items
yield key, iterable[key]
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/ray/rllib/policy/sample_batch.py”, line 932, in getitem
self.intercepted_values[key] = self.get_interceptor(value)
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/ray/rllib/utils/torch_utils.py”, line 262, in convert_to_torch_tensor
return tree.map_structure(mapping, x)
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py”, line 434, in map_structure
return unflatten_as(structures[0],
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py”, line 380, in unflatten_as
_, packed = _packed_nest_with_indices(structure, flat_sequence, 0)
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/init.py”, line 317, in _packed_nest_with_indices
packed.append(_sequence_like(s, child))
File “/home/shilka/miniforge3/envs/ray/lib/python3.9/site-packages/tree/sequence.py”, line 111, in _sequence_like
return type(instance)(args)
ValueError: maximum supported dimension for an ndarray is 32, found 52