My action space is a Box(-1, 1, (2,), np.int). I specifically want integers, and I don’t want to use a Discrete space because I don’t want to manage the interpretation between [0, n) and the potentially multi-dimensional Box space that my environment uses. When I try to train this in RLlib with tensorflow, I get the following error: ValueError: minval must be a scalar; got a tensor of shape [2]. After digging around a bit, I traced this to tensorflow random uniform; apparently, the minval can be broadcasted when the type is float, but not when it is integer. Too bad.
So my first thought is to change to a float type and round the actions in my step function to the nearest integer. However, I just wanted to post this on here to see if anyone else has had a similar issue and any corresponding advice.
Hey @rusu24edward , to my knowledge, we haven’t really tested any Box(int) action spaces. Could you perhaps do a PR that would fix this issue? Maybe just add an if block in the Random Exploration module that checks for the action space and then does a randint instead?
Hi @sven1977, sure I can do that. I think the underlying issue is with tensorflow. I don’t see why it cannot broadcast an integer, but I’ll ping them about this and see what comes up. Certainly, I can dig around a bit and see if I can bypass this for rllib when I have a few free cycles.