Shape of image for CNN

RLlib
1

I am using mixed input observations (i.e. parameters and images)

If I understand this page well:
https://docs.ray.io/en/latest/rllib/rllib-rlmodule.html

It means PPO and Impala are already capable of handling images via ComplexNet?
Not sure if the determination to use ComplexNet is done automatically.

I had been playing with PPO with images of shape (w, h, 3) without an issue (even though I cannot assert that it was threating it as a ComplexNet).

I now tried out Impala, and no matter what I tried I seem to get this error:

(RolloutWorker pid=17488)   File "c:\Users\user_x\anaconda3\envs\GTRay2.6.3_torch_tb\lib\site-packages\ray\rllib\utils\policy.py", line 142, in create_policy_for_framework
(RolloutWorker pid=17488)     return policy_class(observation_space, action_space, merged_config)
(RolloutWorker pid=17488)   File "c:\Users\user_x\anaconda3\envs\GTRay2.6.3_torch_tb\lib\site-packages\ray\rllib\algorithms\impala\impala_torch_policy.py", line 217, in __init__
(RolloutWorker pid=17488)     TorchPolicyV2.__init__(
(RolloutWorker pid=17488)   File "c:\Users\user_x\anaconda3\envs\GTRay2.6.3_torch_tb\lib\site-packages\ray\rllib\policy\torch_policy_v2.py", line 96, in __init__
(RolloutWorker pid=17488)     model, dist_class = self._init_model_and_dist_class()
...
(RolloutWorker pid=17488)     return self._conv_forward(input, self.weight, self.bias)
(RolloutWorker pid=17488)   File "c:\Users\user_x\anaconda3\envs\GTRay2.6.3_torch_tb\lib\site-packages\torch\nn\modules\conv.py", line 459, in _conv_forward
(RolloutWorker pid=17488)     return F.conv2d(input, weight, bias, self.stride,
(RolloutWorker pid=17488) RuntimeError: Calculated padded input size per channel: (1 x 11). Kernel size: (11 x 11). Kernel size can't be greater than actual input size

That to me suggests that it “detected” to use a CNN but also did not pic the axis correctly?
Or is it attempting to apply CNN to it all?
Is there a way for me to force ComplexNet?

2

@NDR008 Thanks for posting this! Do you have any reproducable example for us?

3

Hi, to be honest, I do not remember how I overcame this issue (but I did, and faced new issues)
With Impala:
I think the issue is that they were (3, w, h) instead of (w,h,3) which would still have been recognised as an image by the
len(component.shape == 3 but process the dimensions incorrection, after fixing that, I found a new problem.
If the observation space contained Discrete elements, Complex did not deal with them well. It would complain about the Tensor shape, like "Exepcted flattened obs sahpe of […, 242316], got torch.Size([32, 5305]).
After changing each Discrete element to a Box( , float) that error went away and Impala training could progress. So to some extent - IMPALA with ComplexNet works so long as the observation space does not contain discrete spaces… (I have not managed to achieve successful training) [I am not sure if this is expected, since the documentation says IMPALA can have discrete action spaces…]

With regards to PPO - the reason I never got any error is… it is not picking ComplexNet, it continue to pick the standard model PPOTorchRLModule…

So perhaps this thread should focus on “how to make PPOConfig recognise it should use ComplexNet or how to force it to?”
(I desperately wish to know how to make this PPO + ComplexNet work)

4

@NDR008 I am sorry to hear that you are still struggling there. Can you provide a reproducable example?