"RuntimeWarning: divide by zero encountered in divide" rllib

RLlib
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Hi everyone,

I’m still an RLlib newbie. I’m running a multi-agent environment using RLlib 2.0.0, and I get this warning:

/home/ramrachum/.venvs/ray_env/lib/python3.10/site-packages/ray/rllib/models/tf/misc.py:17: RuntimeWarning: divide by zero encountered in divide
  out *= std / np.sqrt(np.square(out).sum(axis=0, keepdims=True))

I understand that NumPy is dividing a matrix by another matrix, and since some of the values of the second matrix are zero, the result has inf in it.

Is this something that you encountered while using RLlib? I’m noticing that the performance of my agents in this experiment is abysmal, and I’m wondering whether it’s caused by this issue.

What can I do about this?

Thanks for your help,
Ram Rachum.

2

Hi @cool-RR,

That is a somewhat strange place to be getting an error because that is code that is normalizing the weights of a layer during initialization.

My best guess is that maybe you have a layer with no shape information? Totally a wild guess on my part.

Can you share your config and full error message?

A reproduction script would be even better if you have one.

Are you using the standard rllib models or do you have a custom model?

3

Thanks for the quick reply Manny. For some reason it isn’t happening now :slight_smile: And after some tweaks my experiment is working fine. If the problem comes back I’ll update this thread with more details.

Thanks for your help!