Is ray setting memory resource?

I got this warning message. I only set the CPU requirement, and it seems to consider memory resources. So is there a default memory resource setting for one actor?

2021-05-22 11:20:03,844	WARNING worker.py:1115 -- The actor or task with ID d4cbda1bcbc948bdffffffffffffffffffffffff01000000 cannot be scheduled right now. It requires {CPU: 1.000000} for placement, but this node only has remaining {387.000000/460.000000 CPU, 797.301085 GiB/797.301085 GiB memory, 186.264515 GiB/186.264515 GiB object_store_memory, 387.000000/460.000000 scheduler-0, 387.000000/460.000000 scheduler, 1.000000/1.000000 node:10.132.167.88}
. In total there are 1 pending tasks and 0 pending actors on this node. This is likely due to all cluster resources being claimed by actors. To resolve the issue, consider creating fewer actors or increase the resources available to this Ray cluster. You can ignore this message if this Ray cluster is expected to auto-scale.

This issue doesn’t exist when we were using 1.0.0. It appears when we update ray to 1.3.0.

It doesn’t consider memory resources actually. Usually after some time, your task or actor should be scheduled after you see this message, Are your tasks/actors hanging?

(I think we updated the condition to print this log message, and that could be a reason why you are seeing it).

Okay, thanks for clarifying this issue. I was just worried about there was a hidden bug.

1 Like