I have always got this OOM problem as I try to launch the “ray serve” to establish a ML backend service. May I know how to set the memory of “7.68G”? how to enhance this threshold according to the ray dashboard.
The ML code is just like this with higher num_replicas set and then the OOM will happen since more replicants occupy much more memory?
The specific error (RayOutOfMemoryError) you are seeing here is being removed in Ray 2.2 - you may want to give it a try
As for the error you are seeing, it seems the node does not have enough memory to serve 2 replicas - you may want to increase the memory of the node or add another node