Ray OOM issue when the ray serve is launched

1

I have always got this OOM problem as I try to launch the “ray serve” to establish a ML backend service. May I know how to set the memory of “7.68G”? how to enhance this threshold according to the ray dashboard.

image
image1920×1189 242 KB

2

the dashboard memory limit is 7.68GB from the ray dashboard. How to enhance this value?

image
image2868×1288 266 KB

3

The ML code is just like this with higher num_replicas set and then the OOM will happen since more replicants occupy much more memory?

image
image1836×1128 282 KB

4

The specific error (RayOutOfMemoryError) you are seeing here is being removed in Ray 2.2 - you may want to give it a try

5

As for the error you are seeing, it seems the node does not have enough memory to serve 2 replicas - you may want to increase the memory of the node or add another node