How to configure resource used by proxyactor

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Hi, I’m wondering if there is way to reserve cpu resource for the proxyactor when deploying serve applications using kuberay. Currently the situation is, for example, I assigned 1 cpu for each serve replica, and a worker node have 2 cpus, which corresponds to 2 serve replicas. From the dashboard I noticed that the cpu util of each serve replica is around 50% and the proxyactor takes the other 100% cpu util. However, I want to increase the cpu util of serve replica to maximize the model’s performance.

If I assign more cpus to the pod, then additional replicas will be scheduled on this node. If I assign 2 cpus to each replica and 5 cpus to the pod, the model can’t fully utilize the 2 cpus, so either way is not a good solution.

2

Currently, there’s no way to configure num_cpus for the proxy actors. However, this should not affect your use case.

Ray uses num_cpus as logical resources for scheduling, not for physical isolation. Setting num_cpus only controls where your replicas run; not how many physical CPUs they consume. That’s left up to your application code. In other words, increasing your replica’s num_cpus won’t increase their CPU utilization. That would require changes in the model code.

If you want to change how your replicas are scheduled, one other option is to use custom resources. You can assign some number of custom resources to each node and replica.