Resources used by HTTPProxyActor

I’ve been testing serve to create a few endpoints and i noticed that as soon as I run serve.start() i see that each worker starts up an instance of HTTPProxyActor.

  1. What is this Actor’s purpose?
  2. It has 1 CPU resource assigned. This seems like a lot, can it be configured to a different value?

Thank you.

Hi the actor is used as a proxy that listens to http port. In new version of Ray (ray 1.2.0), we only start one of them in head node. (the new version should be coming out this week!)

Will it be possible to change the resources from the HTTPProxyActor in the new release?

No not yet. Can I ask why do you need to change it? If you want to put more models into the system you can start ray with more workers by running ray.init(num_cpus=16*2) or ray start --num-cpus= 32 (assume your system has 16)

Well, actually it has nothing to do with serving models for now. I have a small graphql endpoint made available through serve. Ray is running on kubernetes and the cluster node resources are very limited, 2 cpus/8gb ram, but i am able to spawn as many of them as needed. This means that the ray head lives on one of these nodes and its CPU count is 2. Just deploying the endpoint will take up those 2 CPUS, 1 for the proxy and another for the backend.

Since the endpoint usage is light and not very regular, making its resource requirements smaller seemed like a good option. Managed to reduce the backend CPU requirement but not the proxy, therefore my question here.

right… i don’t see a clear path to customize the resource usage of the HTTPProxy actor though because it can vary so much. The proposed solution to

  • keep your k8s pod asking for 2 cpus
  • but tell ray via init argument it has more than 2 cpus so it will spawn more than 2 workers.
  • in this case, the 2+ worker processes (including the http proxy) will multiplex on the 2 cpus pod.