How to use a particular node for ray serve?

Ray Serve
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Hi,
I have a ray cluster on AWS with multiple node types like V100s, A10Gs, etc.
I want to understand a way in which I can define ray serve to use a particular worker node. For example, if the name of the worker node is “my_a10g_node”, how can I use it? I looked through the custom resource but didn’t understand how to use it with Ray serve.

Any examples of usage will be helpful.

2

Hi @plum9, does using ray_actor_options work, together with custom resources? Scaling and Resource Allocation — Ray 2.0.0

https://docs.ray.io/en/latest/ray-core/configure.html#configuring-ray

So you could run ray start --resources='{"a10g": 1}' on your special node when starting Ray, and then pass in the resource requirement to Serve deployment as follows:

@serve.deployment(ray_actor_options={"resources": {"a10g": 0.1}})
def func_1(*args):
    return do_something_with_my_a10g_node()