Running Ray actors on worker nodes directly on Ray

Ray Core
1

**How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Hi, I want to do a workflow like the following:

  • Set up a Ray cluster using a config file which will give me both head and worker nodes (along with the CPU / GPU resources I need for each)
  • Launch a Ray Actor on a given worker node and be able to poll the job status
  • Have the option to add or get rid of worker nodes as I want

Is the above workflow possible in Ray? Looking in the docs I find bits and pieces that answer part of the above. For example I know how to launch a cluster, and use Ray actors to remotely run functions on Ray clusters, however if possible I’d like to more finely control where specifically they’re run. Is it possible to poll Ray actors and get a status like you would for launching jobs?

2

@humzaiqbal Actor has options to control it’s placement using placement group.

Alternatively if you know the node’s ip you can use actor.options(resources={"node:node.ip.address"}...) to schedule the actor on a specific node.

For downscaling @Dmitri might be able to help here.

3

There’s no standard workflow to support that.

4

Actually, there is a notion of “unmanaged worker” for this.
But it’s not a commonly used or well-tested feature (no stability guarantees).

You can read about it here –
https://docs.ray.io/en/latest/cluster/config.html