I browsed the documentation and could not find a way to specify a task/actor to be run on a specific machine. Is there a way to do this easily? The best work around I can think of is to create many iterations to force the propagation of the task to all available nodes, then have the code compare its hostname against some type of argument.
TLDR: How do I to send a broadcast message to all available nodes and have a specific subset of node to do something?
Yes, but I don’t think that is enough. If I understand how it works, it is a schedule strategy only, and will not guarantee communication to all nodes.
For examples, I have N nodes.
If I do something like
object_ids = [test_remote.remote() for i in range(20)],
it MIGHT reach the first 20 nodes. Chances are, many of the tasks will be executed on the same node.
Another issues is, Ray is really dealing with virtual cores, and not “machines” per se. What I end up doing is what I had mentioned in my original post, you create many task hoping to that is enough to to reach every single machines. If the number is too low, some machines might never receive the message. If the number is too high, it is wasteful of resources.
What I’m looking for is, a way run a task only once per machine (not per core), only a subset of machines available on the network. This would be trivial with Kafka, since you can subscribe to a command topic and respond when called, but I don’t think it is a feature that is currently supported within Ray.
thanks, I do not think ray has fine grained task placement mechanism that you need for your requirement. As far as I understand, ray users submit tasks and scheduler assign workers based on compute units requested.