How severe does this issue affect your experience of using Ray?
- Medium: It contributes to significant difficulty to complete my task, but I can work around it.
I am starting ray cluster with custom resources in each node using cluster.yaml
file. For example:
node1 resources => {"work1" : 1}
node2 resources => {"work2" : 1}
node3 resources => {"work3" : 1}
In my pipeline, I have three actors,
cam_actor (resource required : `work1` to force it to run in node1,
det_actor (resource required : `work2` to force it to run in node2,
stream_actor (resource required : `work3` to force it to run in node3
Now, to pass data from one actor to another, I have two ray queues,
frame_queue (to share data between cam and det)
det_queue (to share data between det and stream)
Here, my question is, is it valid / preferable to share ray queue between multiple nodes?
If yes, How ray queue internally works for multiple nodes?
or What are the other ways to share data between multiple nodes?