I am trying to port a complex application into Ray that uses multiple classes. Pretty much nothing works as expected or at all with any hierarchical complexity beyond single-class implementations.
I’m wondering if anyone has employed Ray for a complex application and if you would be willing to share your code so I could review what the best practices are. I really want to love and employ Ray but nothing seems to work.
Hey @mbw it would be great to hear more about what isn’t working. Perhaps there’s a reproduction we could take a look at?
A couple hierarchical libraries within the Ray package:
Serve uses a central coordinator actor to control many replica actors which actually serve models.
Within ray.data the PipelineSplitCoordinator is used in conjunction with remote tasks, to coordinate the execution/materialization of dataset shards.
Workflows use multiple coordinating actors which spawn tasks to execute workflows. Those in turn, may spawn other tasks (that talk to actors) for serialization/checkpointing purposes.