Out of curiosity:
At Out-of-band Communication — Ray 2.23.0 it says “You can start a http server inside the actor and expose http endpoints to clients so users outside of the ray cluster can communicate with the actor.”
With a non-AI/ML application, are requests sent from the master to the http server running on each worker node (actor) using ray’s collective library?
Alternatively, if the application is already using a different communications stack (such as mpi) then this is possible too but with the documentated limitations?
Hi Henry,
When an HTTP server is used, then the HTTP userspace library + OS network stack will be used rather than Ray’s collective library to communicate between the server and clients. Thus, MPI will not be used either for the client-server HTTP communication.
Please let me know if I misunderstood your question or you have any follow-ups.
Thanks,
Jack
To clarify the question: A master (server) sends a request to the nodes. Each node (a server or core) runs an http server. On receiving a request, each node executes it and returns a result to the master. Does the communication between the master and the nodes use the ray collective library eg. gloo?
If all of the communication is happening within HTTP (both the request and the response that contains the result), then the Ray Collective library is not used. You would need to specifically call into the ray.util.collective library for it to be used, and you would not be able to send the result in an HTTP reply in this case.
Just in case the question is over the “out-of-band” language mentioned in the documentation you linked, this simply means that the communication happens outside of the Ray Distributed Object Store, which an application interacts with via ray.put() and ray.get(). It does not mean that third-party libraries will automatically use the Ray Collective library, including AIOHTTP.