We have a centralized server that is essentially routing requests from a UX to the ray cluster deployed to EKS and I’m curious as to the limitations around a single ray client connection.
Is there a MAX client connection count when connecting to a ray cluster via the ray client?
think its 50, there’s a variable CLIENT_SERVER_MAX_THREADS which is set to 100, and each client connections creates at least two threads (one for dataservicer and one for logservicer).
We don’t measure the performance implication of having each connection, so you may need some testing here. Also, we recommend you not to use ray client if possible. See Ray Client — Ray 3.0.0.dev0