It seems that data transfer through the Ray client is via a gRPC call, but that inside a Ray cluster will use multiple gRPC connections to maximize throughput (I saw this from here).
Additionally, is it possible that the Ray client introduces extra memory copies / serializations compared to inside a Ray cluster? I know that the Plasma Object Store implements zero-copy for numpy arrays.
So I’m wondering if there is difference between the efficiencies of the two. Do you have any analysis or benchmarks on this?
Thanks!
hi @Hanyu
Exactly as you mentioned; the Ray client is not as efficient as Ray object transfer, and it does incur extra copy/serializations.
This is due to the fact the Ray client is designed to submit a script or do interactive development to a remote cluster, where the network quality is unpredictable, and performance is not a high priority requirement.
If you care about the performance and overhead, I’d suggest you not using Ray client, and consider either connecting Ray on the head node directly or consider Ray’s job submission Ray Job Submission — Ray 1.13.0.
I see. Thank you so much!