Recently I came across this paper: https://dl.acm.org/doi/abs/10.1145/3452296.3472897
I wonder if this stuff has been implemented in Ray. Maybe as part of Ray collective?
Please let me know what’s the degree of integration/if any of Hoplite with Ray. Thanks!
Hi, currently Hoplite is not integrated with Ray OSS, mostly because of the complexity & garbage collection issues, so integration is not a high priority. Also, some refactoring of the object store is required before the integration.
However, I believe the general direction and some design choices are correct (e.g., fine-grained pipelining of buffers), and we have made some progress (e.g., merging raylet and object store).