is there a (pragmatic) way for streaming data between tasks or actors? Via the built-in object storage?
The idea is that one task or actor would be producing data, which should be streamed to another one?
is there a (pragmatic) way for streaming data between tasks or actors? Via the built-in object storage?
The idea is that one task or actor would be producing data, which should be streamed to another one?
There is no native streaming between task/actor from Ray as of now. However, there is a native library Ray Dataset built on top of object store, and users can use DatasetPipeline to achieve something similar to what you described. You may check out details here: Pipelining Compute — Ray 2.0.0.dev0