How severe does this issue affect your experience of using Ray?
- Medium: It contributes to significant difficulty to complete my task, but I can work around it.
Hi! I know there is ray.util.ActorPool and the possibility to submit asynchronous jobs via ray.util.ActorPool.submit. However it doesn’t return a future and results can only be retrieved using get_next or get_next_unordered which are blocking, return the result value, and offer no way of connecting the retrieved result to a submission call.
OTOH python’s concurrent.futures.Executor.submit returns a concurrent Future, which can be awaited in calling code. This is very useful if several concurrent processes call the same PoolExecutor, bc. each process can await its respective future separately.
With ray.util.ActorPool I don’t see a way to submit tasks to the ActorPool from several concurrent remote tasks, s.t. each calling task gets a future for the job it submitted and await it separately.
Is there a way to achieve concurrent.futures.Executor.submit behavior in that regard with current ray code?
@Valentin_Stauber The ActorPool returns an iterator object, hence using pool.get_next for any submitted tasks to the pool of actors.
I don’t believe it supports the behavior concurrent.futures.Executor.submit.
cc: @chengsu for any more context or feedback
@Jules_Damji looking at the source code of ActorPool you can see that the futures are there and used internally of course. But they are not exposed to any calling code to the pool. I would like to know if it is possible to expose these futures, s.t. they can be awaited outside of the pool (e.g. in calling ray tasks).
@Valentin_Stauber Right, it’s internal and not exposed by design. If you want these exposed, please
file an issue and the reason why you think it’s necessary for a use case. We will be more than happy to evaluate it, since it would require a bit of behavior and API change.
cc @chengsu
will do, thanks @Jules_Damji
@Valentin_Stauber Thanks for the issued. For now, I’ll close this issue as resolved with a link to the github issue:
Keep on using the APIs and let’s us of any feedback by filing issues, enhancements, and even file PRs if needed. The team will more than happy to review your contributions.