Is ray.shutdown() a blocking call?

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Is ray.shutdown() a blocking call? Can I be sure that once this function returns there’s nothing ray-related else running on workers?

I have seen examples where people do below and I want to avoid that

ray.shutdown()
while ray.is_initialized():
  sleep(2)
2

hi @mil.ad unfortunately today the ray.shutdown is not a blocking call as today.

3

oh, that’s not good at all. So what is the endorsed way of shutting down ray safely (i.e. in a blocking way)

4

Hmm actually I just read code, and I think ray.is_initialized() is guaranteed to be False if you called ray.shutdown(). And it looks like we reset all the states when shutdown is returned. But it doesn’t guarantee all states are reset when ray.shutdown() is returned. For example, some of worker procs are probably still alive when ray.shutdown is returned (however, they will be cleaned up very soon)

5

That’s exactly what I need for my applications. The CLI version of shutdown seems to do that so I think I can call that through subprocess.