Ray.init() hangs on MacOS

Ray Core
1

I have a problem with Ray on my macos machine. I had been using Ray with Rllib for a while but after some break, I cannot make it work again. Ray is hanging in the ray.init() function.

I have Sonoma 14.6.1 and Ray 2.4.0. I tried both conda and venv virtual environments with the same result.

My code looks as follows:

import ray
ray.init()

In the /tmp/ray/sessiong_…/logs folder I have the following files/folders:
debug_state_gcs.txt
events (contains empty event_GCS.log file)
gcs_server.err (empty)
gcs_server.out
monitor.err (empty)
monitor.out (empty)
old (empty)

The debug_state_gcs.txt file ends with lines:

Event stats:
RaySyncer.deadline_timer.report_resource_report - 71784 total (1 active), CPU time: mean = 37.238 us, total = 2.673 s
GcsInMemoryStore.Put - 8795 total (0 active), CPU time: mean = 12.086 us, total = 106.300 ms
InternalKVGcsService.grpc_server.InternalKVPut - 8794 total (0 active), CPU time: mean = 26.208 us, total = 230.470 ms
InternalKVGcsService.grpc_client.InternalKVPut - 8790 total (0 active), CPU time: mean = 10.006 us, total = 87.953 ms
RayletLoadPulled - 7264 total (1 active), CPU time: mean = 4.294 us, total = 31.191 ms
GCSServer.deadline_timer.debug_state_dump - 735 total (1 active, 1 running), CPU time: mean = 12.766 ms, total = 9.383 s
GCSServer.deadline_timer.debug_state_event_stats_print - 152 total (1 active), CPU time: mean = 4.932 ms, total = 749.731 ms
GcsInMemoryStore.GetAll - 6 total (0 active), CPU time: mean = 5.460 ms, total = 32.758 ms
PeriodicalRunner.RunFnPeriodically - 4 total (0 active), CPU time: mean = 97.500 us, total = 390.000 us
InternalKVGcsService.grpc_server.InternalKVGet - 1 total (0 active), CPU time: mean = 36.000 us, total = 36.000 us
GcsInMemoryStore.Get - 1 total (0 active), CPU time: mean = 16.000 us, total = 16.000 us

I would be grateful for any help investigating and solving the problem.

2

I have found that ray.init() hangs in the start_ray_process function called with process_type=“monitor”. The start_ray_process function calls the Popen constructor which calls _execute_child() which calls _posixsubprocess.fork_exec and hangs in it.

3

TY for reporting this @Elena . 2.4 is pretty old; do you mind upgrading to latest and see if it still repros?

4

I have upgraded Ray to 2.37.0 and it helped. Thank you!