[Dask on Ray] Limit threads when using enable_dask_on_ray?

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

When using enable_dask_on_ray() is there a way to limit the number of threads per worker without changing environmental variables or changing how I initialize ray? When using it an a cluster, I’m getting many warnings like the following.

(raylet) OpenBLAS blas_thread_init: pthread_create failed for thread 49 of 64: Resource temporarily unavailable
(raylet) OpenBLAS blas_thread_init: RLIMIT_NPROC 1450887 current, 1450887 max
(raylet) OpenBLAS blas_thread_init: pthread_create failed for thread 50 of 64: Resource temporarily unavailable
(raylet) OpenBLAS blas_thread_init: RLIMIT_NPROC 1450887 current, 1450887 max
2

I don’t think this is possible. I think many threads are created by ray system, which is not controllable.

Besides, in the old Ray, there were bugs where we create threads more than expected. What version of ray are you using?