I have a ray cluster of workers with 2 cores each
set it up the resources_per_worker variable of ray.train.Trainer to {“CPU”:2}
while training it’s only using one core of each worker
need to know how to utilise all cores of the workers
I have a ray cluster of workers with 2 cores each
set it up the resources_per_worker variable of ray.train.Trainer to {“CPU”:2}
while training it’s only using one core of each worker
need to know how to utilise all cores of the workers
Hi @siva14guru, can you share your code with us? Then we can help better to find the setting you need to adjust.
resources_per_worker
specifies how many resources you reserve for each worker, the worker itself then has to make sure it utilizes the resources. If you’re using multithreading, you may have to set OMP_NUM_THREADS=2
in the environment to utilize another CPU. This will show up in the dashboard as one CPU not being utilized and the other one as e.g. “200%”.
Hey @kai thank you for the reply
here is the script i am using on
https://gist.github.com/sivaguru-pat/fe1fcd3a6f80f6047752ded1dc8f64f2
as you said by setting OMP_NUM_THREADS=2 as environment variable it utilizes 200 % cpu