when running the following simple code, a large number of “ray::IDLE” shows up in the process. The number is equal to the number of CPUs in the node.
import ray.data as rd
rd.read_json("example.json")
I test it in a node with 8 processor and found the following processes:
15851 user 35 15 6922232 179840 76064 S 1.0 1.1 0:02.06 ray::IDLE
15852 user 35 15 6996100 178016 76260 S 1.0 1.1 0:02.05 ray::IDLE
15853 user 35 15 6996112 181764 75852 S 1.0 1.1 0:02.03 ray::IDLE
15854 user 35 15 6995976 179676 75928 S 1.0 1.1 0:02.10 ray::IDLE
15855 user 35 15 6996096 180160 76284 S 1.0 1.1 0:02.01 ray::IDLE
15856 user 35 15 6969584 185676 79136 S 1.0 1.1 0:01.70 ray::IDLE
15857 user 35 15 6996100 180148 76288 S 1.0 1.1 0:02.00 ray::IDLE
15858 user 35 15 6922244 183924 76064 S 1.0 1.1 0:01.95 ray::IDLE
Is this a bug or a feature?