How to assign tasks to node evenly

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hi all, I have a problem with ray clusters that really hinders me.

The basic situation:

A total of 5 machines (each with only 2 CPUs,)
1 as head(IP: 10.1.0.4), others as 4 nodes(IP: 10.1.0.5-7)
they are all debian 11 (bullseys) systems, ray 2.2.0, python 3.9.16

I limit the number of CPUs available to head to 0, and the number of CPUs available to each node to 0, My number of tasks is also equal to the total number of cpu available in Node
but after each task is submitted, the tasks are not evenly distributed to the nodes, I have tried many times, and the distribution is always different each time.

1

2

3

Thanks guys, I really need to solve this problem.

# start the head
ray start --head --num-cpus=0

# start nodes
ray start --address='10.1.0.4:6379' --num-cpus=2

I’m happy to say that this problem seems to have been solved! I went back to the documentation and the “Specifying Task or Actor Resource Requirements” section solved my problem, I love this project!