Can we make ray evenly schedule tasks on different GPUs?

Ray Core
1

Hi guys. I am facing a problem here.

I have a computation task that roughly utilizes 70% of single GPU computation power but with little consumption of the memory. Ideally, I want to put two tasks on a single to fully utilize the resources. In my code, I set num_gpus=0.5 for each task.

When I run 8 tasks on a server with 4 GPUs, everything works as expected. However, if I only run 4 tasks, these tasks will only be scheduled on 2 GPUs, leaving the other two unused. I understand ray is trying to save the card for other tasks that may require a whole GPU. But it is not efficient in my case. Is there any hack that I can use to make the 4 tasks scheduled evenly on 4 cards?

2

cc @Alex Isn’t it just supposed to be running on 4 gpus? Can you answer his question?

3

Hmmm there isn’t really a good way to change the scheduling behavior here. One hack you could do is to split your machine into 4 virtual nodes, then use placement groups with a SPREAD policy for your tasks. It admittedly isn’t very elegant, but it should work.

4

Thanks for your reply. I will give it a try.