When using ray data, how can we provision two GPUs and share them across multiple tasks?

Ray Data
1

We wanted to specify num_gpus=2 for our tasks but we also want other tasks to be coscheduled and share the two GPUs. How can we establish this?

2

Hi Suren and welcome to the Ray community!

Have you looked into fractional GPU usage? For example if you want 2 tasks to share the same gpu you can allocate num_gpu=0.5 for each task. So they can concurrently run on 1 gpu. You can allocate them accordingly based on num_gpus and the fractions. Not sure if this helps the use case you are describing but let me know.

3

but we still want at minimum two GPUs for correctness. how would we do that?

4

Hi! So for clarity, you have tasks to use 2 gpus no matter what but still have the exact same 2 gpus be allocated for other tasks, correct?

For example, Task A wants num_gpu=2, Task B is num_gpu=2, etc… but you want them all to run on the same 2 GPUs?

5

Yes! We want to be able to specify minimum capacity and optimize for utilization.

6

Could you share a bit more detail about your workload? Like what are the tasks, what you’re trying to do at a higher level?

7

We have a task that runs a C++ binary for us and that binary uses a fraction of the GPU as of today. Now that binary wants to run the workload with two GPUs but it will not use the two GPUs fully. We want to put co-run more tasks that need two GPUs at minimum.