I was trying to use the cluster resources feature to tell each node how much of a custom resource
gpu_memory_mb it had, and then schedule my tasks based on their GPU RAM needs (same way I’m using
memory= for regular RAM). However, I seem to have to give each task a
num_gpus option > 0.0 for it to see the GPU at all.
I’ve been sticking in
num_gpus=0.1, assuming that the GPU RAM resource specification will be the only one that matters, but it feels very hacky. I also have to specify the GPU RAM totals manually when launching the worker. Any chance this would be something Ray could support in the core API, similarly to the
(The use case here is that I have limited access to a small number of GPUs with 32GB RAM, and easier access to some 16 GB GPUs. I’d like to tie them all together in a Ray cluster, but the GPU fraction required depends on which kind of node we’re talking about. Related to Gpu wise memory allocation )