I haven’t see anything like this in the documentation so I believe it does not exist. Would it be possible or even make sense to have ray worker (as in a worker node) have labels that could be defined as an option same as resources?
The would allow work to be redirected to nodes with a specific label. Right now it can be done with custom resources, although seems to be a misuse of the resource functionality, as it’s not really using up any resource just pointing to workers that has a specific set of libraries installed for example.
Hmm this is an interesting use case. Ray worker processes are usually dynamically created and destroyed rather than sitting there forever (and resources are allocated to them on demand, not like each worker has allocated resources). How do you have a specific set of libraries for a specific worker now?
There are 2 types of worker nodes with different custom resources and setup commands and eventually even images. This allows for a task to define that it needs a ResourceB and the cluster will create a worker_b instance if one is not available to run that task. This makes sense if ResourceB is actually a resource, but in a use case where you just want the task to have a different environment it’s just serving for a workaround to what could be a Label to be matched against the available workers.
Ah, I see. I definitely misunderstood the question. The following way is actually the recommended way from our end (treat env like resources). We discussed about introducing constraints or env type of resources (which is more straightforward), but it didn’t actually happen at the end. If you think that sort of APIs are more desirable, please create an enhancement request!
@Alex
I can achieve what i need using custom resources, it just seems odd that I have to assign a number to the resource when it does not really have a resource count. Are there any special values to assign, such as infinite?