Num_cpus as a parameter?

Ray Core
1

is it possible to specify the value of the num_cpus parameter of the ray.remote decorator as a parameter determined during runtime?

2

Hi @mbehrendt,

You can dynamically specify the value through .options - see Dynamic Remote Parameters.

3

great – thank you @matthewdeng

Just a quick follow-up question, based on this statement in the docs you linked to:

Ray will need to be started with sufficient CPU resources and the relevant custom resources:

Is that also true if the max_workers value in the cluster.yaml is not yet reached? E.g. all resources on the currently running nodes are exhausted, but if ray scaled by adding more resources, it would be able to run the current task?

Also, if I’d to run nodes of different sizes in the same cluster – would that be possible?

4

Is that also true if the max_workers value in the cluster.yaml is not yet reached? E.g. all resources on the currently running nodes are exhausted, but if ray scaled by adding more resources, it would be able to run the current task?

Yeah, as long as your cluster scales so that the total available resources is greater than what’s requested by the current task, it will eventually run.

Also, if I’d to run nodes of different sizes in the same cluster – would that be possible?

Yep, you can specify different available_node_types.

5

for example – if i created 10 available_node_type definitions, each of them backed by a different machine profile. Which algorithm does ray use to pick the node type to provision? Is it based on the closest amount of cpu/mem capacity, or something else?

6

The heuristic used is here.

# Prioritize matching multiple resource types first, then prioritize
# using all resources, then prioritize overall balance
# of multiple resources.
7

great - thanks a lot. Is there also a way for me as a developer to somehow explicitly specify deterministically which node type i’d like to have used (and if needed, provisioned on demand) ? E.g. as a dynamic remote parameter?

8

You can annotate the node with a custom resource in the autoscaling config and then specify that resource in the remote decorator in a ray program.