Limiting a driver

1

Is it possible to limit a driver to use only part of a cluster?

For example, if I have a cluster with 500 CPUs, can I use ray.init(address='auto', num_cpus=100) to limit a driver to only use 100 of them?

@sangcho

2

This is not supported in an API level. But, there are ways to achieve it.

  1. Use a placement group API Placement Groups — Ray v1.2.0. You can imagine you are creating a sub-cluster that is only dedicated to the driver. For example,
bundle = {"CPU": 4}
pg = placement_group([bundle] * 25, strategy="SPREAD") # Will create 4 CPUs bundle that will be spread across the cluster

@ray.remote
def f():
    pass

ray.get(f.options(placement_group=pg).remote())
  1. Otherwise, you might need some personal classes that manage the number of in-flight tasks (something like queue) to restrict your driver to use only 100 of cpus.

If this feature is important for your case, please create a feature request with the use case here; Issues · ray-project/ray · GitHub (I can imagine this could be an important multi tenancy feature)