Specifying extra resources for functions (tasks) running inside an Actor?

How severe does this issue affect your experience of using Ray?

  • Low: It annoys or frustrates me for a moment.

I’m running Ray in a relatively constrained environment, with a single GPU, running soft-realtime ML inference tasks. These tasks use known amounts of GPU-memory. They are all instantiated into an Actor so that the we only pay initialization cost once, and the models stay in memory.

I’m tracking GPU memory as a custom gpu_memory resource that is set when starting Ray. Let’s pretend that’s a value in GB to make this example easier.

I’ve set the Actor to need a custom resource:

@ray.remote(max_concurrency=10, resources={'gpu_memory': 1})
class Actor:
  def __init__(self):
     # Load models into memory...

  def do_work(...):

That covers the steady-state GPU memory usage, but each task also requires more GPU memory to actually run, based on its inputs. What I’d like to do is something like this:

@ray.remote(max_concurrency=10, resources={'gpu_memory': 1})
class Actor:
  @ray.remote(resources={'gpu_memory': 2}) # or similar
  def do_work(...):

This doesn’t work, I assume because the @ray.remote decorator is not designed to work on class methods.

I was hoping specifying options on the method call would work like it does with a normal task:

actor.do_work.options(resources={'gpu_memory': 2}).remote(...)

This also doesn’t work.

The workaround I’m using right now is to create a separate task that calls into the actor to reserve the resources:

@ray.remote(resources={'gpu_memory': 2})
def do_work(actor, ...):
  ref = actor.do_work.remote(...)
  return ref

This works, but is a little odd as a pattern. Is there a simpler built-in way to do this?


It is currently not possible. I remember there was a feature request for it, but I cannot find it now… Do you mind creating a enhancement request to our github repo and tag rkooo567 there?

Done: [Core] Ability to specify resource requirements on Actor methods · Issue #39910 · ray-project/ray · GitHub