How to enable @ray.remote decorator programmatically

Ray Core
1

Hello,
I would like to be able, also for debug purposes and speed of program start, to be able to activate or deactivate ray and therefore the @ray.remote decorator programmatically. Is there a way to do this?
Thanks

1 Like
2

Maybe you can try instead of using the decorator, using:

def func(): ...

func = ray.remote(func)

NOTE: this is equivalent to:

@ray.remote
def func(...):
1 Like
3

@rliaw How do I use func = ray.remote(func) but also specify resources?

I tried this:
driver.py

from main import train_model_for_one_customer
for selected_customer in selected_customers[:num_of_models]:
    remote_training_function = ray.remote(num_cpus=12, max_retries=1)(train_model_for_one_customer)
    ray_refs.append(remote_training_function.remote(selected_customer,
                                                    *selected_dates[selected_customer],
                                                    model_version=model_version,
                                                    customer_df_dict_ref=customer_df_dict_ref,
                                                    num_of_jobs=num_of_jobs))

And train_model_for_one_customer is defined in main.py which is a file in the same dir as driver.py.

However Iā€™m getting

Status message: Job failed due to an application error, last available logs (truncated to 20,000 chars):
  File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/_private/worker.py", line 2380, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(RuntimeError): ray::train_model_for_one_customer() (pid=27166, ip=10.12.3.2)
RuntimeError: The remote function failed to import on the worker. This may be because needed library dependencies are not installed in the worker environment:

ray::train_model_for_one_customer() (pid=27166, ip=10.12.3.2)
ModuleNotFoundError: No module named 'main'

It used to work with if I use the remote decorator.

4

Okay, I was able to resolve it by myself by adding my package to pymodule. Thanks!