Using nodeid as custom resource

1

I have the following code of using node id as custom resource, but looks like it doesn’t work.

import ray,time
ray.init()
@ray.remote
def fun():
    time.sleep(10)
    return ray._private.state.current_node_id()

f1 =[fun.options(num_cpus=1,resources={"node:127.0.0.1":0.0005}).remote() for _ in range(3)]

result=[]
#result=ray.get(f1)
for ff1 in f1:
    print('waiting 3 sec for obj:{}'.format(ff1))
    try:
        result.append(ray.get(ff1,timeout=3))
    except Exception as e:
        print('cancelling task')
        ray.cancel(ff1)
print(result)
2

Hey @valiantljk,

Since you have a timeout of 3 seconds for a task that takes at least 10 seconds to finish, even when the task is scheduled the ray.get call is expected to error out with a ray.exceptions.GetTimeoutError.

You should be able to get this to run successfully by removing the sleep call:

-     time.sleep(10)

Alternatively, you can increase or remove the timeout:

-        result.append(ray.get(ff1,timeout=3))
+        result.append(ray.get(ff1))

Furthermore, to test just the behavior of using node ID as a custom resource, you can do so by simplifying the script a little as such:

import ray
ray.init()

@ray.remote
def fun():    
    return 1

result = ray.get(fun.options(num_cpus=1,resources={"node:127.0.0.1":0.0005}).remote())
print(result)
1 Like
3

Scheduling — Ray 3.0.0.dev0 from ray 2.0, we recommend you to use NodeAffinityScheduling instead.

1 Like