ray.put puts a thing in an actor’s local object store on machine X. When a remote worker on machine Y decides it needs the thing, it fetches the thing from X, incurring some latency.
Is it possible to implement a ray.push, where you directly put the thing on Y, assuming I know what Y is by the time I generate the thing?
Could you put an Actor on Y and then you can just
ActorOnY.store() on X?
Right. Then you will have to implement the store method right. This seems to be something that should admit some efficient lower level implementation that avoids serialization similar to ray.put
Ok, so do you mean that you want to put the object in X and then push the ObjectRef of the object to Y?
In this case, you can push a list of ObjectRef to Y to avoid dereference (no data move and ser-de). Some pseudocode is like:
# On X
def __init__(self, actor_on_y):
self.actor_on_y = actor_on_y
def store(self, data):
obj = ray.put(data)
# On Y
def push(self, objs):
obj_ref = objs
# do something with obj_ref