How does serve sends objects over to another deployments?

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

How severe does this issue affect your experience of using Ray?

  • None: I am curious.

Hi,
I am wondering what happens when I have 2 deployments in one script and I for the sake of argument send a DB connection from one deployment to method of the other? It seems to work like pickle the DB client object, but the object is initialized on the other deployment with the defaults. Is this the case?

For example.

@deployment
class A:
    def __init__(self, config, B_handle):
        self.db_client = init_from(config)
        self.B = B_handle

    def do_work(self):
        self.B.do_stuff.remote(self.db_client)

@deployment
class B:
    def do_stuff(self, db_client):
        ...

Is it best to send objects that are pure data? Eg dataclasses, dicts, lists etc?

2

Serve uses ray.cloudpickle to serialize objects when sending them to other deployments. Cloudpickle is pretty robust, so you can use it with most Python objects, including the db_client from this example.

Check out the cloudpickle repo for more info.

3

From what I understand pickle.load would reinit the object? Is that correct?

4

Vanilla pickle serializes by reference, so it would reinit the object. cloudpickle can serialize by value, so it doesn’t necessarily reinit the object. The cloudpickle repo has more details about this.

By the way, you can define custom serializers in Ray to control how certain objects get serialized/deserialized.

1 Like