Unable to pickle protobuf objects for parallelisation

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

We’re using grpc and protobuf objects as our communication method for a reinforcement learning project, Using python as a client and an external program as a server.
We’re currently in the process of trying to get multiple environments running in parallel to speed up training times, hence the use of ray.
However I am running into issues getting our existing communication methods to serialize properly for ray.remote etc.

This is the main error message that comes up:
“TypeError: cannot pickle ‘google.protobuf.pyext._message.MessageDescriptor’ object”

Any potential solutions or ideas? My python knowledge is lacking a little some i’m not exactly sure what the best way around this is.

@LukeWMG I think it might be a version issue. But I’m not sure. Is it possible for you to get some minimal reproducible examples and we can investigate on that.

You can try ray.util.inspect_serializability: Serialization — Ray 2.2.0. It might tell you why it’s not pickle-able.

1 Like