Is there a way to programmatically check whether or not some particular piece of data can be serialized and de-serialized without copy?
One reason I am asking is that I did not find the documentation very clear on what kind of data exactly is eligible for the zero-copy behavior. The documentation indicates that:
You can often avoid serialization issues by using only native types (e.g., numpy arrays or lists/dicts of numpy arrays and other primitive types), or by using Actors hold objects that cannot be serialized.
However, this leaves many questions unanswered:
- Do pytorch arrays count? (it seems to be the case)
- What about an object whose type is a subclass of np.ndarray?
- What about nested lists and dictionaries?
- What about nested instances of python dataclasses with array fields (it is unclear to me why this would not be supported if ray uses the Pickle 5 protocol but the documentation is making me doubt).
Ideally, I would love a function like ray.util.inspect_serializability having a flag to also troubleshoot issues with the zero-copy behavior.