Zero-copy read support for sparse Scipy matrix

Ray Core
1

The Ray Core documentation states

The numpy array is stored as a read-only object, and all Ray workers on the same node can read the numpy array in the object store without copying (zero-copy reads). (source)

Does is this also apply to sparse Scipy matrices, i.e., spmatrix?

2

We support anything that can do zero-copy read over pickle5 protocol. I’m not sure whether this is on their roadmap.

3

Thank you for the clarification. It is unclear from my searching if spmatrix supports zero-copy read, or (if it doesn’t) when it will. Looking through their GitHub, I assume an implementation of __reduce_ex__ is necessary, based on PEP 574. Looking through the detailed roadmap, it is not mentioned. I also searched for “pickle” in Scipy’s release notes and did not find any mention of it there. The last thing I checked was pydata/sparse, which is Scipy’s new implementation of sparse arrays. No luck there either.

So I guess I’ll just try using a dict or list of numpy arrays to try and approximate a sparse matrix implementation.