Hey, I am working with Ray’s object store and spilling in external storage. I wonder are the restored objects pinned in the object store?
Restored objects are not pinned because it won’t have owners anymore. But usually objects are restored when they are needed (e.g., we are executing tasks), and at this time, we temporarily pin them to avoid thrashing (until tasks are executed).
1 Like
Thank you! I am also a bit confused about objects being evicted and objects being spilled to the external storage. From what I understand, only pinned objects can be spilled to external storage and then restored providing the spilled URL. The non-pinned objects are simply evicted from Plasma following an LRU policy? Are they somehow reconstructed when needed?
Thank you! I am also a bit confused about objects being evicted and objects being spilled to the external storage. From what I understand, only pinned objects can be spilled to external storage and then restored providing the spilled URL. The non-pinned objects are simply evicted from Plasma following an LRU policy?
Yes it’s correct! There’s actually more sophisticated rules than LRU for evicting non-pinned objects (non-primary copy) which you can find from the whitepaper. Architecture Whitepapers — Ray 3.0.0.dev0
Are they somehow reconstructed when needed?
We have object reconstruction support (Fault Tolerance — Ray 3.0.0.dev0) when the owner of the object is dead (meaning the primary copy is removed) by retrying the lineage of the objects. This feature is currently not battle tested, but we are planning to do that very soon in a couple month. If you’d like to try out and give us feedback, that’d be awesome 
1 Like