Learning about shared memory

Ray Core
1

Hi all, I’m trying to learn more about ray and how it’s sharing memory between processes. I know it was using plasma but from what I’ve heard it is now deprecated. Where can I learn details about how does ray allocate the shared memory? I’m interested in most up-to-date information and I tried to look in the Python code but there is many layers of abstraction that obstruct the view. If someone could point to a specific part of the code in Python and/or C++ side where does the allocation happen I would be greatly appreciative. I’m only interested in how it works on Linux.

Thank you all.

2

Hey @crackcomm welcome to the community!

I guess in short, ray plasma store uses mmaped files for shared memory between processes.

I think these are the most relevant source codes for plasma store shared memory allocation:

  • ObjectStore: for managing the allocated objects
  • PlasmaAllocator: for the actual memory allocation for objects.
  • dlmalloc: a fork of the dlmalloc for efficient malloc on the mapped files.
1 Like
3

Thank you for your answer. I have even more questions now.

It seems like much more complex memory management system than I have imagined.

My main question though is why does dlmalloc use malloc on the mapped files? Maybe you could refer me to some resource where I can read about it.

You referenced plasma allocator. From what I gathered it has it’s origins in ray project and was then a part of arrow project but is now deprecated. Is plasma only deprecated in context of arrow and still being big part of ray? It would be interesting to see what it was replaced with in arrow project.

I very much appreciate your time, if you are open to it I would have more questions.

4

My main question though is why does dlmalloc use malloc on the mapped files? Maybe you could refer me to some resource where I can read about it.

Hmm, where do you see malloc being used on the mapped files? Could you provide a pointer to it?

From what I gathered it has it’s origins in ray project and was then a part of arrow project but is now deprecated. Is plasma only deprecated in context of arrow and still being big part of ray? It would be interesting to see what it was replaced with in arrow project.

Probably more of a maintainance issue : [ARROW-17860] [Plasma] Deprecate Plasma - ASF JIRA

5

I took it from your answer, I don’t understand it really.

6

Oh I see, sorry about the confusion. We used the dlmalloc implementation for a malloc interface to manage memory on a mmaped files.

7

I now understand the plasma implementation. Thank you for your support. I will mark your first response as solution.

1 Like