Memory Profiling in Ray

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I want to measure the memory usage of my Python code which uses Modin with Ray. Based on the the Memory Management page, the application memory usage consists of 3 parts. I have the following code to measure one of them. How can I complete this code, to have the complete application memory usage?

mem_before = int(c3.Ray.availableResources()["object_store_memory"])
# do some operation
mem_after = int(c3.Ray.availableResources()["object_store_memory"])
return mem_after - mem_before
2

@tech_wanderer this is an area we are working on. Is total worker heap memory usage enough, or per-worker / per-node usages are needed?

3

We are interested in both. The total memory is higher priority for us.

Also, is there any related GitHub ticket we can follow?