# Memory leakage analysis via \`memray\` module

**URL:** <https://discuss.ray.io/t/memory-leakage-analysis-via-memray-module/15466>\
**Category:** Ray Core\
**Created:** [August 6, 2024, 11:06am UTC](https://discuss.ray.io/t/memory-leakage-analysis-via-memray-module/15466 "2024-08-06T11:06:20Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![shyampatel](https://avatars.discourse-cdn.com/v4/letter/s/cab0a1/32.png) [@shyampatel](https://discuss.ray.io/u/shyampatel)\
**Post date:** [August 6, 2024, 11:06am UTC](https://discuss.ray.io/t/memory-leakage-analysis-via-memray-module/15466/1 "2024-08-06T11:06:20Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- High: It blocks me to complete my task.

I am getting memory leakage in my pipeline. To analyse that I am using `memray` (python module) with live mode. While analysis, I could find that there was some method call, of which allocations are continuously increasing, but there is no Location for that, can check yellow rectangle in attached pic.

Can anyone help me here to get the location of that method? Please let me know if I can pass any further information.

 ![Screenshot from 2024-08-06 16-34-10](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/e/e9150f6768cf0ed72aed150a0a33adaccf30f574.jpeg)

---

<div class="post-metadata">

**Author:** ![Ruiyang\_Wang](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/ruiyang_wang/32/4818_2.png) [@Ruiyang\_Wang](https://discuss.ray.io/u/Ruiyang_Wang)\
**Post date:** [August 6, 2024, 11:29pm UTC](https://discuss.ray.io/t/memory-leakage-analysis-via-memray-module/15466/2 "2024-08-06T23:29:37Z")

</div>

Maybe this is the native calls (c++ calls). can you do `memray run --native`? It may reveal the ???.

---

<div class="post-metadata">

**Author:** ![shyampatel](https://avatars.discourse-cdn.com/v4/letter/s/cab0a1/32.png) [@shyampatel](https://discuss.ray.io/u/shyampatel)\
**Post date:** [August 7, 2024, 5:34am UTC](https://discuss.ray.io/t/memory-leakage-analysis-via-memray-module/15466/3 "2024-08-07T05:34:56Z")

</div>

Thanks @Ruiyang_Wang for your support. I have tried with `--native` option and it indeed help in resolving the `???`. Issue here is that, it’s not giving stats (memory, allocations, etc.) with this option enable.

I am running live mode in remote via following code lines:

```auto
destination = memray.SocketDestination(server_port = 6121, address = '127.0.0.1')
memray.Tracker(destination=destination, trace_python_allocators=True, native_traces=True). __enter__ ()

```

I guess it’s not supported in live mode. Can you confirm this once? It would be better, if you can suggest any other method to get those native calls or to find memory leakage.

---

<div class="post-metadata">

**Author:** ![Ruiyang\_Wang](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/ruiyang_wang/32/4818_2.png) [@Ruiyang\_Wang](https://discuss.ray.io/u/Ruiyang_Wang)\
**Post date:** [August 7, 2024, 4:04pm UTC](https://discuss.ray.io/t/memory-leakage-analysis-via-memray-module/15466/4 "2024-08-07T16:04:37Z")

</div>

can you try this Ray-built in memray support? In the Ray Dashboard (localhost:8265) find your worker process and click “Memory Profiling”

 ![image](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/6/65f7b51930a73758bac90bfb33c3c2f3d2a8d778.png)

 ![image](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/5/5d37a19adfe24cd8afe78bfd4ab3843374c84d62.png)

---

<div class="post-metadata">

**Author:** ![shyampatel](https://avatars.discourse-cdn.com/v4/letter/s/cab0a1/32.png) [@shyampatel](https://discuss.ray.io/u/shyampatel)\
**Post date:** [August 9, 2024, 4:24am UTC](https://discuss.ray.io/t/memory-leakage-analysis-via-memray-module/15466/5 "2024-08-09T04:24:53Z")

</div>

@Ruiyang_Wang I am using ray `2.8.0` version, which does not have the mentioned feature.

---

<div class="post-metadata">

**Author:** ![Sam\_Chan](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sam_chan/32/5011_2.png) [@Sam\_Chan](https://discuss.ray.io/u/Sam_Chan)\
**Post date:** [August 9, 2024, 5:54am UTC](https://discuss.ray.io/t/memory-leakage-analysis-via-memray-module/15466/6 "2024-08-09T05:54:44Z")

</div>

@shyampatel can you please upgrade to latest ray?

---

<div class="post-metadata">

**Author:** ![shyampatel](https://avatars.discourse-cdn.com/v4/letter/s/cab0a1/32.png) [@shyampatel](https://discuss.ray.io/u/shyampatel)\
**Post date:** [August 9, 2024, 6:13am UTC](https://discuss.ray.io/t/memory-leakage-analysis-via-memray-module/15466/7 "2024-08-09T06:13:07Z")

</div>

Hi @Sam_Chan, I am working on upgrading the pipeline to `ray=2.34` and `python3.12`. Meanwhile, we could able to reproduce memory leakage via script with `ray==2.8.0` and `python3.8`. Once, the analysis is done, I will create another thread for the same.

---

<div class="post-metadata">

**Author:** ![shyampatel](https://avatars.discourse-cdn.com/v4/letter/s/cab0a1/32.png) [@shyampatel](https://discuss.ray.io/u/shyampatel)\
**Post date:** [August 9, 2024, 7:00am UTC](https://discuss.ray.io/t/memory-leakage-analysis-via-memray-module/15466/8 "2024-08-09T07:00:50Z")

</div>

@Sam_Chan @Ruiyang_Wang I have created separate thread ([Memory leakage with ray.put()](https://discuss.ray.io/t/memory-leakage-with-ray-put/15501)) for addressing memory leakage issue. One thing, I could relate here is that, `ray.put(object)` is causing the continuous increment in allocations count of `???`.
