I’m trying to debug a very slow memory leak in rllib that occurs when i am using IMPALA + multi-agent. I cannot find any leak using tools like tracemalloc so I dont think the memory issue is in python. ray memory also does not show anty obvious leakage at all. All of the workers very slowly (over…

thank you for your reply. I found the problem after tracing, it is not a memory leak of ray, it is just that the speed of the learner consuming data is too slow, which leads to the accumulation of sampled data.

Hey @Bam4d , thanks for the reproduction script! I’ll take a look.

I can’t reproduce this on my Mac. Memory consumption seems very stable. The only change I did was to take out the GPU (num_gpus=0). I can try again on a GPU machine.

Like i said, its very hard to reproduce, It only seems to happen when on linux and cgroups are enabled. You might have to spin up a docker image (which i think uses cgroups) to reproduce. The issue is that most HPC services use cgroups for resource allocation, so running this on servers/docker will …

These lines simple_list_collector.py:488: # Make sure our mappings are up to date. agent_key = (episode.episode_id, agent_id) self.agent_key_to_policy_id[agent_key] = policy_id Is the episode ID unique to all episodes? That would mean that this “agent_key_to_policy_id” map would grow f…

Created a branch to test this theory here: GitHub - Bam4d/ray at ma_memory_leak

Awesome, could you PR this when confirmed?

yeah just in the process of trying to confirm if this is infact the problem I’ve also learned that python garbage collections does not actually “see” the limits set by cgroups, so it might just be that python thinks there is alot more memory than there is, and just keeps growing the objects. This m…

Hi @Bam4d , it seems like things are mostly wrapped up here, but in the future, you can try running ray memory --help to unlock some more helpful features. It seems like only object spilling is turned on by default. You can check out the corresponding docs here .

Either way, we are not cleaning up that dict ever, so that’s definitely a great catch by you, even if it’s just a small leak (leaking strings)! @Bam4d let me know, what else you find. I prepped this PR here it’s good to go, but feel free to do your own PR and ping me here for merging it. …

Small leaks over 1B timesteps end up being big leaks :wink:

Help debugging a memory leak in rllib

RLlib

Blubberblub September 25, 2022, 7:07am 22

@hridayns mannyv provided some advise and code on how to check the sample queue in this question if you are interested: Memory issue debugging

Topic		Replies	Views
Memory Leak when training PPO on a single agent environment RLlib	15	1962	December 24, 2022
Cannot identify which ObjectRef causes a memory leak and results in large object store spills Ray Core	10	467	June 6, 2024
[RLlib] GPU Memory Leak? Tune + PPO, Policy Server + Client RLlib	18	1509	May 29, 2023
RayOutOfMemoryError RLlib	2	847	May 24, 2021
[RLlib][Tune] Major memory leak 80GB (!) in 3 days (!) RLlib	1	380	June 3, 2021

Help debugging a memory leak in rllib

Related topics