PPO with PyTorch GPU has a RAM memory leak for Ray 1.6.0


I am seeing a steady increase in RAM usage while training with multi-agent PPO on a GPU with Ray 1.6. The GPU memory usage is stable.
I am also running the same training on CPU with Ray 1.0.0 and have no memory issues.

Any ideas for a possible solution will be appreciated.

Thank you,

Seen similar leak with Ray 1.6 Torch SAC with replay buffer set to 500k:

Eventually I get (15 GB machine):

11076  7.48GiB ray::SAC.train()
11077  4.34GiB ray::RolloutWorker

Are you also using a GPU?

Are you also using a GPU?


Follow up, in my case the environment render was leaking. I haven’t tested ray after the fix, so don’t know if there was an additional leak in ray.

Thank you. I do not think that is the case for me since it is working fine without GPU.