PPO with PyTorch GPU has a RAM memory leak for Ray 1.6.0

RLlib
1

Hi,

I am seeing a steady increase in RAM usage while training with multi-agent PPO on a GPU with Ray 1.6. The GPU memory usage is stable.
I am also running the same training on CPU with Ray 1.0.0 and have no memory issues.

Any ideas for a possible solution will be appreciated.

Thank you,
priyam

2

Seen similar leak with Ray 1.6 Torch SAC with replay buffer set to 500k:
ray_torch_sac_500K_buffer_leak

Eventually I get (15 GB machine):

PID    MEM    COMMAND
11076  7.48GiB ray::SAC.train()
11077  4.34GiB ray::RolloutWorker
3

Are you also using a GPU?

4

Are you also using a GPU?

Yes.

5

Follow up, in my case the environment render was leaking. I haven’t tested ray after the fix, so don’t know if there was an additional leak in ray.

6

Thank you. I do not think that is the case for me since it is working fine without GPU.