[RLlib] GPU Memory Leak? Tune + PPO, Policy Server + Client

[mannyv]

Hi @Denys_Ashikhin and @kourosh,

Here are two posts that might be relevant here.

Memory issue debugging

Hi @Blubberblub, My guess is that your environment is generating samples much faster than the training is consuming them. This is causing the sample queue to fill up in the policy server which is causing the memory issue. Try adding the following print and see if this is growing: @override(InputReader) def next(self): print(f"Size of samples queue is: {self.samples_queue.qsize()}", ) return self.samples_queue.get() My env is not much faster than the training but if I…

[RLlib] tf2 framework slight memory leak