### Author:  [@mannyv](https://discuss.ray.io/u/mannyv)
#### Post date: [December 20, 2022, 4:32pm UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/5 "2022-12-20T16:32:29Z")
@MrDracoG,
Well that really doesn’t clear much up does it? Can you share anything about your configuration or setup? A reproduction script?
One thing I would try next is to swap out the real emvironment for a Random Env to try and disentangle if the leak is in the environment or the rl algorithm.
If you have a custom model I would also try switching to a built in model.
>
---
### Author:  [@MrDracoG](https://discuss.ray.io/u/MrDracoG)
#### Post date: [December 20, 2022, 4:59pm UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/6 "2022-12-20T16:59:18Z")
I agree that it doesn’t really clear much up. I have a custom model and environment that I am working with. I think it would be hard to get you a reproduction script without sending in the whole repository. Like I said before, I am working inside of a docker container. However, I am currently running a test outside of a container. If that doesn’t clear things up, I will be sure to test a random environment and get you a reproduction script if the memory leak still persists. I’ll post back shortly with info about training outside of docker.
Btw, thank you for taking the time to respond.
---
### Author:  [@MrDracoG](https://discuss.ray.io/u/MrDracoG)
#### Post date: [December 20, 2022, 5:03pm UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/7 "2022-12-20T17:03:31Z")
Here is the ram usage outside of docker ( 2 workers )…
The line seems to be fairly flat and, to me, this seems to signal that training inside of a docker container may be causing the memory leak .
Here is the other thread that referenced docker containers and linux cgroups related to a memory leak: [Help debugging a memory leak in rllib](https://discuss.ray.io/t/help-debugging-a-memory-leak-in-rllib/2100)
I would also like to note that I don’t think there was a memory leak when using a single worker ( and no gpu ) inside of a docker container. I will go back and check that out.
---
### Author:  [@MrDracoG](https://discuss.ray.io/u/MrDracoG)
#### Post date: [December 20, 2022, 5:45pm UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/8 "2022-12-20T17:45:27Z")
Nevermind, even with a single worker there is a memory leak inside of a docker container.
---
### Author:  [@arturn](https://discuss.ray.io/u/arturn)
#### Post date: [December 20, 2022, 7:11pm UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/9 "2022-12-20T19:11:55Z")
We also run some memory leak tests. Specifically for PPO. So I’d say it’s not super likely that this stems from inside RLlib itself. This can also happen if your rollouts are crazy long I guess.
---
### Author:  [@MrDracoG](https://discuss.ray.io/u/MrDracoG)
#### Post date: [December 20, 2022, 11:57pm UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/11 "2022-12-20T23:57:44Z")
I went back and did a longer (relative to earlier) run with a single worker both inside and outside of docker today and it doesn’t seem like there is a difference for the memory leak, so please ignore what I said earlier about it being in docker.
Inside docker:
Outside docker:
---
### Author:  [@MrDracoG](https://discuss.ray.io/u/MrDracoG)
#### Post date: [December 21, 2022, 12:03am UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/12 "2022-12-21T00:03:34Z")
Yeah, I think that it is possible to be external to RLlib itself.
What would be an example of a crazy long rollout?
I appreciate your response.
---
### Author:  [@MrDracoG](https://discuss.ray.io/u/MrDracoG)
#### Post date: [December 21, 2022, 12:09am UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/13 "2022-12-21T00:09:08Z")
What are your thoughts on the MemoryTrackingCallbacks callback [[How To Contribute to RLlib — Ray 3.0.0.dev0](https://docs.ray.io/en/master/rllib/rllib-dev.html?highlight=memory%20leak#finding-memory-leaks-in-workers)] returning worker/data\_mean, worker/rss\_mean and vms\_mean as some of the top 20 memory users that also tick up with the ram usage.
I am gonna try to run the training with the MemoryTrackingCallbacks callback longer… just gonna wait until overnight to do it because the callback seems to slow down the training quite a bit.
---
### Author:  [@arturn](https://discuss.ray.io/u/arturn)
#### Post date: [December 21, 2022, 9:06am UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/14 "2022-12-21T09:06:50Z")
I think the first thing you should do would be what @mannyv suggested - replace your env with a random env. It’s very little work.
A crazy long rollout depends on your setting. If your episodes are 10k steps long and you set `rollout_fragment_length` to 10k, RLlib’s sample collection code will buffer these 10k samples for each env you are evaluating on. So for 100 workers, that would be 1M samples. That’d be a lot of memory even for simple atari envs.
---
### Author:  [@arturn](https://discuss.ray.io/u/arturn)
#### Post date: [December 21, 2022, 9:07am UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/15 "2022-12-21T09:07:08Z")
Please use @mannyv’s advice or post a reproduction script.
---
### Author:  [@MrDracoG](https://discuss.ray.io/u/MrDracoG)
#### Post date: [December 21, 2022, 4:52pm UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/16 "2022-12-21T16:52:46Z")
I used the RandomEnv class for the environment. It doesn’t seem like the memory leak persists. I guess I’ll have to check my environment again… I don’t believe my episodes are anywhere close to 10k steps long.
Thank you for your help.
---
### Author:  [@MrDracoG](https://discuss.ray.io/u/MrDracoG)
#### Post date: [December 24, 2022, 3:41am UTC](https://discuss.ray.io/t/memory-leak-when-training-ppo-on-a-single-agent-environment/8712/17 "2022-12-24T03:41:46Z")
Can confirm that I found a leak in my environment. Sorry for wasting the time. Thanks for the help!
