Hi @Ofir_Abu ,
this manipulation is certainly not trivial. As you can see in the source code, the ReplayBuffer holds in _storage a list of SampleBatches that contain the experiences from the environment. This means, you either need to add (by using add() or remove SampleBatches directly from this buffer.