PER Buffer throws KeyError during training of SAC

[araman5]

@araman5

• Expected: I am training an SAC agent on a custom environment (as shown in the code snippet below). I have put the algo.train() in a for loop for 5000 epochs of training.

Hi,

Yes, I shall first test it out with the latest version of Ray (If there has been an update since when I posted this) before raising an issue in github/ray-project/ray