My prioritised replay buffer slows down my code massively

[mannyv]

@kourosh,

He resolved his issue in the thread linked below.

I think the issue ended up being race conditions due to remote calls in a loop. The fix was to use ray.get as a synchronization barrier.

I have a feeling there might be something more fundamental going on for the core team to investigate but I cannot point to something more specific unfortunately.

Code slows down when using prioritised replay buffer vs. vanilla replay buffer

I am using a distributed setup to train a DQN. When I used the following replay buffer, the code works perfectly fine: @ray.remote class ReplayBuffer: def __init__(self, capacity, batch_size=128): self.capacity = capacity self.buffer = [] self.batch_size = batch_size def push(self, data): for experience in data: self.buffer.append(experience) def sample(self): return random.sample(self.buffer, self.batch_size) def __len__(se…