In the IMPALA trainer when the learner queue is full (I am using an expensive model so the GPU training is slow) the algorithm seems to move training to the CPU, making the training unbearably slow. The desired behavior would be that the CPUs go idle if the GPU is catching up. This effect can be pos…

[RLlib] Ray trains extremely slow when learner queue is full

smorad May 3, 2021, 12:59pm 8

Topic		Replies	Views
Very slow gradient descent on remote workers RLlib	14	2518	June 8, 2021
Impala does not respect GPU allocation RLlib	4	637	February 26, 2021
Impala Bugs and some other observations RLlib	9	1133	April 27, 2023
Impala seems inefficient (slow), how to properly initialize? RLlib	1	293	September 22, 2022
IMPALA agent not working RLlib	1	340	January 9, 2023