SAC with RNN/lstm

How severe does this issue affect your experience of using Ray?

  • None: Just asking a question out of curiosity

I am using SAC for my project and would benefit from the lstm=True option. If I understand correctly, this feature is not implemented for SAC. The RNNSAC is mentioned in some old topics but I cannot find it in the code or documentation so I assume it was removed. I am wondering whether RNN/lstm support is planned for SAC or other off-policy methods?

Hi @Elena, thanks for raising this question. RNNSAC/RNNDQN is planned and parts are already implemented in these two PRs:

What is missing, yet, is sequence sampling for prioritized sampling and rewriting RNN states into buffers. RIght now RNN for Offline RL has priority as well as a larger customer support topic. But this will be done. Feel free to fill in the missing parts for yourself based on what is there.