RNN support + RAM usage for RL algorithms

Dear developers,

I am AI researcher and very new to Ray, and it looks to me as a very diverse and powerful ML package. I’m looking for a working example of RL algorithm with support of RNN. Let’s say PPO+LSTM that solves Cart Pole, or anything like that. Will appreciate your help a lot.

And also, please, explain why Ray instances take so much RAM on my PC (Windows 11, Anaconda, no CUDA). Is it possible to reduce the memory usage in Ray instances?

Hi @qtomcat,

Here is an example with PPO / IMPALA in a stateless version of the cartpole environment using an LSTM.

many thanks, I will try it