# RLLib PPO with centralized critic and LSTM (torch)

**URL:** <https://discuss.ray.io/t/rllib-ppo-with-centralized-critic-and-lstm-torch/4303>\
**Category:** RLlib\
**Created:** [November 30, 2021, 7:16pm UTC](https://discuss.ray.io/t/rllib-ppo-with-centralized-critic-and-lstm-torch/4303 "2021-11-30T19:16:57Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![rantav](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/rantav/32/1828_2.png) [@rantav](https://discuss.ray.io/u/rantav)\
**Post date:** [November 30, 2021, 7:16pm UTC](https://discuss.ray.io/t/rllib-ppo-with-centralized-critic-and-lstm-torch/4303/1 "2021-11-30T19:16:57Z")

</div>

I have seen the example of how to create [PPO with a centralized critic](https://github.com/ray-project/ray/blob/master/rllib/examples/centralized_critic.py) and it’s been useful, thanks!

I’m trying to adapt [this codebase](https://github.com/ray-project/ray/blob/ray-1.6.0/rllib/examples/models/centralized_critic_models.py#L96) using Torch to use an LSTM or GRU instead of just a feed-forward network and I’ve been unsuccessful at that. ☹  
Is there any such example or hint how to do use an LSTM as a central critic? (preferably with PPO)

Specifically I’m having trouble understanding how to properly set `seq_lens` but maybe I’ve just gone down the wrong rabbit hole and there are easier ways to do that.

Using RLLib 1.6

Thanks!