# 'use\_lstm' with centralized critic for PPO

**URL:** <https://discuss.ray.io/t/use-lstm-with-centralized-critic-for-ppo/5659>\
**Category:** RLlib\
**Created:** [April 3, 2022, 3:09pm UTC](https://discuss.ray.io/t/use-lstm-with-centralized-critic-for-ppo/5659 "2022-04-03T15:09:29Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![hkml](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/hkml/32/2351_2.png) [@hkml](https://discuss.ray.io/u/hkml)\
**Post date:** [April 3, 2022, 3:09pm UTC](https://discuss.ray.io/t/use-lstm-with-centralized-critic-for-ppo/5659/1 "2022-04-03T15:09:29Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- High: It blocks me to complete my task.

Hello, I am implementing a centralized value function for PPO following the example for the [TwoStepGame](https://github.com/ray-project/ray/blob/releases/1.6.0/rllib/examples/centralized_critic.py) and I have managed to make the training work for it without LSTM, but if I set ‘use\_lstm’, I am receiving a size mismatch error:

```auto
(pid=14475) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
(pid=14475) return forward_call(*input, **kwargs)
(pid=14475) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/rnn.py", line 677, in forward
(pid=14475) self.check_forward_args(input, hx, batch_sizes)
(pid=14475) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/rnn.py", line 620, in check_forward_args
(pid=14475) self.check_input(input, batch_sizes)
(pid=14475) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/rnn.py", line 207, in check_input
(pid=14475) self.input_size, input.size(-1)))
(pid=14475) RuntimeError: input.size(-1) must be equal to input_size. Expected 63, got 276

```

If we override the PPO model as noted in the example i.e. if we have

```auto
CCPPOTorchPolicy = PPOTorchPolicy.with_updates(
    name="CCPPOTorchPolicy",
    postprocess_fn=centralized_critic_postprocessing,
    loss_fn=loss_with_central_critic,
    before_init=setup_torch_mixins,
    mixins=[
        TorchLR, TorchEntropyCoeffSchedule, TorchKLCoeffMixin,
        CentralizedValueMixin
    ])

```

Should we still expect the ‘use\_lstm’ to work out of the box, or would we need to accommodate the updated model. I have looked into the [wrapper’s code](https://github.com/ray-project/ray/blob/releases/1.6.0/rllib/models/torch/recurrent_net.py#L211), but I am lost since my model itself seems to work given that the training proceeds as expected without ‘use\_lstm’