# Impala Deep Residual (Custom) Model

**URL:** <https://discuss.ray.io/t/impala-deep-residual-custom-model/8233>\
**Category:** RLlib\
**Created:** [November 11, 2022, 10:03am UTC](https://discuss.ray.io/t/impala-deep-residual-custom-model/8233 "2022-11-11T10:03:50Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jorgen\_Svane](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jorgen_svane/32/2929_2.png) [@Jorgen\_Svane](https://discuss.ray.io/u/Jorgen_Svane)\
**Post date:** [November 11, 2022, 10:03am UTC](https://discuss.ray.io/t/impala-deep-residual-custom-model/8233/1 "2022-11-11T10:03:50Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- Low: It annoys or frustrates me for a moment.

Ray: 2.1.0  
TensorFlow: 2.10.0

Hi

I’m trying to replicate the Impala Deep Residual Model from the paper but without the embedding part - right-hand side. See Image

![image](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/7/74c82cc99c013baf1d15fb62a10eba314752e7a9.png)

So far I’ve managed to forward the residual CNN part into the RNN and get that working in RandomEnv.

Code can be found here:  
[Impala\_Deep\_Residual\_Model](https://github.com/jlsvane/RLLIB-Impala/blob/main/basic_impala_deep_lstm_test.py)

However, like in the paper I would also like to include the previous action and reward into the LSTM block. I’ve had a close look into this part of the documentation (ViewRequirement) but appears to be unable to access the input\_dict in the forward\_rnn(…)

Can anyone help?

BR

Jorgen

---

<div class="post-metadata">

**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\
**Post date:** [November 11, 2022, 3:08pm UTC](https://discuss.ray.io/t/impala-deep-residual-custom-model/8233/2 "2022-11-11T15:08:18Z")

</div>

Hi @Jorgen_Svane,

The easiest way to do this in rllib will be to do all the layers up through the last relu in you custom models forward method. then concatenate that with r\_t-a and a\_t-1. Store that in input\_dict[“obs\_flat”]. Then call self.forward\_rnn(…).

---

<div class="post-metadata">

**Author:** ![Jorgen\_Svane](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jorgen_svane/32/2929_2.png) [@Jorgen\_Svane](https://discuss.ray.io/u/Jorgen_Svane)\
**Post date:** [November 23, 2022, 9:28pm UTC](https://discuss.ray.io/t/impala-deep-residual-custom-model/8233/3 "2022-11-23T21:28:02Z")

</div>

Hi @mannyv

Thanks for your replay. For now I opted for setting config = {… "use\_lstm:True, “lstm\_use\_prev\_action”: True, “lstm\_use\_prev\_reward”: True, …} and got it working [here](https://github.com/jlsvane/RLLIB-Impala/blob/main/lstm_wrapper_impala_action_reward.py).

I’ll probably revert to your solution later to get better control over the custom model.

BR

Jorgen