# Errors during training using BC with custom rnn

**URL:** <https://discuss.ray.io/t/errors-during-training-using-bc-with-custom-rnn/9652>\
**Category:** RLlib\
**Created:** [March 6, 2023, 8:37pm UTC](https://discuss.ray.io/t/errors-during-training-using-bc-with-custom-rnn/9652 "2023-03-06T20:37:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![R-Liebert](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/r-liebert/32/3991_2.png) [@R-Liebert](https://discuss.ray.io/u/R-Liebert)\
**Post date:** [March 6, 2023, 8:37pm UTC](https://discuss.ray.io/t/errors-during-training-using-bc-with-custom-rnn/9652/1 "2023-03-06T20:37:46Z")

</div>

Hi all!  
I’m trying out BC on some different gym env and have been encountered some issues I hoped someone could help me out with. I’ve been using a custom rnn from the rllib PPO example from [ncps.readthedocs.io](http://ncps.readthedocs.io) where the trajectories have been stored using SampleBatch as shown in the documentation. But I had to do add some of the keys due to errors, so the callback is updating also updating “eps\_id”, “agent\_index” and I had to set “state\_in\_0” to zeros of the size of my rnn.

As I understod from sampleBatch.py “eps\_id”, “agent\_index” and “state\_in\_0” should be set by rllib and I had no problem with when I didn’t use a custom model. Is this something that I have to set because of the custom om model, or because of the rnn? state\_in\_0 I assume is from the rnn, but it’s built on the RecurrentNetwork class so I’m a bit suprised that came up.

Where everything stops is during the training. In /rllib/policy/rnn\_sequencing.py I get a TypeError in line 289 as it says unroll\_ids is NoneType. Anyone who knows how this can be fixed?

Thanks,  
Robin

---

<div class="post-metadata">

**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\
**Post date:** [March 6, 2023, 9:48pm UTC](https://discuss.ray.io/t/errors-during-training-using-bc-with-custom-rnn/9652/2 "2023-03-06T21:48:44Z")

</div>

Hi @R-Liebert,

As far as I know BC does not support RNNs. This is from a long time ago so it may have changed but as far as I could tell it has. not.

> [@How to using self-definded tensor as padding observations for LSTM/Attention models](https://discuss.ray.io/t/how-to-using-self-definded-tensor-as-padding-observations-for-lstm-attention-models/2168/9):
>
> Also great catch on MARWIL! 🙂 We do say in the docs that it supports RNNs, but it’s not true (I’ll change that). The only off-policy RNN supporting algo afaik is currently R2D2.

---

<div class="post-metadata">

**Author:** ![R-Liebert](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/r-liebert/32/3991_2.png) [@R-Liebert](https://discuss.ray.io/u/R-Liebert)\
**Post date:** [March 7, 2023, 8:23am UTC](https://discuss.ray.io/t/errors-during-training-using-bc-with-custom-rnn/9652/3 "2023-03-07T08:23:28Z")

</div>

Thanks for the reply! Seems like it’s still unsupported as I ran the same sampleabatch on another algo without issues. As I’m still new to rllib it was a good exercise in getting to know the library 🙂

Edit: R2D2 has the same issues. So I’m going back to searching for errors as the docs states that rnn is supposed to work in BC aswell.