Problem with handling states in RNN

[mannyv]

Hi @hossein836,

Try reading this post and see if it helps.

[rllib] SampleBatch "state_in_0" dimension shorter than expected

What is your max_seq_len? Before passing the model into forwad_rnn two things happen. 1. The data in the sample batch is padded so that all inputs are the same size as your longest episode. So if you had sequences of length [5,12,10,4] they would all be padded with zeros to be 12 steps long. You would end up with a total of12 * 4 timesteps in your sample batch. They are also shortened to be no larger than max sequence length. Let’s say your max_seq_len in the model config was 20. Your state in …