LSTM Auto Wrapper

[mannyv]

1. max _seq_len handles truncating back prop through time for the lstm. So the lstm will get inputs of size [B//4,4,f_in_size]

I dont think it will affect the curiosity module but I have not verified that.

2. It is the second. Take a look at the second diagram in this thread.

What is the intended architecture of PPO vf_share_layers=False when using an LSTM

I am wondering what the intended architecture should look like when using ppo with vf_share_layers = False and an lstm. Based on the comment in the ppo config and reading the documentation I would have thought that if it is set to false then there would be no sharing of the layers above the lstm. When I was looking at the LSTMWrapper code I realized that they would be sharing those layers even when vf_share_layers=False. Here is a picture of the network with use_lstm=False. To make it more cle…