Ppo add the lstm NN

[mannyv]

Hi @zzchuman,

You can add an automatic lstm with config["model"]["use_lstm"]=True

If by critic you mean the value network then yes,the action network and value network will both use the output from a single lstm layer.

You can have a look at the network architectures with and without an lstm here:

What is the intended architecture of PPO vf_share_layers=False when using an LSTM

I am wondering what the intended architecture should look like when using ppo with vf_share_layers = False and an lstm. Based on the comment in the ppo config and reading the documentation I would have thought that if it is set to false then there would be no sharing of the layers above the lstm. When I was looking at the LSTMWrapper code I realized that they would be sharing those layers even when vf_share_layers=False. Here is a picture of the network with use_lstm=False. To make it more cle…