Actor/Critic model settings for PPO

RLlib
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

It looks like RLlib allows altering model settings (hidden layers, activation, etc.) for both actor and critic separately in DDPG, TD3, and SAC, but I’m not seeing this as an option for PPO. Is this not implemented?

For example, I might want to train using PPO such that my actor has 2 hidden layers of size 128, and the critic has 2 hidden layers of size 512. In the config options, I see a setting for not sharing the layers between the actor and critic, but I don’t see any way to change the architecture of each.

Thanks in advance.

2

This is also implemented. You can use the same settings in PPO.
Please have at the appropriate section in our docs!