Ape-X DQN with custom first part net

RLlib
1

Hi there, and thanks for this beautiful RL library ! I would like to use a residual convolutional network with batch normalization (written using pytorch) as first part of the network in an Ape-X DQN model. I’ve seen in the doc and in the examples how to create and register a custom model.
However it is not clear whether my custom model will then be added a DQN head with dueling and other network related DQN advancement (if I’ve set them in the config) or if I have to implement them as part of my custom model.

1 Like
2

Great question @Jogima-cyber .
There are two options:

  • If your custom model is not a child of DQNTF|TorchModel, RLlib will automatically mix-in your class into the DQNTF|TorchModel class and then create an instance of that new “wrapped” class.
  • You can also directly sub-class DQNTF|TorchModel (override whatever methods you need to change, including the c’tor), in which case RLlib will not do the above auto-wrapping, but use your custom model directly (b/c it already satisfies the DQN API requirements).
1 Like
3

Thank you @sven1977 !

1 Like