I’m looking to use a GNN to preprocess some information before sending it into a DQN network.
I was wondering how I best connect these layers to the model generated by RLLib.
I prefer on using the RLLib generated models, as they are tried and true.
I found the templates available for custom models (TorchModelV2), but they demand complete custom implementations. I also discovered that they build upon the FullyConnectedNetwork, but this seems the wrong approach to append layers.
Any insight would be greatly appreciated!
Hey @Vesyrak , the best way to customize the DQN model is to simply sub-class the existing ones and add new layers into these as required.
You can plug in your custom DQN model via:
custom_model: [your new sub-class of `DQNTorch|TFModel`]
[some c'tor args for your model class]
A simpler mechanism is maybe to use the already existing
hiddens config parameter of DQN. This allows you to specify layer sizes after(!) the Advantages(A)/Value(V)-split. Hence, each of the A- and V-branches will have this structure of Dense layers. To customize the NN before this A/V-split, use