From my understanding, “fcnet_hiddens” in “model” corresponds to a neural network that approximates the Q function.
What “hidden” is used for?
great questions! You are correct with fcnet_hiddens that defines an array of layer sizes for the fully connected neural network that is used to approximate for example the Q-function values.
For the hiddens configuration setting take a look on the configuration of DQN:
# Dense-layer setup for each the advantage branch and the value branch
# in a dueling architecture.
"hiddens": [256],
The dueling architecture is presented in Wang et al. (2016) and has shown to perform better than simple DQN due to the latter overestimating action values. The DDQN architecture estimates two functions: (1) the value function and (2) the state-dependent action advantage function.
1 Like