Is a TorchPolicy or the DL model in this policy automatically set to evaluation mode (model.eval()) during the default evaluation phase?

For example, I have a model that includes torch.nn.Dropout layers that behave differently during training and evaluation. In Torch, I need to use model.train() or model.eval() to set which mode the model should be in. Is this also done by RLlib if I use a custom_eval_fn or the default routine?

Hi @LukasNothhelfer,

The Torch trainer switches between eval and train modes in

compute_actions

and

learn_on_batch:

If you are using the standard way to customize, for example using “with_updates” then it should be switched. You could add break points or print statements there if you want to be extra sure.

3 Likes

+1 for sharing code references, thx