Monitor surrogate loss for PPO

nethtrick · November 19, 2024, 2:47pm

Medium: It contributes to significant difficulty to complete my task, but I can work around it.

How can I monitor the surrogate loss for SGD within a step for PPO? For example, I want to see num_epochs loss values (num_epochs being the AlgorithmConfig setting) so I can verify that it is set large enough to converge at each step.

Thanks.

Topic		Replies	Views
~~Possible PPO surrogate policy loss sign error~~ RLlib	2	772	October 4, 2022
Breakdown of config and metrics of PPO implementation RLlib	0	643	February 23, 2022
Confusing behavior in PPO training loop (train_batch_size, sgd_minibatch_size, num_sgd_iter) RLlib	1	477	July 27, 2022
Entropy Regularization in PG? RLlib	9	839	September 17, 2022
Proper implement of reward scaling in PPO RLlib	0	325	December 17, 2020

Monitor surrogate loss for PPO

Related topics