Breaking changes for TorchPolicyV2?

RLlib
1

How severe does this issue affect your experience of using Ray?

  • None: Just asking a question out of curiosity
  • Low: It annoys or frustrates me for a moment.
  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.
  • High: It blocks me to complete my task.

PPOTorchPolicy in the latest master (starting from Ray 2.0, I assume) inherits TorchPolicyV2, which has some methods missing that TorchPolicy had. For me the one that breaks my custom class that inherits PPOTorchPolicy is the extra_grad_info method. Was that removed in purpose?

I checked the migration doc, but it doesn’t mention this change.

2

  • We don’t use extra_grad_info anymore, but rather stats_fn. I’m sorry there is no deprecation warning - it’s just part of the V2 API.

  • It’s not part of the guide because it is tagged as DeveloperAPI and we have excluded many smaller changes that are not part of our public API.

3

Ah, yes, it’s stats_fn now. Thanks!