Unable to save native torch model for a model implemented under ray.rllib.algorithms.*

RLlib
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Pickling of native torch model fails when saving for a ray.rllib.algorithms.* model implementation.
RLLIB error message: “”“Could not save torch model under self[TorchPolicy].model!
This is most likely due to the fact that you are using an Algorithm that
uses a Catalog-generated TorchModelV2 subclass, which is torch.save() cannot pickle.
“””
What is the recommended way of saving a native torch model for e.g. ray.rllib.algorithms.dqn.dqn_torch_model, which is not implemented under ray.rllib.models.* ?

2

I have a similar problem. Were you able to solve it?

3

No, I’ve used tensorflow native models instead.