PPO - Load checkpoint from previous version fails

RLlib
1

Hi,

I am trying to load a checkpoint of ppo_trainer created on v1.6 after upgrading to 1.10, but getting the following error:

AttributeError: Can't get attribute 'KLCoeffMixin' on <module 'ray.rllib.agents.ppo.ppo_torch_policy

I see that this class was deprecated with build_policy, but can’t think of how to solve it.
Is there any work around I can do to load this checkpoint successfully?

Thanks!

2

Ah, we are unable to support checkpointing across different versions of RLlib at the moment, but we’re working on a solution for this, and should be able to soon. In the meantime, you’ll need to uncheckpoint your policy using v1.6.

3

Thanks for you answer @avnishn. Hope you’ll be able to publish it soon.

To anyone else who needs a solution - since I only wish to test my agent, for now I’m saving only the weights and loading them in the new version.

# In the old version
weight = agent.get_policy().get_weights()
with open('my_weights.pickle', 'wb') as handle:
    pickle.dump(weight, handle, protocol=pickle.HIGHEST_PROTOCOL)

# In the new version
with open('my_weights.pickle', 'rb') as handle:
    weights = pickle.load(handle)
    agent.get_policy().set_weights(weights)