Restore agent and continue training with tune.run()

RLlib
1

Hi guys, I hope some basic questions are welcome as well.
After reading a bunch of guthub issues and community topics, I’m still struggling with the very basic task of restoring a previously trained agent and continuing the training.

I thought something like this should work:

trainer = ppo.PPOTrainer(env="CartPole-v0")
trainer.restore(path)
tune.run(trainer)

…but it gives me

ray.tune.error.TuneError: Improper 'run' - not string nor trainable.

which confuses me, because when I print the type of the trainer object it returns
<class ‘ray.rllib.agents.trainer_template.PPO’>

Can you, please point me to the correct way of doing it?

2

Hi @Vladimir_Uspenskii,

I think tune.run wants a callable function or a trainable type and not an instantiated object. This should work for you:

tune.run(
    "PPO", #You can replace this string with ppo.PPOTrainer if you want / have customized it
    name="RestoredExp", # The name can be different.
   # stop={""}, Your stopping criterion. This is cumulative so if it was N before it needs to be N+new value after restore
    restore=PATH, #This is the path to the checkpoint file not just the directory
    config={"env": "CartPole-v0"},
)
1 Like
3

@mannyv , oh yeah, it works this way! Thanks a lot!