# How to make checkpoint by ray.tune.run and load it?

**URL:** <https://discuss.ray.io/t/how-to-make-checkpoint-by-ray-tune-run-and-load-it/6680>\
**Category:** RLlib\
**Created:** [June 30, 2022, 4:37pm UTC](https://discuss.ray.io/t/how-to-make-checkpoint-by-ray-tune-run-and-load-it/6680 "2022-06-30T16:37:01Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Peter\_Pirog](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/peter_pirog/32/234_2.png) [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)\
**Post date:** [June 30, 2022, 4:37pm UTC](https://discuss.ray.io/t/how-to-make-checkpoint-by-ray-tune-run-and-load-it/6680/1 "2022-06-30T16:37:01Z")

</div>

What is the best way to :

1. make checkpoint by ray.tune.run
2. load checkpoint by PPOTrainer
3. Show checkpoint by tensorflow model.summary ?

I try to make checkpoint by :

```auto
from ray import tune

tune.run("PPO",
         config={
             "env":"CartPole-v0",
             "framework": "tf2",
             "evaluation_interval":2,
             "evaluation_duration":20,
         },
         local_dir="cartpole",
         checkpoint_freq=2)

```

next, load model by:

```auto
from ray.rllib.agents.ppo.ppo import PPOTrainer
import gym

config = {
    "env": "CartPole-v0",
    "framework": "tf2",
    "evaluation_interval": 2,
    "evaluation_duration": 20,
}

agent = PPOTrainer(config=config)
agent.restore("./cartpole/PPO/PPO_CartPole-v0_23a86_00000_0_2022-06-30_16-24-39/checkpoint_000002/checkpoint-2")
agent.load_checkpoint()

env = gym.make("CartPole-v0")
obs = env.reset()
while True:
    action = agent.compute_action(obs)
    obs, reward, done, _ = env.step(action)
    if done:
        break
env.close()

```

but there is error:

```auto

 File "/home/ppirog/projects/Mastering-Reinforcement-Learning-with-Python/3D_observations/run_saved_checkpoint.py", line 13, in <module>
    agent.load_checkpoint()
TypeError: load_checkpoint() missing 1 required positional argument: 'checkpoint_path'
Exception ignored in: <function RolloutWorker. __del__ at 0x7fd2f9785e50>
Traceback (most recent call last):
  File "/home/ppirog/projects/Mastering-Reinforcement-Learning-with-Python/venv/lib/python3.8/site-packages/ray/util/tracing/tracing_helper.py", line 461, in _resume_span
TypeError: 'NoneType' object is not callable
Exception ignored in: <function RolloutWorker. __del__ at 0x7fd2f9772a60>
Traceback (most recent call last):
  File "/home/ppirog/projects/Mastering-Reinforcement-Learning-with-Python/venv/lib/python3.8/site-packages/ray/util/tracing/tracing_helper.py", line 461, in _resume_span
TypeError: 'NoneType' object is not callable
Process finished with exit code 1

```

At last i try to show model:

```auto
import tensorflow as tf
model_path="./cartpole/PPO/PPO_CartPole-v0_23a86_00000_0_2022-06-30_16-24-39/checkpoint_000002/"
new_model = tf.keras.models.load_model(model_path)

new_model.summary()

```

But there is error:

```auto
OSError: SavedModel file does not exist at: /home/ppirog/projects/Mastering-Reinforcement-Learning-with-Python/3D_observations/cartpole/PPO/PPO_CartPole-v0_2890c_00000_0_2022-06-30_16-03-19/checkpoint_000002/{saved_model.pbtxt|saved_model.pb}

```

---

<div class="post-metadata">

**Author:** ![xwjiang2010](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/xwjiang2010/32/1476_2.png) [@xwjiang2010](https://discuss.ray.io/u/xwjiang2010)\
**Post date:** [July 5, 2022, 5:08pm UTC](https://discuss.ray.io/t/how-to-make-checkpoint-by-ray-tune-run-and-load-it/6680/2 "2022-07-05T17:08:10Z")

</div>

@Peter_Pirog  
For loading a checkpoint with RLlib agent, please refer to this example: [Training APIs — Ray 1.13.0](https://docs.ray.io/en/latest/rllib/rllib-training.html)  
Specifically,

 ![Screen Shot 2022-07-05 at 10.06.53 AM](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/9/9523dd93de1b2c73a15833a6407be5f2ea158643.png)

Let me know how it goes!

---

<div class="post-metadata">

**Author:** ![Peter\_Pirog](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/peter_pirog/32/234_2.png) [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)\
**Post date:** [July 5, 2022, 6:27pm UTC](https://discuss.ray.io/t/how-to-make-checkpoint-by-ray-tune-run-and-load-it/6680/3 "2022-07-05T18:27:28Z")

</div>

@xwjiang2010 Thank You, I will try it 🙂

---

<div class="post-metadata">

**Author:** ![Peter\_Pirog](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/peter_pirog/32/234_2.png) [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)\
**Post date:** [July 7, 2022, 7:10pm UTC](https://discuss.ray.io/t/how-to-make-checkpoint-by-ray-tune-run-and-load-it/6680/4 "2022-07-07T19:10:37Z")

</div>

@xwjiang2010 , I tested the code it works fine 🙂

Here is the code how to visualize the trained model:

```auto
import ray.rllib.agents.ppo as ppo
from tensorflow.keras.utils import plot_model

checkpoint_path=" " # <-in this place put the path to saved checkpoint
config={ } # <- in this place put configuration the same like during training process

trainer = ppo.PPOTrainer(config=config)
trainer.restore(checkpoint_path)
model=trainer.get_policy().model.base_model # this is typical tf2 model format 
# typical tensorflow visualization
model.summary()
plot_model(model, to_file='model_plot.png', show_shapes=True, show_layer_names=True,rankdir="TB",expand_nested=False,show_layer_activations=True)

```

Result is:

```auto
Model: "model"
__________________________________________________________________________________________________
 Layer (type) Output Shape Param # Connected to                     
==================================================================================================
 observations (InputLayer) [(None, 4)] 0 []                               
                                                                                                  
 fc_1 (Dense) (None, 256) 1280 ['observations[0][0]']           
                                                                                                  
 fc_value_1 (Dense) (None, 256) 1280 ['observations[0][0]']           
                                                                                                  
 fc_2 (Dense) (None, 256) 65792 ['fc_1[0][0]']                   
                                                                                                  
 fc_value_2 (Dense) (None, 256) 65792 ['fc_value_1[0][0]']             
                                                                                                  
 fc_out (Dense) (None, 2) 514 ['fc_2[0][0]']                   
                                                                                                  
 value_out (Dense) (None, 1) 257 ['fc_value_2[0][0]']             
                                                                                                  
==================================================================================================
Total params: 134,915
Trainable params: 134,915
Non-trainable params: 0

```

 ![model_plot](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/3/39f55b855747c211f97d1d69c48b45161f24c984.png)