# Best way to save policy

**URL:** https://discuss.ray.io/t/best-way-to-save-policy/2013
**Category:** RLlib
**Created:** [May 2, 2021, 3:09pm UTC](https://discuss.ray.io/t/best-way-to-save-policy/2013 "2021-05-02T15:09:50Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Rory](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/rory/32/688_2.png) [@Rory](https://discuss.ray.io/u/Rory)
#### Post date: [May 2, 2021, 3:09pm UTC](https://discuss.ray.io/t/best-way-to-save-policy/2013/1 "2021-05-02T15:09:50Z")

</div>

Hi there,

I’d like to be able to save a policy (not a checkpoint) that can be loaded into a trainer later on.

I have one policy which is trained over say 10000 training iterations, and I’d like to save it every 100 to be able to run experiments on later (like in the self play paper: [http://arxiv.org/abs/2006.04471](http://arxiv.org/abs/2006.04471)). Having to load a new checkpoint each time is too slow, and the files are pretty large and come with all the other policies in the multiagent config.

I saw the policy.export\_model method which creates a torchscript file but I’m not sure that’s what I want.

I guess maybe a better way is to pickle the output of policy.get\_state(), then have one trainer which is initialized and use policy.set\_state() with the loaded state for the experiments.

---

<div class="post-metadata">

### Author: ![smorad](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/smorad/32/272_2.png) [@smorad](https://discuss.ray.io/u/smorad)
#### Post date: [May 3, 2021, 1:12pm UTC](https://discuss.ray.io/t/best-way-to-save-policy/2013/2 "2021-05-03T13:12:25Z")

</div>

You can do something like:

```auto
    train = ImpalaTrainer(...)
    # Do some training
    ...
    torch.save(train.get_policy().model, model_path)
    ...
    train.restore(model_path)

```

---

<div class="post-metadata">

### Author: ![spring](https://avatars.discourse-cdn.com/v4/letter/s/ee59a6/32.png) [@spring](https://discuss.ray.io/u/spring)
#### Post date: [August 26, 2021, 1:57am UTC](https://discuss.ray.io/t/best-way-to-save-policy/2013/3 "2021-08-26T01:57:20Z")

</div>

Traceback (most recent call last):  
File “attention\_net.py”, line 227, in   
torch.save(trainer.get\_policy().model, model\_path)  
File “D:\Anaconda3\envs\tensorflow-gpu\lib\site-packages\torch\serialization.p  
y”, line 372, in save  
\_save(obj, opened\_zipfile, pickle\_module, pickle\_protocol)  
File “D:\Anaconda3\envs\tensorflow-gpu\lib\site-packages\torch\serialization.p  
y”, line 476, in \_save  
pickler.dump(obj)  
\_pickle.PicklingError: Can’t pickle \<class ‘ray.rllib.models.catalog.FullyConnec  
tedNetwork\_as\_AttentionWrapper’\>: attribute lookup FullyConnectedNetwork\_as\_Atte  
ntionWrapper on ray.rllib.models.catalog failed