# What is the recommended way to make use of a trained model?

**URL:** <https://discuss.ray.io/t/what-is-the-recommended-way-to-make-use-of-a-trained-model/4962>\
**Category:** RLlib\
**Created:** [February 7, 2022, 6:55pm UTC](https://discuss.ray.io/t/what-is-the-recommended-way-to-make-use-of-a-trained-model/4962 "2022-02-07T18:55:49Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jamuus](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jamuus/32/2100_2.png) [@jamuus](https://discuss.ray.io/u/jamuus)\
**Post date:** [February 7, 2022, 6:55pm UTC](https://discuss.ray.io/t/what-is-the-recommended-way-to-make-use-of-a-trained-model/4962/1 "2022-02-07T18:55:50Z")

</div>

Hey,

I’m training some models for a custom MuJoCo env that closely matches a robot I’ve built irl. I’m now looking at how to use that model outside of the training infrastructure(ideally).

I see that there’s `compute_actions` on the `ray.rllib.agents.trainer.Trainer` object, but that would require setting up the trainer object with all the config that goes with it.

I can get the `ray.rllib.policy.policy.Policy` and call `export_model`, I’m using SAC with torch and this successfully creates a TorchScript model. However it seems the `forward` function for SAC doesn’t actually do anything for an unrelated reason [https://github.com/ray-project/ray/blob/master/rllib/agents/sac/sac\_torch\_model.py#L140](https://github.com/ray-project/ray/blob/master/rllib/agents/sac/sac_torch_model.py#L140)

It looks like the TorchScript has the model in it but the forward func appears to be a noop

```auto
def forward(self,
    input_dict: Dict[str, Tensor],
    state: List[Tensor],
    seq_lens: Tensor) -> Tuple[Tensor, List[Tensor]]:
  _0, = state
  return (input_dict["obs"], [_0])

```

I haven’t had any success getting the TensorFlow version of SAC to converge and it seems to be slower than torch on my machine.

Is the only way to use the policy to recreate the training object from a checkpoint?

Cheers,  
James.

---

<div class="post-metadata">

**Author:** ![avnishn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/avnishn/32/1544_2.png) [@avnishn](https://discuss.ray.io/u/avnishn)\
**Post date:** [February 8, 2022, 1:27am UTC](https://discuss.ray.io/t/what-is-the-recommended-way-to-make-use-of-a-trained-model/4962/2 "2022-02-08T01:27:00Z")

</div>

> [@jamuus](#):
>
> I can get the `ray.rllib.policy.policy.Policy` and call `export_model` , I’m using SAC with torch and this successfully creates a TorchScript model. However it seems the `forward` function for SAC doesn’t actually do anything for an unrelated reason [https://github.com/ray-project/ray/blob/master/rllib/agents/sac/sac\_torch\_model.py#L140](https://github.com/ray-project/ray/blob/master/rllib/agents/sac/sac_torch_model.py#L140)

I think that you can do without needing the entire config that was used for training your SAC agent.  
In this example, we uncheckpoint a trained experiment, but only using the environment that we originally trained the agent on:

> <https://github.com/ray-project/ray/blob/ee1711fe418e44818ff791b266cd9dea42a65957/rllib/examples/sb2rllib_rllib_example.py#L33>

see if this solves your problem.

thanks,  
Avnish

---

<div class="post-metadata">

**Author:** ![jamuus](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jamuus/32/2100_2.png) [@jamuus](https://discuss.ray.io/u/jamuus)\
**Post date:** [February 8, 2022, 8:08pm UTC](https://discuss.ray.io/t/what-is-the-recommended-way-to-make-use-of-a-trained-model/4962/3 "2022-02-08T20:08:30Z")

</div>

Thanks for the quick reply.

It seems this example looks like it doesn’t need config because its mostly using defaults. I managed to get it working for my env and training checkpoint but I had to create a config to change the framework and model parameters before it was loaded successfully. I guess that makes sense but it would be nice if I didn’t have to worry about exporting model metadata separate from the files that somewhere inside probably contain the same info.

Cheers,  
James.