How to save weights in tensorflow's form?

RLlib
1

Hello everyone!

I am working on flow, a computational framework for reinforcement learning in traffic control, which is developed by Berkeley too. In the tutorial, i have already known that the trained neural network models’ parameter can be saved into a ray[rllib] form and rerun by the following code:

trials = run_experiments({
    flow_params["exp_tag"]: {
        "run": alg_run,
        "env": gym_name,
        "config": {
            **config
        },
        "restore": "/ray_results/experiment/dir/checkpoint_50/checkpoint-50"
        "checkpoint_freq": 1,
        "checkpoint_at_end": True,
        "max_failures": 999,
        "stop": {
            "training_iteration": 1,
        },
    },
})

By running this code, i can restart from a checkpoint and continue transfer learning.
However, i am wondering if i can save the weights in a tensorflow’s form so that i can use the model in other simulator (environment).

Thank you! :eyes:

2

Hi @gavin,

Welcome to the forums. I think calling either export_policy_model on the trainer or export_model model on the policy should do the trick for you. There is an option in those methods to get the model in onnx format. Here is the export_model method for tf.

3

Appreciate your answer! :star_struck: