Load exported policy

RLlib
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Hello, I am training ten PPO policies using RLlib in a Multi-Agent setting. At the end of training, I would like to export the policies and then load them individually in their own objects. Is there a way to do this? For example, I would like to be able to run something like this:

# save the policies
agent.get_policy("1").export_model("policy1")

# in a separate process
# load one of the policies - load_model does not seem to exist
policy_one = PPOTorchPolicy.load_model("policy1")

# query for an action
policy_one.compute_single_action(...)
2

I guess one possibility would be to just save a checkpoint of the full model and then load it at each of the policy processes and extracting from there. But isn’t there a better alternative?