Multiple training phases with different configurations

I want to do some complicated training using RLlib and I’m not sure how.

I have an environment for two agents, and I want to train the first agent while I’m forcing the policy of the second agent to be a hard-coded policy that I write. I want to run that training for 10 steps. Then I want to continue training both agents normally for 10 more steps. That means that in the second training, the first agent is starting out with the policy that I trained in the first training phase, while the second agent is starting with a blank policy.

Is that possible with RLlib? How?

Thanks for your help,
Ram Rachum.

Hi @cool-RR, very interesting use-case.

I think yes, it’s possible; and here is a sketch of how using callbacks.

  • Consider having two policies. Let’s call them {‘main’, ‘hard-coded’}.
  • make main the only trainable policy.
  • using the on_train_result() hook of your callback you can check a criteria (e.g. self.iteration % 10 == 0) and do what ever you want with the algorithm and policy states. You can switch out the policy states, and let the new policy start from scratch. You can add a new policy. You can update the policy_mapping fn, etc. I recommend taking a look at to get ideas on how this can be done.

I hope it helps.

Hi Kourosh!

Thanks for your reply. This is a thread from mid-November, I already did something similar and it worked :slight_smile:

Thank you for helping anyway.