Multi agent partial parameter sharing

RLlib
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I know it’s possible to implement parameter sharing with multi-agent by making the policy_mapping_fn map all agents to the same policy id. However, what if I want all my policies to share the network except say the last layer, how could this be done with RLlib (using TF)?

2

Hey @Fady_Ben ,

This is one of the major reasons we have created a new stack for RLlib for model composition and training that we have just released in alpha mode. This is exactly one of the advanced use-cases of that API. Have you checked it out?

https://docs.ray.io/en/latest/rllib/rllib-rlmodule.html#writing-custom-multi-agent-rl-modules-advanced

3 Likes
3

@kourosh Thanks looks like this is what I need. Is this compatible with the GTrXLNet as a shared encoder (or any RNN in general)?