How to integrate GGNN model to agent policy models

[Siddharth_Jain]

I am using the PPO trainer with a custom environment whose observation space is a graph, I am modeling the graph as a GGNN model. Node representations returned from GGNN are given input to the agent as observations. I want the GGNN model to get updated when agent networks are getting updated, how can I integrate GGNN with the agent model?

[gjoliver]

Thanks for the question Siddharth.

We started to work on a more native support for GNNs in RLlib recently.
At this point though, the best bet may be to just padding your observations to max number of nodes and features. and unpack them in your custom model.

You may find this thread interesting:

[RLlib] Variable-length Observation Spaces without padding

@alexanderb14 is your custom model a TF Keras model? I am interested in how you try to handle variable-sized inputs in your model? BTW: I think it is helpful when you label your question “rllib”