Hi, I am at the beginning of my journey of Reinforcement Learning and I would like to know if it is possible to use DGL (Deep Graph Library) to define a custom model and integrate it with RLlib?
I don’t know if this is relevant by my use case is a multi-agent one, that needs the environment to be somehow defined as a homogeneous directed graph, hence the use of DGL.
I use pytorch_geometric and it works. I assume DGL handles graphs in a similar way. Basically you define a Box observation space and build the graph from it in the model. If you have different sized graphs use a Repeated space to handle variable number of edges/ node embeddings.
Thanks a lot for your answer. This is exactly what I have in mind. But I do not seem to be able to start with something. Do you mind sharing with me some docs, articles, or tutorials concerning this? Any material would be very helpful as I am just a beginner. I have searched a lot and I cannot seem to find something to start with which is combining GNN (with Pytorch Geometric/DGL) + RL.
I found this post, where someone creates the graph in the model init method:
I use the observation approach:
In the obs space you can use:
edges = Repeated(Box(low=0, high=N, shape=(2,), dtype=np.int64), max_len=max_edges)
obs_space = Dict({‘edges’:edges})
in the model use the unflattened observation
edges = input_dict[“obs”][‘edges’]
which is of type repeated_values:
Make sure your dimensions add up and don’t forget to cast the tensor if it’s needed. By default observations in the model are floating point numbers.
You can extend this for node embeddings
1 Like
This is awesome, I will look into this and try to start something. I will just mark the first answer as a solution as it is somehow answering my question and the example is clearing that out. Thanks a lot 