# GNN (DGL) with Multi-Agent Reinforcement Learning **URL:** \ **Category:** RLlib\ **Created:** [January 4, 2022, 12:08am UTC](https://discuss.ray.io/t/gnn-dgl-with-multi-agent-reinforcement-learning/4567 "2022-01-04T00:08:38Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![mbe19](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mbe19/32/1942_2.png) [@mbe19](https://discuss.ray.io/u/mbe19)\ **Post date:** [January 4, 2022, 12:08am UTC](https://discuss.ray.io/t/gnn-dgl-with-multi-agent-reinforcement-learning/4567/1 "2022-01-04T00:08:38Z")
Hi, I am at the beginning of my journey of Reinforcement Learning and I would like to know if it is possible to use DGL (Deep Graph Library) to define a custom model and integrate it with RLlib? I don’t know if this is relevant by my use case is a multi-agent one, that needs the environment to be somehow defined as a homogeneous directed graph, hence the use of DGL. ---
**Author:** ![Sertingolix](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sertingolix/32/649_2.png) [@Sertingolix](https://discuss.ray.io/u/Sertingolix)\ **Post date:** [January 5, 2022, 7:14am UTC](https://discuss.ray.io/t/gnn-dgl-with-multi-agent-reinforcement-learning/4567/2 "2022-01-05T07:14:06Z")
I use pytorch\_geometric and it works. I assume DGL handles graphs in a similar way. Basically you define a Box observation space and build the graph from it in the model. If you have different sized graphs use a Repeated space to handle variable number of edges/ node embeddings. ---
**Author:** ![mbe19](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mbe19/32/1942_2.png) [@mbe19](https://discuss.ray.io/u/mbe19)\ **Post date:** [January 5, 2022, 8:30am UTC](https://discuss.ray.io/t/gnn-dgl-with-multi-agent-reinforcement-learning/4567/3 "2022-01-05T08:30:23Z")
Thanks a lot for your answer. This is exactly what I have in mind. But I do not seem to be able to start with something. Do you mind sharing with me some docs, articles, or tutorials concerning this? Any material would be very helpful as I am just a beginner. I have searched a lot and I cannot seem to find something to start with which is combining GNN (with Pytorch Geometric/DGL) + RL. ---
**Author:** ![Sertingolix](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sertingolix/32/649_2.png) [@Sertingolix](https://discuss.ray.io/u/Sertingolix)\ **Post date:** [January 5, 2022, 9:03am UTC](https://discuss.ray.io/t/gnn-dgl-with-multi-agent-reinforcement-learning/4567/4 "2022-01-05T09:03:19Z")
I found this post, where someone creates the graph in the model init method: > [@Pytorch Geometric in RLLib?](https://discuss.ray.io/t/pytorch-geometric-in-rllib/3079/3): > > I use torch geometric in ray rllib here: [graph-conv-memory-paper/ray\_graph.py at master · smorad/graph-conv-memory-paper · GitHub](https://github.com/smorad/graph-conv-memory-paper/blob/master/src/models/ray_graph.py) I use the observation approach: In the obs space you can use: edges = Repeated(Box(low=0, high=N, shape=(2,), dtype=np.int64), max\_len=max\_edges) obs\_space = Dict({‘edges’:edges}) in the model use the unflattened observation edges = input\_dict[“obs”][‘edges’] which is of type repeated\_values: > Make sure your dimensions add up and don’t forget to cast the tensor if it’s needed. By default observations in the model are floating point numbers. You can extend this for node embeddings ---
**Author:** ![mbe19](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mbe19/32/1942_2.png) [@mbe19](https://discuss.ray.io/u/mbe19)\ **Post date:** [January 5, 2022, 9:28am UTC](https://discuss.ray.io/t/gnn-dgl-with-multi-agent-reinforcement-learning/4567/5 "2022-01-05T09:28:52Z")
This is awesome, I will look into this and try to start something. I will just mark the first answer as a solution as it is somehow answering my question and the example is clearing that out. Thanks a lot 😃