# RLlib compatible with GNNs (e.g. TF-GNN, GraphTensor) or Spektral **URL:** \ **Category:** Configure Algorithm, Training, Evaluation, Scaling\ **Created:** [November 17, 2022, 11:43am UTC](https://discuss.ray.io/t/rllib-compatible-with-gnns-e-g-tf-gnn-graphtensor-or-spektral/8334 "2022-11-17T11:43:48Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![klausk55](https://avatars.discourse-cdn.com/v4/letter/k/f17d59/32.png) [@klausk55](https://discuss.ray.io/u/klausk55)\ **Post date:** [November 17, 2022, 11:43am UTC](https://discuss.ray.io/t/rllib-compatible-with-gnns-e-g-tf-gnn-graphtensor-or-spektral/8334/1 "2022-11-17T11:43:49Z")
Is RLlib directly compatible with GNN models (e.g. TF-GNN, GraphTensor)? I guess not, though a support for GraphTensors would be amazing and ease the use of GNNs in RLlib. Or does it support [Spektral](https://graphneural.network/)? ---
**Author:** ![kourosh](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kourosh/32/2130_2.png) [@kourosh](https://discuss.ray.io/u/kourosh)\ **Post date:** [January 4, 2023, 7:32am UTC](https://discuss.ray.io/t/rllib-compatible-with-gnns-e-g-tf-gnn-graphtensor-or-spektral/8334/2 "2023-01-04T07:32:15Z")
Hi @klausk55, We still don’t have native support for GNN frameworks in RLlib. I found some old threads that you were already part of. I am not sure if you will find them helpful, but these are the only solutions that we have found in the short term. > [@How to integrate GGNN model to agent policy models](https://discuss.ray.io/t/how-to-integrate-ggnn-model-to-agent-policy-models/5815): > > I am using the PPO trainer with a custom environment whose observation space is a graph, I am modeling the graph as a GGNN model. Node representations returned from GGNN are given input to the agent as observations. I want the GGNN model to get updated when agent networks are getting updated, how can I integrate GGNN with the agent model? ---
**Author:** ![klausk55](https://avatars.discourse-cdn.com/v4/letter/k/f17d59/32.png) [@klausk55](https://discuss.ray.io/u/klausk55)\ **Post date:** [January 4, 2023, 7:59am UTC](https://discuss.ray.io/t/rllib-compatible-with-gnns-e-g-tf-gnn-graphtensor-or-spektral/8334/3 "2023-01-04T07:59:54Z")
Hey @kourosh, I work on a workaround/solution integrating TF-GNN for that purpose. I’m confident to get it to work and if so, I hope I can do a PR later this months. ---
**Author:** ![kourosh](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kourosh/32/2130_2.png) [@kourosh](https://discuss.ray.io/u/kourosh)\ **Post date:** [January 4, 2023, 5:53pm UTC](https://discuss.ray.io/t/rllib-compatible-with-gnns-e-g-tf-gnn-graphtensor-or-spektral/8334/4 "2023-01-04T17:53:42Z")
That would be awesome. 😃 ---
**Author:** ![klausk55](https://avatars.discourse-cdn.com/v4/letter/k/f17d59/32.png) [@klausk55](https://discuss.ray.io/u/klausk55)\ **Post date:** [February 23, 2023, 4:38pm UTC](https://discuss.ray.io/t/rllib-compatible-with-gnns-e-g-tf-gnn-graphtensor-or-spektral/8334/5 "2023-02-23T16:38:45Z")
@kourosh I have implemented an example for GNN models based on TF-GNN. So far, it’s just a toy/dummy example but it shows a way how to integrate GNN models into the cosmos of RLlib. If this is of interest for you guys, let me know and then I could do a PR. ---
**Author:** ![kourosh](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kourosh/32/2130_2.png) [@kourosh](https://discuss.ray.io/u/kourosh)\ **Post date:** [February 24, 2023, 6:23am UTC](https://discuss.ray.io/t/rllib-compatible-with-gnns-e-g-tf-gnn-graphtensor-or-spektral/8334/6 "2023-02-24T06:23:36Z")
This is awesome to hear. Could make a PR and assign me to asses. We are working on a new API for models and policies that we could combine with your work to validate the support for GNNs with. ---
**Author:** ![klausk55](https://avatars.discourse-cdn.com/v4/letter/k/f17d59/32.png) [@klausk55](https://discuss.ray.io/u/klausk55)\ **Post date:** [February 24, 2023, 4:26pm UTC](https://discuss.ray.io/t/rllib-compatible-with-gnns-e-g-tf-gnn-graphtensor-or-spektral/8334/7 "2023-02-24T16:26:07Z")
@kourosh > > > \## Why are these changes needed? > > > There was no example/support for GNN model…s in RLlib. I've introduced a new \`Graph\` space (built upon \`Dict\` space) and added an example which shows how to set up an env with an obs in form of a graph. This dict-like obs stores the current state of a graph and gets converted into a \`GraphTensor\` (a composite tensor introduced in TF-GNN library). This \`GraphTensor\` then is the input for a custom GNN model based on the TF-GNN library. This library allows to build a wide variety of state-of-the-art GNNs. > The env and GNN in my example are for illustration purposes only (no deeper sense). More realistic examples might be small use cases of Shortest Path or Traveling Salesman Problem. > > The PR is related to my \[topic\](https://discuss.ray.io/t/rllib-compatible-with-gnns-e-g-tf-gnn-graphtensor-or-spektral/8334?u=klausk55) in the discuss forum. > > \## Checks > > \- \[\] I've signed off every commit(by using the -s flag, i.e., \`git commit -s\`) in this PR. > \- \[x\] I've run \`scripts/format.sh\` to lint the changes in this PR. > \- \[\] I've included any doc changes needed for https://docs.ray.io/en/master/. > \- \[\] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ > \- Testing Strategy > - \[\] Unit tests > - \[\] Release tests > - \[x\] This PR is not tested :(