I have been experimenting with RLLib recently, what i have noticed is a terrible code base with lots of bugs. Most of them stem from PyTorch integration.
Tensorflow is a matured production ready code. Why would anyone ditch that in favor of ever evolving buggy PyTorch. I dont mind PyTorch but should a framework like Ray not work nice with all of these great platforms?
I am thinking of asking Google to come up with a framework so we can replace Ray. It is not workable if there are too many buggy code base.
Hi @Sheshan_Kutty,
Sorry to hear you are having such troubles. The issues you are having are squarely in the RLlib code base as opposed to pytorch. The majority of the problems you and many others have had would still exist if you tensorflow was the backend instead. I just thought I would mention that because I don’t think that the attack on pytorch was really warranted.
RLLIB has a long history of changing frequently and it is in the middle of a major refractor, which I am not part of, so you should expect that to continue for some time. Offering that up in case it is relevant for your decision making.
Personally if I need something that is stable for a project and I want to use rllib and I want it to be stable then I pin it at version 2.2. That is not without issues though on the environment side because that is just before the gym->gymnasium switch.