Massive RL parallelization by End-to-End GPU-accelerated learning

RLlib
1

How severe does this issue affect your experience of using Ray?

  • None: Just asking a question out of curiosity

Inspired by Isaac Gym and by Brax capability to obtain massive speed factor through parallelization. by running both the environment and the inference on the GPU,
I would also like to know if such a computing model is possible on RLLIB.

A few months ago I asked on Slack and someone told me this feature would require writing a GPUSampleCollectorclass derived from the current SampleCollector class,
and a GPUVectorEnv class derived from the VectorEnv, that works on the GPU and produces GPU outputs (possibly storing all data in PyTorch GPU tensors).

There was some interest, but I heard nothing in the last months.
Is anybody interested in this subject?

2

We are actually all interested in this subject! We’ve also been discussing this internally but cannot commit to a timeline right now because of higher-priority items, sorry.

If you are interested in implementing this, or anybody else, please open a PR and we’ll try to support you on your way.

Cheers