Hi RLlib community,
I’m starting to work on robot locomotion environments similar to Genesis Go2 that use tensor-based computations (PyTorch tensors on GPU) internally for speed and efficiency.
In the past, I mostly used standard Gym environments with NumPy arrays, but now I’m considering switching to fully tensor-based custom environments.
Before diving in, I’d like to ask for suggestions
For RLlib, is it better to stick with traditional Gym-style environments outputting NumPy arrays?
Or is it recommended / supported to build custom environments that directly work with PyTorch tensors and leverage GPU acceleration?
What are the pros and cons of each approach in terms of performance and ease of integration?
Are there any best practices or examples of integrating tensor-based environments efficiently with RLlib?
I’m looking for guidance on which approach to choose to get the best training performance and smooth integration with RLlib.
Thanks in advance!