How severe does this issue affect your experience of using Ray?
- None: Just asking a question out of curiosity
- Low: It annoys or frustrates me for a moment.
- Medium: It contributes to significant difficulty to complete my task, but I can work around it.
- High: It blocks me to complete my task.
High. I’m unable to proceed with my hyperparameter tuning due to a low steps per second.
I have computer with 1 gpu and 1 cpu with 8 logical processors (aka 8 cpus) and used to train my models using stablebaselines3 and would be able to train as fast as 1000 steps per second using the synchronous DummyVecEnv with 16 workers. I just recently migrated to ray and rllib recently and am having trouble reaching higher than 100 steps per second. I have read the documentation over and over and attempted to have multiple workers, multiple envs, fractional gpus, local trainer and/or local worker. Please let me know how I can speed up the training of my models, I heard ray and rllib are optimized for speed so I’m hoping it’s just a config error on my part. Thank for helping.
Note: I have been training using PPO and the default model