CQL Algorithm Multi GPU torch not working

RLlib
1

How severe does this issue affect your experience of using Ray?*

  • High

Hi,

I am using ray version-2.3.0. I am trying to use the CQL algorithm and multi gpu is not working. I checked the repo and the release notes(release mentions that CQL multi-gpu support is enabled(https://github.com/ray-project/ray/pull/31466) for torch but the release does not actually have the changes.

I was able to manually make changes mentioned in the pull request. This introduces another problem when I use lagrangian on multiple gpus. Is there a fix coming up for these issues?

2

Hi @Varun_Jammula m

Thanks for pointing this point.
Multi GPU CQL does not work and we reverted the PR you reference iirc.
I made it work at one point put it became much slower with multiple GPUs because I had the mess with the loss.