DD-PPO not working on Win10

RLlib
1

Hi all,

I’ve been trying to get the built in DD-PPO trainer working and run into the following issue:

  File "C:\Users\denys\AppData\Roaming\Python\Python38\site-packages\ray\thirdparty_files\psutil\_pswindows.py", line 681, in wrapper
    raise convert_oserror(err, pid=self.pid, name=self._name)
psutil.AccessDenied: psutil.AccessDenied (pid=72)

I have tried running the command in administrator CMD to no avail. Is there any fix for this?

2

@Denys_Ashikhin thanks for making this issue. Could you post the full stacktrace and repro on Github?

Thanks!

3

Sorry for the long delay, I had wanted to exhaust all the possible test cases. I did make open an issue at: DD-PPO not working on Win10 [rrlib] · Issue #15623 · ray-project/ray · GitHub

That being said, I have managed to get an almost identical setup working with an external env for my game with a PPO trainer but the synchronous overhead tends to bottleneck things too much (mainly because the server I have is upload bandwidth starved) so remote clients take too long to update. This is where I was hoping that DD-PPO would have less internet overhead and if it it’s the same, as long as the updating of values from the main process is non-blocking to the main actor playing the game, that would be perfect.

Please let me know if my hopes/understanding of DD-PPO here is flawed!

4

Actually, @Denys_Ashikhin , this is a known issue and unfortunately not captured with a proper error message. DD-PPO uses torch.distributed.init_process_group which throws an error on Win.
I’ll add a check to DD-PPO’s validate_config and at least provide a meaningful NotSupported error message.

5

This means that unfortunately the fix would have to wait until PyTorch fixes it on Windows right?

As a follow up, would it be enough to make a Linux VM on to run the policy_server, while the clients run windows (for DD-PPO)?

Also, I have had trouble getting APPO working as well - is that related to windows incompatibility or should I open an issue for that?