Run DD-PPO in multiple GPUs

Hi all!

I am trying to run the DD-PPO algorithm, providing 1 GPU/worker, using this configuration file:
I have a machine with 4 GPUs, thus I set num_workers: 4.

The problem is, that all 4 workers use 1 GPU, instead of using 1 each.
When I print the resources for the workers, I get the following output for each of them:

{‘CPU_group_05d7394261ee785f26194bc7e08bb664’: [(0, 1.0)], ‘GPU_group_05d7394261ee785f26194bc7e08bb664’: [(0, 1.0)]}

I understand that Ray then sets CUDA_VISIBLE_DEVICES=0 for all workers, and this might be why they all use GPU 0.

Why is the GPU_group showing always GPU 0, for each worker? Is this expected behavior?


Yes, this is expected behavior. It is the models that get distributed across GPUs.

1 Like

Then, how to distribute multiple workers to multiple GPUs?