# Can RLlib use GPU accelerator? **URL:** \ **Category:** RLlib\ **Created:** [November 29, 2021, 6:25am UTC](https://discuss.ray.io/t/can-rllib-use-gpu-accelerator/4281 "2021-11-29T06:25:33Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Roller44](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/roller44/32/1064_2.png) [@Roller44](https://discuss.ray.io/u/Roller44)\ **Post date:** [November 29, 2021, 6:25am UTC](https://discuss.ray.io/t/can-rllib-use-gpu-accelerator/4281/1 "2021-11-29T06:25:33Z")
I run RLlib programs using a PC with a GTX 1060 GPU and I have noticed the following line in the terminal: > Resources requested: 10.0/10 CPUs, 0.9999999999999999/1 GPUs, 0.0/18.28 GiB heap, 0.0/9.14 GiB objects (0.0/1.0 accelerator\_type:G) Apparently, the GPU’s accelerator is not used. My question is: how can I use the GPU’s accelerator? +++++++++++++++++++++++++++++++++++++++++++++++++++++ Edit: I use Tune and RLlib to find the optimal hyperparameters for an RL agent, and I set `num_gpus=1` to `ray.init()` and set `"num_gpus: 1"` in the config dict. When typing the command `nvidia-smi` in the terminal, I got: ```auto +-----------------------------------------------------------------------------+ | NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA GeForce ... On | 00000000:1C:00.0 Off | N/A | | 0% 53C P2 29W / 120W | 4889MiB / 6070MiB | 2% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 979 G /usr/lib/xorg/Xorg 146MiB | | 0 N/A N/A 1145 G /usr/bin/gnome-shell 5MiB | | 0 N/A N/A 1071629 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071630 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071631 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071632 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071633 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071634 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071635 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071636 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071637 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071638 C ray::DQN.train_buffered() 473MiB | +-----------------------------------------------------------------------------+ ``` Obviously, the RLlib program used the GPU to train the RL agent. However, I think the program did not adopt the GPU’s accelerator, because the terminal outputs > 0.0/18.28 GiB heap, 0.0/9.14 GiB objects (0.0/1.0 accelerator\_type:G) during training. My goal is to enable the RLlib program to adopt the GPU’s accelerator so that it can (hopefully) speed up the training. ---
**Author:** ![bmanczak](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/bmanczak/32/1802_2.png) [@bmanczak](https://discuss.ray.io/u/bmanczak)\ **Post date:** [November 29, 2021, 11:17am UTC](https://discuss.ray.io/t/can-rllib-use-gpu-accelerator/4281/2 "2021-11-29T11:17:40Z")
Did you specify `"num_gpus":1` in the `config` supplied to the trainer? ---
**Author:** ![Roller44](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/roller44/32/1064_2.png) [@Roller44](https://discuss.ray.io/u/Roller44)\ **Post date:** [November 29, 2021, 1:22pm UTC](https://discuss.ray.io/t/can-rllib-use-gpu-accelerator/4281/3 "2021-11-29T13:22:43Z")
@bmanczak Yes. I did. ---
**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\ **Post date:** [November 30, 2021, 2:05am UTC](https://discuss.ray.io/t/can-rllib-use-gpu-accelerator/4281/4 "2021-11-30T02:05:51Z")
Did you also add “num\_gpus=1” to ray.init? Have you checked that the tensor library is working correctly with gpu support? ---
**Author:** ![Roller44](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/roller44/32/1064_2.png) [@Roller44](https://discuss.ray.io/u/Roller44)\ **Post date:** [November 30, 2021, 2:22am UTC](https://discuss.ray.io/t/can-rllib-use-gpu-accelerator/4281/5 "2021-11-30T02:22:03Z")
@mannyv Yes, I also did that. ---
**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\ **Post date:** [November 30, 2021, 2:25am UTC](https://discuss.ray.io/t/can-rllib-use-gpu-accelerator/4281/6 "2021-11-30T02:25:44Z")
Does it start running? If so it looks like it is registering your request to use 1 gpu. Have you run nvidia-smi to see if the gpu is used during the run? ---
**Author:** ![Roller44](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/roller44/32/1064_2.png) [@Roller44](https://discuss.ray.io/u/Roller44)\ **Post date:** [November 30, 2021, 2:50am UTC](https://discuss.ray.io/t/can-rllib-use-gpu-accelerator/4281/7 "2021-11-30T02:50:08Z")
Yes. I didn’t clearly describe my situation in the post, sorry about that. I use Tune and RLlib to find the optimal hyperparameters for an RL agent, and I set `num_gpus=1` to `ray.init()` and set `"num_gpus: 1"` in the config dict. When training the RL agent, the terminal outputs the following: > Resources requested: 10.0/10 CPUs, 0.9999999999999999/1 GPUs, 0.0/18.28 GiB heap, 0.0/9.14 GiB objects (0.0/1.0 accelerator\_type:G) Furthermore, when typing the command `nvidia-smi` in the terminal, I got: ```auto +-----------------------------------------------------------------------------+ | NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA GeForce ... On | 00000000:1C:00.0 Off | N/A | | 0% 53C P2 29W / 120W | 4889MiB / 6070MiB | 2% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 979 G /usr/lib/xorg/Xorg 146MiB | | 0 N/A N/A 1145 G /usr/bin/gnome-shell 5MiB | | 0 N/A N/A 1071629 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071630 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071631 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071632 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071633 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071634 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071635 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071636 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071637 C ray::DQN.train_buffered() 473MiB | | 0 N/A N/A 1071638 C ray::DQN.train_buffered() 473MiB | +-----------------------------------------------------------------------------+ ``` Obviously, the RLlib program used the GPU to train the RL agent, but it did not adopt the GPU’s accelerator. My goal is to enable the RLlib program to adopt the GPU’s accelerator so that it can (hopefully) speed up the training. I will add the above info to the original post. ---
**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\ **Post date:** [November 30, 2021, 3:09am UTC](https://discuss.ray.io/t/can-rllib-use-gpu-accelerator/4281/8 "2021-11-30T03:09:22Z")
@Roller44, It is using the gpu as an accelerator. The accelerator\_type setting is a way of specifying what type of resource an actor or task should run on. For example let’s say you had 2 nodes with different types of gpus. You could use the accelerator type to say this needs to run only on the node that has accelerator\_type=x. [https://docs.ray.io/en/latest/advanced.html?highlight=accelerator\_type#accelerator-types](https://docs.ray.io/en/latest/advanced.html?highlight=accelerator_type#accelerator-types)