# Error when running on GPU

**URL:** <https://discuss.ray.io/t/error-when-running-on-gpu/2725>\
**Category:** RLlib\
**Created:** [July 1, 2021, 2:00am UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725 "2021-07-01T02:00:53Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![ironv](https://avatars.discourse-cdn.com/v4/letter/i/8e8cbc/32.png) [@ironv](https://discuss.ray.io/u/ironv)\
**Post date:** [July 1, 2021, 2:00am UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725/1 "2021-07-01T02:00:53Z")

</div>

I am trying to run the following code on a VM with 2 GPUs. (ray 1.4 / python 3.7.3 / torch 1.8.1)

```auto
import logging
import ray

import ray.rllib.agents.ppo as ppo

import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0,1"

_ = ray.init(ignore_reinit_error=True)

config = ppo.DEFAULT_CONFIG.copy()
config["num_gpus"] = 1
config["num_workers"] = 2
config["framework"] = "torch"

trainer = ppo.PPOTrainer(config=config, env="CartPole-v0")

```

The last part of the stack trace is

```auto
~/.conda/envs/py373_cuda/lib/python3.7/site-packages/ray/rllib/policy/torch_policy.py in __init__ (self, observation_space, action_space, config, model, loss, action_distribution_class, action_sampler_fn, action_distribution_fn, max_seq_len, get_batch_divisibility_req)
    153 for i, id_ in enumerate(gpu_ids) if i < config["num_gpus"]
    154 ]
--> 155 self.device = self.devices[0]
    156 ids = [
    157 id_ for i, id_ in enumerate(gpu_ids) if i < config["num_gpus"]

IndexError: list index out of range

```

This works fine when I set `num_gpus = 0` . Also this code works just fine on GPUs  
[Trials placed on the same GPU on a 2 GPU machine despite "num\_gpus": 1](https://discuss.ray.io/t/trials-placed-on-the-same-gpu-on-a-2-gpu-machine-despite-num-gpus-1/1508)

---

<div class="post-metadata">

**Author:** ![ironv](https://avatars.discourse-cdn.com/v4/letter/i/8e8cbc/32.png) [@ironv](https://discuss.ray.io/u/ironv)\
**Post date:** [July 1, 2021, 3:11am UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725/2 "2021-07-01T03:11:47Z")

</div>

Looking at `torch_policy.py` line 956 has `gpu_ids = ray.get_gpu_ids()`. When I run ray.get\_gpu\_ids() I get an empty list.

---

<div class="post-metadata">

**Author:** ![michaelzhiluo](https://avatars.discourse-cdn.com/v4/letter/m/41988e/32.png) [@michaelzhiluo](https://discuss.ray.io/u/michaelzhiluo)\
**Post date:** [July 1, 2021, 7:17am UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725/3 "2021-07-01T07:17:53Z")

</div>

Check out this git issue: [[rllib][regression] num\_gpus=1 on a device with GPU uses CPU for Pytorch in 0.8.7 · Issue #10271 · ray-project/ray · GitHub](https://github.com/ray-project/ray/issues/10271)

Also, check if torch/tensorflow is detecting gpus!

---

<div class="post-metadata">

**Author:** ![ironv](https://avatars.discourse-cdn.com/v4/letter/i/8e8cbc/32.png) [@ironv](https://discuss.ray.io/u/ironv)\
**Post date:** [July 1, 2021, 1:26pm UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725/4 "2021-07-01T13:26:37Z")

</div>

`torch.cuda.is_available()` returns `True`.

And I am running Ray v1.4. The relevant snippet of code (when compared to the github fix)

```auto
        if config["_fake_gpus"] or config["num_gpus"] == 0 or \
                not torch.cuda.is_available():

```

I am not setting num\_gpus or \_fake\_gpus in the config and `not is_available` will be False as well. So, the control is passed to the else part of that block where it is failing.

---

<div class="post-metadata">

**Author:** ![Vladimir\_Uspenskii](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/vladimir_uspenskii/32/1233_2.png) [@Vladimir\_Uspenskii](https://discuss.ray.io/u/Vladimir_Uspenskii)\
**Post date:** [July 1, 2021, 8:50pm UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725/5 "2021-07-01T20:50:19Z")

</div>

I got the same error. This code:

```auto
import ray
import ray.rllib.agents.ppo as ppo
import torch

print(torch.cuda.is_available())
ray.init()
config = ppo.DEFAULT_CONFIG
config["num_gpus"] = 1
config["framework"] = "torch"
trainer = ppo.PPOTrainer(config=config, env="CartPole-v0")

```

returns “True” for cuda and this at the end of the the Traceback:

```auto
 File "/home/bukovskiy/.local/lib/python3.8/site-packages/ray/rllib/policy/torch_policy.py", line 159, in __init__
    self.device = self.devices[0]
IndexError: list index out of range

```

referring to these lines in torch\_policy.py:

```auto
self.devices = [
                torch.device("cuda:{}".format(i))
                for i, id_ in enumerate(gpu_ids) if i < config["num_gpus"]
            ]
            self.device = self.devices[0]

```

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [July 28, 2021, 7:34pm UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725/6 "2021-07-28T19:34:43Z")

</div>

Yes, we should have a better error message for this case:

- torch.cuda.is\_available() returns True, but
- ray.get\_gpu\_ids returns an empty list

Not sure what the solution here is, though. Could this be a ray core issue not detecting the GPUs on some machines?

---

<div class="post-metadata">

**Author:** ![Guy\_Tennenholtz](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/guy_tennenholtz/32/1431_2.png) [@Guy\_Tennenholtz](https://discuss.ray.io/u/Guy_Tennenholtz)\
**Post date:** [August 20, 2021, 11:42am UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725/7 "2021-08-20T11:42:54Z")

</div>

I had the same error and fixed it by locally copying the rllib repository to my project.

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [August 23, 2021, 7:17am UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725/8 "2021-08-23T07:17:55Z")

</div>

Hey everyone (@ironv , @Guy_Tennenholtz , @Vladimir_Uspenskii , @michaelzhiluo 🙂 ), thanks for this discussion and surfacing these issues.  
We did make lots of improvements on the multi-GPU/GPU frontier recently and a lot of these bugs should be fixed by now in the current master.  
We also deployed nightly 2-GPU learning tests for all major algos and both tf and torch. We’ll add LSTM=True 2-GPU tests for all RNN-supporting algos in the next 1-2 weeks as well.

---

<div class="post-metadata">

**Author:** ![yz\_x](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/yz_x/32/1505_2.png) [@yz\_x](https://discuss.ray.io/u/yz_x)\
**Post date:** [September 24, 2021, 8:32am UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725/9 "2021-09-24T08:32:08Z")

</div>

I also encountered this problem. I don’t know how to solve it. Have you solved this problem?

---

<div class="post-metadata">

**Author:** ![GoingMyWay](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/goingmyway/32/1102_2.png) [@GoingMyWay](https://discuss.ray.io/u/GoingMyWay)\
**Post date:** [February 23, 2022, 3:13am UTC](https://discuss.ray.io/t/error-when-running-on-gpu/2725/10 "2022-02-23T03:13:23Z")

</div>

Same issue. Anyone can help?

> <https://github.com/ray-project/ray/issues/1221#issuecomment-1048411022>
>
> \<!--
> General questions should be asked on the mailing list ray-dev@googlegroups….com.
> 
> Before submitting an issue, please fill out the following form.
> \--\>
> 
> \### System information
> \- \*\*OS Platform and Distribution (e.g., Linux Ubuntu 16.04)\*\*: Ubuntu 16.04
> \- \*\*Ray installed from (source or binary)\*\*: binary
> \- \*\*Ray version\*\*: 0.2.1
> \- \*\*Python version\*\*: 2.7.12
> \- \*\*Exact command to reproduce\*\*:
> 
> \<!--
> You can obtain the Ray version with
> 
> python -c "import ray; print(ray.\_\_version\_\_)"
> \--\>
> 
> \### Describe the problem
> \`ray.get\_gpu\_ids()\` crashes when \`ray.init\` was called with \`PYTHON\_MODE\`.

With PyTorch on machines with one GPU, it cannot run.

I think `ray.get_gpu_ids()` failed to get the ids of GPU. `ray.get_gpu_ids()` is everywhere in RLlib and now it seems I cannot use RLlib with Pytorch.