# Ray multiprocessing with multi pytorch model inference

**URL:** <https://discuss.ray.io/t/ray-multiprocessing-with-multi-pytorch-model-inference/12490>\
**Category:** Ray Core\
**Created:** [October 18, 2023, 3:44am UTC](https://discuss.ray.io/t/ray-multiprocessing-with-multi-pytorch-model-inference/12490 "2023-10-18T03:44:39Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![QinlongHuang](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/qinlonghuang/32/5204_2.png) [@QinlongHuang](https://discuss.ray.io/u/QinlongHuang)\
**Post date:** [October 18, 2023, 3:44am UTC](https://discuss.ray.io/t/ray-multiprocessing-with-multi-pytorch-model-inference/12490/1 "2023-10-18T03:44:39Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- High: It blocks me to complete my task.

Hi, I got stuck when I use ray to do multiprocessing inference. To be exactly, I have 8 GPU, and I want to do some inference with multiprocessing to speed up my work. On each process, a model will be put on a single GPU, which means there are 8 processes and 8 models. It is normally when I use `concurrent.futures.ProcessPoolExecutor`, but when I use ray, only one GPU is activated. Here is a snippet of my code.

```python
@ray.remote(num_gpus=8)
def my_job(args):
    # do some jobs

def main():
    futures = [my_job.remote(j) for j in jobs]
    results = ray.get(futures)

if __name__ == " __main__":
    main()

```

---

<div class="post-metadata">

**Author:** ![rliaw](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/rliaw/32/24_2.png) [@rliaw](https://discuss.ray.io/u/rliaw)\
**Post date:** [October 18, 2023, 4:18am UTC](https://discuss.ray.io/t/ray-multiprocessing-with-multi-pytorch-model-inference/12490/2 "2023-10-18T04:18:01Z")

</div>

Try setting num\_gpus = 1. This means that each model will be put on 1 GPU.