# When to use multi gpus per worker for a training job
**URL:** https://discuss.ray.io/t/when-to-use-multi-gpus-per-worker-for-a-training-job/15805
**Category:** Uncategorized
**Created:** [September 14, 2024, 5:11am UTC](https://discuss.ray.io/t/when-to-use-multi-gpus-per-worker-for-a-training-job/15805 "2024-09-14T05:11:56Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@HCharlie](https://discuss.ray.io/u/HCharlie)
#### Post date: [September 14, 2024, 5:11am UTC](https://discuss.ray.io/t/when-to-use-multi-gpus-per-worker-for-a-training-job/15805/1 "2024-09-14T05:11:56Z")
Hello everyone,
I would like to know when should I use multi GPUs per worker for a ray training job by specifying the scaling\_config to be something like
`ScalingConfig(num_workers=2, use_gpu=True, resources_per_worker={"GPU":4})`
So far I haven’t found any working examples using this setup yet, only seeing the [document](https://docs.ray.io/en/latest/train/user-guides/using-gpus.html#assigning-multiple-gpus-to-a-worker) saying it’s possible to do so.
Also this raises a question in my mind, would the code be significantly different from the one with only one GPU per worker? Is there really any benefit using multi GPUs per worker for a ray training job?
For launching a distributed training job in ray with 8 GPUs, which scaling config is recommended, and when should multi gpu per worker scaling config should be used?
```auto
ScalingConfig(num_workers=2, use_gpu=True, resources_per_worker={"GPU":4})
```
```auto
ScalingConfig(num_workers=8, use_gpu=True, resources_per_worker={"GPU":1})
```
Take the [pytorch fashion mnist code](https://docs.ray.io/en/latest/train/examples/pytorch/torch_fashion_mnist_example.html) for example, how should I modify this in order to fully utilize the 8 GPUs with a multi gpu per worker scaling config like this? Is this a good practice?
```auto
ScalingConfig(num_workers=2, use_gpu=True, resources_per_worker={"GPU":4})
```
---
### Author:  [@eric](https://discuss.ray.io/u/eric)
#### Post date: [September 15, 2024, 4:28am UTC](https://discuss.ray.io/t/when-to-use-multi-gpus-per-worker-for-a-training-job/15805/2 "2024-09-15T04:28:59Z")
For a default training setup torch will only use one gpu per process. This is more of a torch DDP question than a ray one.
For 99% of use-cases stir away from manually specifying `resources_per_worker` for training jobs, just set `num_workers` to the number of gpus in your cluster.