# Using fractional GPU with TorchTrainer and Tuner API
**URL:** https://discuss.ray.io/t/using-fractional-gpu-with-torchtrainer-and-tuner-api/8598
**Category:** Uncategorized
**Created:** [December 8, 2022, 6:53pm UTC](https://discuss.ray.io/t/using-fractional-gpu-with-torchtrainer-and-tuner-api/8598 "2022-12-08T18:53:32Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@cupe](https://discuss.ray.io/u/cupe)
#### Post date: [December 8, 2022, 6:53pm UTC](https://discuss.ray.io/t/using-fractional-gpu-with-torchtrainer-and-tuner-api/8598/1 "2022-12-08T18:53:32Z")
Hi, I have a job using the TorchTrainer API, that looks something like this:
```auto
trainer = TorchTrainer(...)
tuner = Tuner(trainable=trainer, ...)
tuner.fit()
```
Everything is working as expected, including training with a single worker on GPU. However, I am using only a single GPU, and the models are relatively small such that several could fit on the same GPU. _I’d like to know how to allow the tuner to use fractional GPUs_, so that I can run multiple concurrent trials at once.
The docs [here](https://docs.ray.io/en/latest/tune/tutorials/tune-resources.html) seem to suggest wrapping the Trainer with `tune.with_resources`, but this doesn’t work with a Trainer, because Trainer doesn’t inherit from `Trainable`.
What’s the correct way to specify fractional GPU usage with the Tuner API and TorchTrainer (or Trainer more generally)?
---
### Author:  [@Yard1](https://discuss.ray.io/u/Yard1)
#### Post date: [December 8, 2022, 7:28pm UTC](https://discuss.ray.io/t/using-fractional-gpu-with-torchtrainer-and-tuner-api/8598/2 "2022-12-08T19:28:21Z")
You can specify resource requirements for a Trainer using a `ScalingConfig`: [Configurations User Guide — Ray 2.1.0](https://docs.ray.io/en/latest/train/config_guide.html#scaling-configuration-scalingconfig)
In your case, you’d do:
`TorchTrainer(..., scaling_config=ScalingConfig(resources_per_worker={"GPU": 0.5}))`.
---
### Author:  [@cupe](https://discuss.ray.io/u/cupe)
#### Post date: [December 20, 2022, 1:14am UTC](https://discuss.ray.io/t/using-fractional-gpu-with-torchtrainer-and-tuner-api/8598/3 "2022-12-20T01:14:29Z")
Just wanted to say: thank you! This worked 🙂
---
### Author:  [@Xinchengzelin](https://discuss.ray.io/u/Xinchengzelin)
#### Post date: [August 22, 2023, 9:03am UTC](https://discuss.ray.io/t/using-fractional-gpu-with-torchtrainer-and-tuner-api/8598/4 "2023-08-22T09:03:42Z")
@cupe Hi, did you use ` ray.train.torch.prepare_model` in the TorchTrainer? if not, could the fractional GPUs be used for multiple GPUs ?
Because [my code](https://discuss.ray.io/t/how-to-use-fraction-gpu-in-ray-tune-tuner/11863/2) show errors:
```auto
2023-08-21 09:01:33,024 ERROR trial_runner.py:993 -- Trial TorchTrainer_32216_00001: Error processing event.
ray.exceptions.RayTaskError(RuntimeError): ray::_Inner.train() (pid=1641548, ip=192.168.1.128, repr=TorchTrainer)
File "/home/xxx/miniconda3/lib/python3.8/site-packages/ray/tune/trainable/trainable.py", line 355, in train
raise skipped from exception_cause(skipped)
File "/home/xxx/miniconda3/lib/python3.8/site-packages/ray/train/_internal/utils.py", line 54, in check_for_failure
ray.get(object_ref)
ray.exceptions.RayTaskError(RuntimeError): ray::RayTrainWorker._RayTrainWorker__execute() (pid=1641581, ip=192.168.1.128, repr=)
File "/home/xxx/miniconda3/lib/python3.8/site-packages/ray/train/_internal/worker_group.py", line 31, in __execute
raise skipped from exception_cause(skipped)
File "/home/xxx/miniconda3/lib/python3.8/site-packages/ray/train/_internal/utils.py", line 129, in discard_return_wrapper
train_func(*args, **kwargs)
File "multipathpp_train.py", line 442, in train_func_per_worker
model = train.torch.prepare_model(model,parallel_strategy_kwargs={"find_unused_parameters":True})
File "/home/xxx/miniconda3/lib/python3.8/site-packages/ray/train/torch/train_loop_utils.py", line 120, in prepare_model
return get_accelerator(_TorchAccelerator).prepare_model(
File "/home/xxx/miniconda3/lib/python3.8/site-packages/ray/train/torch/train_loop_utils.py", line 365, in prepare_model
model = DataParallel(model, **parallel_strategy_kwargs)
File "/home/xxx/miniconda3/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 578, in __init__
dist._verify_model_across_ranks(self.process_group, parameters)
RuntimeError: NCCL error in: ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:957, invalid usage, NCCL version 21.0.3
ncclInvalidUsage: This usually reflects invalid usage of NCCL library (such as too many async ops, too many collectives at once, mixing streams in a group, etc).
Result for TorchTrainer_32216_00001:
date: 2023-08-21_09-01-20
experiment_id: 010ad717f5004de69f94685a292af065
hostname: xxx
node_ip: 192.168.1.128
pid: 1641548
timestamp: 1692579680
trial_id: '32216_00001'
```