# Can you copy data between two node's CPU and GPU **URL:** \ **Category:** Kubernetes\ **Created:** [June 11, 2021, 10:43pm UTC](https://discuss.ray.io/t/can-you-copy-data-between-two-nodes-cpu-and-gpu/2496 "2021-06-11T22:43:11Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![valiantljk](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/valiantljk/32/252_2.png) [@valiantljk](https://discuss.ray.io/u/valiantljk)\ **Post date:** [June 11, 2021, 10:43pm UTC](https://discuss.ray.io/t/can-you-copy-data-between-two-nodes-cpu-and-gpu/2496/1 "2021-06-11T22:43:11Z")
Hi, I have a use case where one node in Ray cluster only has GPU resources left, no CPU available (occupied by other PODs outside Ray cluster), the question is, in case of distributed RaySGD training, can Ray leverage this isolated GPU? How is the data transfer between CPU and GPU handled by Ray? Can Ray copy the data from a remote CPU to this lonely GPU? ---
**Author:** ![Dmitri](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dmitri/32/657_2.png) [@Dmitri](https://discuss.ray.io/u/Dmitri)\ **Post date:** [June 25, 2021, 6:27pm UTC](https://discuss.ray.io/t/can-you-copy-data-between-two-nodes-cpu-and-gpu/2496/2 "2021-06-25T18:27:20Z")
@amogkam do you know about this? ---
**Author:** ![amogkam](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/amogkam/32/17_2.png) [@amogkam](https://discuss.ray.io/u/amogkam)\ **Post date:** [June 29, 2021, 9:03pm UTC](https://discuss.ray.io/t/can-you-copy-data-between-two-nodes-cpu-and-gpu/2496/3 "2021-06-29T21:03:09Z")
Thanks the question @valiantljk, this is an interesting use case. For the first question, if no CPUs are physically available then I don’t think you’d be able to execute any Python code. If it’s just that you don’t want Ray to reserve CPU for the training worker, you can achieve this by setting `num_cpus_per_worker` to 0 in your `TorchTrainer`. For the second question, you could use Ray to copy the data from a remote CPU to a local CPU, and then use some other library like torch to copy it to GPU. But Ray itself does not use GPU memory for the object store. ---
**Author:** ![valiantljk](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/valiantljk/32/252_2.png) [@valiantljk](https://discuss.ray.io/u/valiantljk)\ **Post date:** [June 29, 2021, 10:34pm UTC](https://discuss.ray.io/t/can-you-copy-data-between-two-nodes-cpu-and-gpu/2496/4 "2021-06-29T22:34:09Z")
Thanks @amogkam So, by default, how many resources does the torchtrainer reserve? ---
**Author:** ![valiantljk](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/valiantljk/32/252_2.png) [@valiantljk](https://discuss.ray.io/u/valiantljk)\ **Post date:** [July 9, 2021, 6:08pm UTC](https://discuss.ray.io/t/can-you-copy-data-between-two-nodes-cpu-and-gpu/2496/5 "2021-07-09T18:08:39Z")
@amogkam and is it possible to have 1 cpu to serve multiple GPU? ---
**Author:** ![amogkam](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/amogkam/32/17_2.png) [@amogkam](https://discuss.ray.io/u/amogkam)\ **Post date:** [July 12, 2021, 4:18pm UTC](https://discuss.ray.io/t/can-you-copy-data-between-two-nodes-cpu-and-gpu/2496/6 "2021-07-12T16:18:16Z")
Hey @valiantjik- by default `TorchTrainer` uses 1 CPU per worker, and if you are using GPUs, then 1 GPU per worker. But you can modify the CPUs per worker (even to fractional values) by setting `num_cpus_per_worker` Technically it is possible right now it to have 1 cpu with multiple GPUs if you pass in a fractional value to `num_cpus_per_worker`. But I haven’t tried this and it might not fully work. I completely agree though we want to add support for `num_gpus_per_worker` too.