# \[SGD\] \[Tune\] How about the performance of RaySGD compared with pytorch DDP?

**URL:** <https://discuss.ray.io/t/sgd-tune-how-about-the-performance-of-raysgd-compared-with-pytorch-ddp/1351>\
**Category:** Ray Tune\
**Created:** [March 23, 2021, 2:59am UTC](https://discuss.ray.io/t/sgd-tune-how-about-the-performance-of-raysgd-compared-with-pytorch-ddp/1351 "2021-03-23T02:59:35Z")\
**Posts on this page:** 2\
**Page:** 2

<div class="post-metadata">

**Author:** ![rliaw](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/rliaw/32/24_2.png) [@rliaw](https://discuss.ray.io/u/rliaw)\
**Post date:** [April 20, 2021, 6:38pm UTC](https://discuss.ray.io/t/sgd-tune-how-about-the-performance-of-raysgd-compared-with-pytorch-ddp/1351/21 "2021-04-20T18:38:15Z")

</div>

Yeah, I think the main problem is like the optimizer that you create outside of the operator will have buffers that are based on CPU (rather than GPU).

---

<div class="post-metadata">

**Author:** ![Sheikh\_Nasrullah](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sheikh_nasrullah/32/875_2.png) [@Sheikh\_Nasrullah](https://discuss.ray.io/u/Sheikh_Nasrullah)\
**Post date:** [April 22, 2021, 11:55pm UTC](https://discuss.ray.io/t/sgd-tune-how-about-the-performance-of-raysgd-compared-with-pytorch-ddp/1351/22 "2021-04-22T23:55:53Z")

</div>

A comment !  
Shouldn’t we expect Ray SGD to have lower performance than torch.distributed as Ray incurs overhead (scheduling, updating system state, etc) ?

Is there any tool to measure the various metrics such as time spent on data transfer ?

[Previous page](https://discuss.ray.io/t/sgd-tune-how-about-the-performance-of-raysgd-compared-with-pytorch-ddp/1351.md?page=1)