# Distributed Training & Distributed Tuning using Ray Tune, PLT, Ray Lightning

**URL:** <https://discuss.ray.io/t/distributed-training-distributed-tuning-using-ray-tune-plt-ray-lightning/5411>\
**Category:** Ray Clusters\
**Created:** [March 15, 2022, 7:34pm UTC](https://discuss.ray.io/t/distributed-training-distributed-tuning-using-ray-tune-plt-ray-lightning/5411 "2022-03-15T19:34:27Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Manoj\_Kumar\_Dobbali](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/manoj_kumar_dobbali/32/2172_2.png) [@Manoj\_Kumar\_Dobbali](https://discuss.ray.io/u/Manoj_Kumar_Dobbali)\
**Post date:** [March 15, 2022, 7:34pm UTC](https://discuss.ray.io/t/distributed-training-distributed-tuning-using-ray-tune-plt-ray-lightning/5411/1 "2022-03-15T19:34:27Z")

</div>

Hi, I am using PyTorch Lightning(1.5.10), Ray Lightning(0.2) and Ray Tune(1.10.0) to distribute training and tuning. It is unclear to me on how to do resource allocation using RayPlugin from the documentation

Here([ray\_lightning/ray\_ddp.py at 65f497a3c8bedb2f24bf04a5dbf0ea62b5bcb4d6 · ray-project/ray\_lightning · GitHub](https://github.com/ray-project/ray_lightning/blob/65f497a3c8bedb2f24bf04a5dbf0ea62b5bcb4d6/ray_lightning/ray_ddp.py#L84)) it says, specify GPUs in Pytorch Lightning Trainer to a value \> 0

and Here([ray\_lightning/ray\_ddp.py at 65f497a3c8bedb2f24bf04a5dbf0ea62b5bcb4d6 · ray-project/ray\_lightning · GitHub](https://github.com/ray-project/ray_lightning/blob/65f497a3c8bedb2f24bf04a5dbf0ea62b5bcb4d6/ray_lightning/ray_ddp.py#L107)) in example it says NOT to specify resource in Trainer

Which one is right? I am able to run tuning experiments in parallel but unable to distribute training.I have 5 workers nodes with each 8 GPUs on each node

**How severe does this issue affect your experience of using Ray?**

- High: It blocks me to complete my task.

---

<div class="post-metadata">

**Author:** ![Ameer\_Haj\_Ali](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/ameer_haj_ali/32/279_2.png) [@Ameer\_Haj\_Ali](https://discuss.ray.io/u/Ameer_Haj_Ali)\
**Post date:** [April 25, 2022, 11:51am UTC](https://discuss.ray.io/t/distributed-training-distributed-tuning-using-ray-tune-plt-ray-lightning/5411/2 "2022-04-25T11:51:05Z")

</div>

referring to @xiaowei / @rliaw here.