Now I know RaySGD is a wrapper built on top of pytorch. Quote: “”" The
TorchTrainer is a wrapper around
torch.distributed.launch with a Python API “”" (ray/raysgd_pytorch.rst at 9a93dd9682a216d2028db8edb60ff1485f653721 · ray-project/ray · GitHub)
I was trying to fully understand how RaySGD does the wrapping. Digged a bit into the code (trainer and worker_group part) but couldn’t tell for sure. ‘torch.distributed.launch’ is a script, without providing a python API interface. Does that mean RaySGD under the hood calls ‘torch.distributed.launch’ via shell script still, just with all the hard works taken care of ?
Guess in short I am curious where the entry point is, a pointer to the entry point of code is much appreciated.