Hi,
I created a Ray cluster on AWS using this script. I can submit tasks to the cluster by logging on the head node. However, when I call ray.init from a Jupyter notebook in SageMaker, it times out.
There is a network connection between the notebook instance and the cluster node since it gave me a password mismatch error when I intentionally provided a bad Redis password.
Hey, ray.init() makes the assumption that it is being called from within the same VPC, therefore it requires essentially all ports on all workers to be accessible, which is naturally not a great security practice.
I’d recommend trying Ray Client. It’s still a beta feature, but it’s built for this exact purpose – running a ray driver from outside the cluster.
Hi Alex,
Thanks for the pointer. I was able to connect to the Ray cluster from a Jupyter notebook running on a different host. Remote calls went through fine.
Next, I tried to load a data frame with Modin:
import modin.pandas as pd
import numpy as np
frame_data = np.random.randint(0, 100, size=(2**10, 2**8))
df = pd.DataFrame(frame_data)
But, I am getting: “Exception: Trying to start two instances of ray via client”.
Modin is calling ray.init() internally. Is there a workaround for this?