I was trying to follow documentation Deploying on Kubernetes — Ray v2.0.0.dev0.
Created Ray cluster on Kind cluster locally. Used both - ray up and operator approaches. Operator works great, for ray up it mostly works, but once in a while, it does not process ns creation and consequently fails. This behavior seems to be random.
After cluster is created, Using Ray Client to connect from within the Kubernetes cluster works fine. I can look at the log and see the execution.
But when I tried Using Ray Client to connect from outside the Kubernetes cluster connection always fails. I see that
Handling connection for 10001, but the result is:
Traceback (most recent call last):
File “/Users/boris/Projects/RayOnKind/src/localPython.py”, line 55, in
File “/usr/local/lib/python3.7/site-packages/ray/util/client_connect.py”, line 26, in connect
conn_str, secure=secure, metadata=metadata, connection_retries=3)
File “/usr/local/lib/python3.7/site-packages/ray/util/client/init.py”, line 57, in connect
File “/usr/local/lib/python3.7/site-packages/ray/util/client/worker.py”, line 120, in init
raise ConnectionError(“ray client connection timeout”)
ConnectionError: ray client connection timeout
To ensure Python version I am using image rayproject/ray:b3a717-py37-cpu.
which is the same as my current: