What'd be the best way to export ray cluster service to the internet?

Ray Clusters
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I am trying to launch a ray cluster on k8s and latest doc suggests helm. I managed to install a ray cluster using helm. However, I want to export the port of dashboard to the internet so that I don’t need to run port-forward every time. Then I modified the existing service configuration and then it worked. My first question is: what should I do if I want to add annotations to service dynamically to the helm chart so that I don’t have to add it after the cluster is created?

I also found 1.1.0 (Deploying on Kubernetes — Ray v1.1.0) allowed deploying from a yaml file directly. Will it be encourage to use for the latest version as well without installing crd?

Thanks for your time.

2

cc @Dmitri for helm chart and CRD question

3

Hi there – for now, you will have to modify the service by hand.

Making network configuration simple is on our minds – this will be smoother in the future.

4

thanks. I manually created a service to export the dashboard to internet and now I can see the UI in a URL like https://server.com/ray/dashboard, and I would like to do the same thing to the port 10001 so that I don’t need to run port-forward any time. And I tried ray.init(“ray://https://server.com/ray/conn”) but it failed to connect. Is there any way to work around it? thanks.

5

Could you share the connection error message?
Perhaps it’s generic, but still might be useful to see.

6

sure. I ran ray.init("ray://https://server.com/ray/conn/") and got ConnectionError: ray client connection timeout

/usr/local/lib/python3.7/site-packages/ray/util/client/worker.py in _connect_channel(self, reconnecting)
    262                     "more information."
    263                 )
--> 264             raise ConnectionError("ray client connection timeout")
    265 
    266     def _can_reconnect(self, e: grpc.RpcError) -> bool:

ConnectionError: ray client connection timeout

and https://server.com/ray/dashboard/ works. I used the same approach to export the service.
let me know if you need more info.

7

Hmm…are you able to expose a generic simple gRPC service using an ingress?
It is possible that the issue is somehow related to Ray client’s use of gRPC vs. the dashboard’s plain http.