What'd be the best way to export ray cluster service to the internet?

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I am trying to launch a ray cluster on k8s and latest doc suggests helm. I managed to install a ray cluster using helm. However, I want to export the port of dashboard to the internet so that I don’t need to run port-forward every time. Then I modified the existing service configuration and then it worked. My first question is: what should I do if I want to add annotations to service dynamically to the helm chart so that I don’t have to add it after the cluster is created?

I also found 1.1.0 (Deploying on Kubernetes — Ray v1.1.0) allowed deploying from a yaml file directly. Will it be encourage to use for the latest version as well without installing crd?

Thanks for your time.

cc @Dmitri for helm chart and CRD question

Hi there – for now, you will have to modify the service by hand.

Making network configuration simple is on our minds – this will be smoother in the future.

thanks. I manually created a service to export the dashboard to internet and now I can see the UI in a URL like https://server.com/ray/dashboard, and I would like to do the same thing to the port 10001 so that I don’t need to run port-forward any time. And I tried ray.init(“ray://https://server.com/ray/conn”) but it failed to connect. Is there any way to work around it? thanks.

Could you share the connection error message?
Perhaps it’s generic, but still might be useful to see.

sure. I ran ray.init("ray://https://server.com/ray/conn/") and got ConnectionError: ray client connection timeout

/usr/local/lib/python3.7/site-packages/ray/util/client/worker.py in _connect_channel(self, reconnecting)
    262                     "more information."
    263                 )
--> 264             raise ConnectionError("ray client connection timeout")
    266     def _can_reconnect(self, e: grpc.RpcError) -> bool:

ConnectionError: ray client connection timeout

and https://server.com/ray/dashboard/ works. I used the same approach to export the service.
let me know if you need more info.

Hmm…are you able to expose a generic simple gRPC service using an ingress?
It is possible that the issue is somehow related to Ray client’s use of gRPC vs. the dashboard’s plain http.