Connecting to Ray Cluster through Istio VirtualService


I have a Ray Cluster (version 2.2.0) in Kubernetes environment. We create Istio VirtualServices for application endpoints instead of using direct internal service URLs.

For Ray client endpoint, I have following VirtualService;

kind: VirtualService
  name: ray-cluster-vs
  namespace: kuberay
    - "ray-cluster.<domain_name>"
    - istio-system/some-gateway
    - match:
        - uri:
            prefix: /
        - destination:
            host: kuberay-head-svc.kuberay.svc.cluster.local
              number: 10001

With the above service, I expect to connect Ray Cluster as ray.init(address="https://ray-cluster.<domain_name>"), but this line throws below error;

RuntimeError: Module: https does not exist.
This module was parsed from Address: https://ray-cluster.<domain_name>

I have another VirtualService with exact same structure for port 8265 by which I can access dashboard. This makes me think that the virtual service structure is okay but somehow Ray expects ray:// protocol in address variable to connect to the cluster. Just to test it, when I change the protocol, the mentioned error is gone but as the URL is wrong, it throws connection timeout error after a while.

PS: Connection as in ray.init(address="ray://kuberay-head-svc.kuberay:10001") works fine.

Could anyone please suggest something to me here?

@sercan ray.init(URL) according to docs only support ray:// URL and protocol syntax.

I don’t believe we support the https:// protocol

@Jules_Damji Thank you for your response. In this case I will try to fix the issue by trying to resolve the URL between VirtualService and Ray client.

I am not sure if it is good idea or not, but do you think Ray team would consider adding another parameter to ray.init() by which a different protocol could be specified when needed. If the parameter is empty, the protocol could be read from URL as it happens now.

Please consider my case; I still want to use ray:// protocol, but at the same time I need to connect to Ray cluster through Istio VirtualService which does not support ray:// protocol. If there was a protocol parameter, I could pass VirtualService URL as in https://ray-cluster.<domain_name> but at the same time I could specify protocol as ray://. This would help end users to use custom URLs without taking any other action.