I have a Ray Cluster (version 2.2.0) in Kubernetes environment. We create Istio
VirtualServices for application endpoints instead of using direct internal service URLs.
For Ray client endpoint, I have following
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: ray-cluster-vs namespace: kuberay spec: hosts: - "ray-cluster.<domain_name>" gateways: - istio-system/some-gateway http: - match: - uri: prefix: / route: - destination: host: kuberay-head-svc.kuberay.svc.cluster.local port: number: 10001
With the above service, I expect to connect Ray Cluster as
ray.init(address="https://ray-cluster.<domain_name>"), but this line throws below error;
RuntimeError: Module: https does not exist. This module was parsed from Address: https://ray-cluster.<domain_name>
I have another
VirtualService with exact same structure for port 8265 by which I can access dashboard. This makes me think that the virtual service structure is okay but somehow Ray expects
ray:// protocol in
address variable to connect to the cluster. Just to test it, when I change the protocol, the mentioned error is gone but as the URL is wrong, it throws connection timeout error after a while.
PS: Connection as in
ray.init(address="ray://kuberay-head-svc.kuberay:10001") works fine.
Could anyone please suggest something to me here?