Hello!
I have a Ray Cluster (version 2.2.0) in Kubernetes environment. We create Istio VirtualService
s for application endpoints instead of using direct internal service URLs.
For Ray client endpoint, I have following VirtualService
;
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: ray-cluster-vs
namespace: kuberay
spec:
hosts:
- "ray-cluster.<domain_name>"
gateways:
- istio-system/some-gateway
http:
- match:
- uri:
prefix: /
route:
- destination:
host: kuberay-head-svc.kuberay.svc.cluster.local
port:
number: 10001
With the above service, I expect to connect Ray Cluster as ray.init(address="https://ray-cluster.<domain_name>")
, but this line throws below error;
RuntimeError: Module: https does not exist.
This module was parsed from Address: https://ray-cluster.<domain_name>
I have another VirtualService
with exact same structure for port 8265 by which I can access dashboard. This makes me think that the virtual service structure is okay but somehow Ray expects ray://
protocol in address
variable to connect to the cluster. Just to test it, when I change the protocol, the mentioned error is gone but as the URL is wrong, it throws connection timeout error after a while.
PS: Connection as in ray.init(address="ray://kuberay-head-svc.kuberay:10001")
works fine.
Could anyone please suggest something to me here?