Using Ray clusters as a k8s microservice without port forwarding

Ray Clusters Kubernetes
1
  • High: It blocks me to complete my task.

Hello

We are building an ML platform and trying to use Ray Clusters to execute jobs from other microservices. As far as I can tell, the best way to use kuberay and the clusters as part of the microservice architecture is to submit jobs using a forwarded port, which doesn’t work with the overall k8s architecture, since the pods need to be scaleable and not dependent on specific forwarded ports. What is recommended here? We would love to use ray as our core backend for the ml jobs, but having this job intermediary requirement prevents this. I am relatively new to Ray so would definitely appreciate any advice here! Hopefully I am missing something with ray or k8s in general!

Thanks

2

Following up –

This link may solve my problem, it seems like there is an automatic setup for k8s nodes to interact that works here.

https://docs.ray.io/en/latest/cluster/kubernetes/user-guides/config.html#kuberay-networking