Using Ray clusters as a k8s microservice without port forwarding

  • High: It blocks me to complete my task.


We are building an ML platform and trying to use Ray Clusters to execute jobs from other microservices. As far as I can tell, the best way to use kuberay and the clusters as part of the microservice architecture is to submit jobs using a forwarded port, which doesn’t work with the overall k8s architecture, since the pods need to be scaleable and not dependent on specific forwarded ports. What is recommended here? We would love to use ray as our core backend for the ml jobs, but having this job intermediary requirement prevents this. I am relatively new to Ray so would definitely appreciate any advice here! Hopefully I am missing something with ray or k8s in general!


Following up –

This link may solve my problem, it seems like there is an automatic setup for k8s nodes to interact that works here.