I see that it is possible to create a ray cluster in K8s which, I guess, would span over the nodes already in the k8s cluster so to make use of all the CPUs and GPUs. I guess ray will create a pod or set of pods in each machine? that said, I’m wondering, as I did not get it, how can I deploy a specific app to the ray cluster? I mean I might have a microservice or a script which needs to run in the ray cluster, and I would generally deploy on k86 as a service with say n pods. Do we have to do the same also with ray? The code in the deployed service will call ray.init(address of the cluster master) and the code will run in the ray nodes PLUS the pod with the service in it?
1 Like
The Ray autoscaler supports Kubernetes and should be the easiest way to deploy Ray applications. See the relevant documentation.
If you’re looking to deploy manually on Kubernetes, here is some documentation.
The documentation on how to execute Ray programs on K8s is here:
https://docs.ray.io/en/master/cluster/kubernetes.html#running-ray-programs
You can run a Ray program by executing it directly on the Ray head node:
https://docs.ray.io/en/master/cluster/kubernetes.html#running-a-program-using-kubectl-exec
Or, you can use a K8s Service to expose the head node’s ip and execute the program from a pod outside the cluster:
https://docs.ray.io/en/master/cluster/kubernetes.html#submitting-a-job
In the near future, we plan to establish Ray Client as the preferred way to run a Ray program on a Kubernetes cluster.
OK thanks, I’ll forward this to the devop. In one of our scenarios we need to receive an AMQP message and run some training. We could actually run the training script as one shot program on ray but we still need to have the consumer running somewhere so we might actually have a consumer in a pod and from that call ray I guess?
Yes, there are different ways to call use Ray. If the consumer is in a pod that is part of the Ray cluster, you can just use the Ray API like normal. If it is not part of the Ray cluster, you can use the Ray client API (see documentation for Ray client).