I am experimenting with deployment of Ray Clusters onto a Kubernetes cluster, following the documents to startup an operator and then individual ray clusters.
The setup process is fine, and I have managed to get 3 separate Ray clusters happily co-existing in the same namespace and working as expected.
My question relates to the process of taking down a given Ray Cluster, as I have been getting irregular behaviour with resources not deleting.
What are the best order of commands to achieve this?
Just to be clear, my intention here is to remove a single cluster from the Kubernetes service, whilst keeping up the others.
I understand that to completely take down Ray, I need to uninstall the operator after I have uninstalled any running Ray cluster instances.
This is the current order in which I execute commands:
# Delete a load balancer service for access to head node on private VNET kubectl -n ray-clusters delete service ray1-cluster-head-access # Delete the custom resource for this cluster kubectl -n ray-clusters delete RayCluster ray1-cluster # Uninstall the helm chart for this cluster helm -n ray-clusters uninstall ray1-cluster