I have a Kubernetes setup for creating a ray cluster with kuberay. Still, I could not customize my yaml to add setup commands to be installed before the workers started to process the submitted jobs. I was able to use postStart field, but the workers start processing the jobs before the commands are executed. How I add setup commands, like was possible before the kuberay? I was not able to find any hint in the documentation ;(
@jhowpd This document may be useful: kuberay/pod-command.md at master · ray-project/kuberay · GitHub
@Kai-Hsun_Chen those workarounds are not unstable with auto-scaler? specially in larger clusters with thousands of jobs that might fail and restart the worker? No plans to introduce a field in the template, like was in the previous versions of ray with setup_commands?