Ray cluster worker nodes initialization issues

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I’m trying to launch a Ray cluster on GCP following this and I’ve specified the initialization_commands and setup_commands but they are not run on the worker nodes. I verified that by looking at “monitor.out” log file and it says “No initialization commands to run.”. What could be going wrong here?

I’m constantly modifying the cluster YAML file but always made sure to save the file before creating the cluster. Additionally, I have been using “–no-config-cache” option.

Additionally, I have two worker nodes to be set up and I sometimes can see one of them running the initialization commands while the other one doesn’t.