- Medium: It contributes to significant difficulty to complete my task, but I can work around it.
I am running ray cluster using
cluster.yaml file. In a cluster I have 3 local worker nodes. Now, my project pipeline has different path in all three worker node. So, how can I set different working directory for each worker node?
- I don’t want to send files from head node to worker node. When I submit job, head node should run the command in specific working directory in worker node.
Hi @shyampatel , unfortunately Ray doesn’t support this, please feel free to file a feature request on the Ray Github if this is important to you. There is ongoing work to support specifying an existing working directory without sending files between nodes, but we don’t have plans to specify a different directory for each node.
Is this functionality added in nightly/beta version?
This functionality isn’t available in the nightlies yet unfortunately. If you’re only interested in doing Python imports from the directory, one workaround is to use the
env_vars field of
runtime_env to set the environment variable
PYTHONPATH to the desired directory.