Ray start command hangs when specifying the --temp-dir

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

When I use ray start comamnd with --temp-dir to specify the log dir, ray will hang like the following:

Usage stats collection is enabled. To disable this, add --disable-usage-stats to the command that starts the cluster, or run the following command: ray disable-usage-stats before starting the cluster. See Usage Stats Collection — Ray 3.0.0.dev0 for more details.

Local node IP: XX.XX.XX.XX

What is strange is that I cannot reproduce this issue every time. Yesterday I just found that when both specifing the --temp-dir and --using-dashboard=True, it will return error saying that the dashboard.log file doesn’t exist. While only specifying --temp-dir without --using-dashboard works.

Is there any way to debug the problem? I cannot see the log since I specify the new log dir for that and it is empty after I use ctrl c to stop the hanging ray start process.

Is there any update/recommended fix regarding the issue?

Hey @Eddie_Jin and @sniperyyc , could you provide the entire command that you are trying to run?

Are you trying to do something like ray start --head --include-dashboard=True --temp-dir=/tmp/foo ?

Hi @rickyyx ,
Thanks for reply.

My command is like

ray start --head --port=6379 --temp-dir=~/foo

or

ray start --head --port=6379 --include-dashboard=True --dashboard-host=0.0.0.0 --dashboard-port=8265 --temp-dir=~/foo

I’m just trying to specify the dir to the place under the $HOME which is better for me to keep that log.

  1. Without activating the dashboard, the cluster can be launched successfully yesterday by specifying the temp-dir but today it hangs.
  2. Although the cluster can be launched by specifying a new --temp-dir yesterday, after activating the dashboard, it returns error saying that the dashboard.log doesn’t exist.

@Eddie_Jin

I see, I was able to reproduce some error states with ray start --head --port=6379 --temp-dir=~/foo, although instead of hanging, it fails to launch the dashboard.

I think the code is not handling relative paths in the correct way. But I was able to get it working with absolute paths.

Could you expand your ~/foo into absolute path and see if that works?

Anyway, this is definitely some bugs IMO - thanks for raising it.

@rickyyx Changing to absolute path works! Thank you so much for your help!

@rickyyx It works! Thank you so much!