How severe does this issue affect your experience of using Ray?
- High: It blocks me to complete my task.
When I use ray start comamnd with --temp-dir to specify the log dir, ray will hang like the following:
Usage stats collection is enabled. To disable this, add --disable-usage-stats
to the command that starts the cluster, or run the following command: ray disable-usage-stats
before starting the cluster. See Usage Stats Collection — Ray 3.0.0.dev0 for more details.
Local node IP: XX.XX.XX.XX
What is strange is that I cannot reproduce this issue every time. Yesterday I just found that when both specifing the --temp-dir and --using-dashboard=True, it will return error saying that the dashboard.log file doesn’t exist. While only specifying --temp-dir without --using-dashboard works.
Is there any way to debug the problem? I cannot see the log since I specify the new log dir for that and it is empty after I use ctrl c to stop the hanging ray start process.
Is there any update/recommended fix regarding the issue?
Hey @Eddie_Jin and @sniperyyc , could you provide the entire command that you are trying to run?
Are you trying to do something like ray start --head --include-dashboard=True --temp-dir=/tmp/foo
?
Hi @rickyyx ,
Thanks for reply.
My command is like
ray start --head --port=6379 --temp-dir=~/foo
or
ray start --head --port=6379 --include-dashboard=True --dashboard-host=0.0.0.0 --dashboard-port=8265 --temp-dir=~/foo
I’m just trying to specify the dir to the place under the $HOME which is better for me to keep that log.
- Without activating the dashboard, the cluster can be launched successfully yesterday by specifying the temp-dir but today it hangs.
- Although the cluster can be launched by specifying a new --temp-dir yesterday, after activating the dashboard, it returns error saying that the dashboard.log doesn’t exist.
@Eddie_Jin
I see, I was able to reproduce some error states with ray start --head --port=6379 --temp-dir=~/foo
, although instead of hanging, it fails to launch the dashboard.
I think the code is not handling relative paths in the correct way. But I was able to get it working with absolute paths.
Could you expand your ~/foo
into absolute path and see if that works?
Anyway, this is definitely some bugs IMO - thanks for raising it.
@rickyyx Changing to absolute path works! Thank you so much for your help!
@rickyyx It works! Thank you so much!