# Ray 2.10.0: Can't find a \`node\_ip\_address.json\` file when using \_temp\_dir in ray.init()

**URL:** <https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262>\
**Category:** Uncategorized\
**Created:** [April 2, 2024, 8:54pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262 "2024-04-02T20:54:35Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![wxie2013](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wxie2013/32/3419_2.png) [@wxie2013](https://discuss.ray.io/u/wxie2013)\
**Post date:** [April 2, 2024, 8:54pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/1 "2024-04-02T20:54:35Z")

</div>

This happens in Ray.2.10.0. The following command runs fine:

```
 ray.init(address=ip_head, include_dashboard=False) 

```

but the following command:

```
 ray.init(address=ip_head,
        include_dashboard=False,
        _temp_dir='/scratch')

```

leads to the following errors:

```
 ValueError: Can't find a `node_ip_address.json` file from /scratch/session_2024-04-02_16-32-35_009347_3527526. for 60 seconds

```

I know the default is at the /tmp/ray of the running node, but it’s not convenient for me to access it to look into the details of the jobs running. I’m using slurm for the jobs submission.  
Any help is appreciated.

---

<div class="post-metadata">

**Author:** ![yic](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/yic/32/437_2.png) [@yic](https://discuss.ray.io/u/yic)\
**Post date:** [April 3, 2024, 7:29pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/2 "2024-04-03T19:29:05Z")

</div>

@wxie2013 `_temp_dir` is not a public API so you shouldn’t use it.

To specify the tmp dir into someplace other than `/tmp/ray` just use `ray start --head --temp-dir=/scratch` with start the head node.

---

<div class="post-metadata">

**Author:** ![wxie2013](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wxie2013/32/3419_2.png) [@wxie2013](https://discuss.ray.io/u/wxie2013)\
**Post date:** [April 3, 2024, 9:01pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/3 "2024-04-03T21:01:31Z")

</div>

Thanks @yic . Not quite following. The \_\_temp\_dir is an argument of ray.init(). Actually I’ve been using it for ray.train and it does redirect the output from /tmp to a specified directory. The issue only occurs when using ray.tune

---

<div class="post-metadata">

**Author:** ![yic](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/yic/32/437_2.png) [@yic](https://discuss.ray.io/u/yic)\
**Post date:** [April 3, 2024, 9:42pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/4 "2024-04-03T21:42:10Z")

</div>

Hi @wxie2013 everything started with `_` is a private API. We use it for testing or other purpose. Although you’ve been using it, it doesn’t mean it’s a public API and you are not supposed to use it. In the worst case, it can be deleted in the future.

To change the tmp dir, just provide `--temp-dir=` when you start the head node. This is the right way to setup the tmp dir.

Right now, all ray nodes are supposed to have the same tmp dir. If this doesn’t work for you, you can create a ticket and let me know. I’ll make sure it’s being tracked.

---

<div class="post-metadata">

**Author:** ![wxie2013](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wxie2013/32/3419_2.png) [@wxie2013](https://discuss.ray.io/u/wxie2013)\
**Post date:** [April 3, 2024, 9:47pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/5 "2024-04-03T21:47:30Z")

</div>

Thanks again@yic. I don’t use “ray start” command directly. I use ray.init() in a python script to start the ray process. For example, a python script named “example.py” :

```
 import ray
 ray.init() 

```

Then I run it via

```
 python example.py

```

Is there a way to specify the directory as an init argument?

---

<div class="post-metadata">

**Author:** ![yic](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/yic/32/437_2.png) [@yic](https://discuss.ray.io/u/yic)\
**Post date:** [April 3, 2024, 11:43pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/6 "2024-04-03T23:43:31Z")

</div>

This is not what we recommend to start ray worker nodes.

You can use this to start ray head node for single node usage.

If you really want to do this, make sure `_temp_dir` is the same in all `ray.init` (head and workers)

---

<div class="post-metadata">

**Author:** ![wxie2013](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wxie2013/32/3419_2.png) [@wxie2013](https://discuss.ray.io/u/wxie2013)\
**Post date:** [April 4, 2024, 1:11pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/7 "2024-04-04T13:11:46Z")

</div>

Thanks @yic for the prompt responses.

---

<div class="post-metadata">

**Author:** ![wxie2013](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wxie2013/32/3419_2.png) [@wxie2013](https://discuss.ray.io/u/wxie2013)\
**Post date:** [April 4, 2024, 3:56pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/8 "2024-04-04T15:56:03Z")

</div>

Hi @yic I did a try to use ray start --temp-dir=/scratch, the issue is that in the artifacts directory is missing, i.e. /scratch/ray/session\_latest/artifacts. When using the ray.init(\_\_temp\_dir=‘/scratch’), the artifacts directory does exist. Is there another argument to keep the artifacts in the redirected output when using ray start --temp-dir?

---

<div class="post-metadata">

**Author:** ![yic](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/yic/32/437_2.png) [@yic](https://discuss.ray.io/u/yic)\
**Post date:** [April 4, 2024, 10:00pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/9 "2024-04-04T22:00:32Z")

</div>

Hi @wxie2013 you can and only should setup tmp dir in the head node. The ray worker node will get the same dir as the head node.

---

<div class="post-metadata">

**Author:** ![wxie2013](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wxie2013/32/3419_2.png) [@wxie2013](https://discuss.ray.io/u/wxie2013)\
**Post date:** [April 4, 2024, 11:39pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/10 "2024-04-04T23:39:17Z")

</div>

Hi @yic. Yes I indeed used only head node. The issue is that artifacts directory is missing. I will need the content in that directory to do some detailed checks. By using ray.init(\_\_temp\_dir), the artifacts directory is there.

---

<div class="post-metadata">

**Author:** ![wxie2013](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wxie2013/32/3419_2.png) [@wxie2013](https://discuss.ray.io/u/wxie2013)\
**Post date:** [April 5, 2024, 1:29pm UTC](https://discuss.ray.io/t/ray-2-10-0-cant-find-a-node-ip-address-json-file-when-using-temp-dir-in-ray-init/14262/11 "2024-04-05T13:29:54Z")

</div>

Here are some more observations:

using the command “ray start --head --temp-dir=/scratch” will create a directory “/scratch” with “artifacts” missing. When running my model, it will create a different session at the default “/tmp/ray” which contains “artifacts” directory. This session is different from the one in the /scratch, i.e. there are two different sessions created, one before running the model, the other after running the model.