# Possible reasons for ray data stucks at write\_csv (or write\_parquet)?

**URL:** <https://discuss.ray.io/t/possible-reasons-for-ray-data-stucks-at-write-csv-or-write-parquet/11412>\
**Category:** Uncategorized\
**Created:** [July 14, 2023, 1:53pm UTC](https://discuss.ray.io/t/possible-reasons-for-ray-data-stucks-at-write-csv-or-write-parquet/11412 "2023-07-14T13:53:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![xiszishu](https://avatars.discourse-cdn.com/v4/letter/x/f0a364/32.png) [@xiszishu](https://discuss.ray.io/u/xiszishu)\
**Post date:** [July 14, 2023, 1:53pm UTC](https://discuss.ray.io/t/possible-reasons-for-ray-data-stucks-at-write-csv-or-write-parquet/11412/1 "2023-07-14T13:53:34Z")

</div>

I create this task to perform ray.data.groupby over the csv files (around 370GB).

```auto
dataset = ray.data.read_csv(input_path, parse_options=parse_options)
...
grouped_data = dataset.groupby(key=sort_key)
output = grouped_data.map_groups(lambda a: a)
...
output.write_csv(output_dst) # stuck here

```

Both map and reduce work fine, but stuck at the write\_csv or write\_parquet step.  
 ![Screenshot 2023-07-14 at 9.38.02 PM](https://us1.discourse-cdn.com/flex020/uploads/ray/optimized/2X/0/093c8043a6450e43dcf6436323e40a51a7e31392_2_690x29.png)

I tried on both single machine and two machines, also tried ray 2.3 and 2.4, same problem.  
Any clue to work around this?

---

<div class="post-metadata">

**Author:** ![Jules\_Damji](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jules_damji/32/4058_2.png) [@Jules\_Damji](https://discuss.ray.io/u/Jules_Damji)\
**Post date:** [July 17, 2023, 4:29pm UTC](https://discuss.ray.io/t/possible-reasons-for-ray-data-stucks-at-write-csv-or-write-parquet/11412/2 "2023-07-17T16:29:20Z")

</div>

@xiszishu Can you try with Ray 2.5.1?  
cc: @chengsu

---

<div class="post-metadata">

**Author:** ![xiszishu](https://avatars.discourse-cdn.com/v4/letter/x/f0a364/32.png) [@xiszishu](https://discuss.ray.io/u/xiszishu)\
**Post date:** [July 18, 2023, 12:16am UTC](https://discuss.ray.io/t/possible-reasons-for-ray-data-stucks-at-write-csv-or-write-parquet/11412/3 "2023-07-18T00:16:32Z")

</div>

thank you! yes, we tried 2.3.0, 2.4.0, 2.5.0 and 2.5.1, same issue  
it seems the tasks are pending for scheduling as shown in dashboard (we have two machines with 192 cores and 2 TB DRAM in total):

 ![20230718-081421](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/e/e87be023c98061408d679e77ae998d831d09f818.jpeg)

For ray 2.5.1, it crashes after being stuck in write\_csv for a while. I’ve collected the log files as follows.[https://drive.google.com/file/d/1jYOakLc0ZEZSc-OrURi8iNKFKX1XKz7t/view?usp=sharing](https://drive.google.com/file/d/1jYOakLc0ZEZSc-OrURi8iNKFKX1XKz7t/view?usp=sharing)

---

<div class="post-metadata">

**Author:** ![xiszishu](https://avatars.discourse-cdn.com/v4/letter/x/f0a364/32.png) [@xiszishu](https://discuss.ray.io/u/xiszishu)\
**Post date:** [July 25, 2023, 9:03am UTC](https://discuss.ray.io/t/possible-reasons-for-ray-data-stucks-at-write-csv-or-write-parquet/11412/4 "2023-07-25T09:03:18Z")

</div>

So the issue is resolved by using ray job submit, ray client seems to be not compatible well with the cluster.