# Does \`ray.data.Dataset.iter\_batches\` guarantee order of the original file?

**URL:** https://discuss.ray.io/t/does-ray-data-dataset-iter-batches-guarantee-order-of-the-original-file/10689
**Category:** Ray Data
**Created:** [May 17, 2023, 3:41am UTC](https://discuss.ray.io/t/does-ray-data-dataset-iter-batches-guarantee-order-of-the-original-file/10689 "2023-05-17T03:41:04Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![fengsp](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/fengsp/32/4364_2.png) [@fengsp](https://discuss.ray.io/u/fengsp)
#### Post date: [May 17, 2023, 3:41am UTC](https://discuss.ray.io/t/does-ray-data-dataset-iter-batches-guarantee-order-of-the-original-file/10689/1 "2023-05-17T03:41:04Z")

</div>

For example, we read a csv or parquet file into a ray dataset, would the following code get the same result?

```auto
# ray dataset
for batch in ray_dataset.iter_batches():
    for row in batch:
        print(row)
# raw file
for row in pd.read_parquet('example.parquet').iterrows():
    print(row)

```

What about a shard of the dataset, would we get the same order as the original file?

```auto
shard1, _, _ = ds.split_at_indices([2000, 5000])
# same order as the original_file[:2000] ?
for row in shard1.iter_batches():
    print(row)

```

---

<div class="post-metadata">

### Author: ![fengsp](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/fengsp/32/4364_2.png) [@fengsp](https://discuss.ray.io/u/fengsp)
#### Post date: [May 25, 2023, 2:50am UTC](https://discuss.ray.io/t/does-ray-data-dataset-iter-batches-guarantee-order-of-the-original-file/10689/2 "2023-05-25T02:50:35Z")

</div>

Can I get any help here? No explicit docs could be found for this.

---

<div class="post-metadata">

### Author: ![gjoliver](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/gjoliver/32/1490_2.png) [@gjoliver](https://discuss.ray.io/u/gjoliver)
#### Post date: [May 26, 2023, 4:40pm UTC](https://discuss.ray.io/t/does-ray-data-dataset-iter-batches-guarantee-order-of-the-original-file/10689/3 "2023-05-26T16:40:50Z")

</div>

can you try setting `ray.data.context.DatasetContext.get_current().execution_options.preserve_order = True` like this example here?  
[https://docs.ray.io/en/latest/data/glossary.html#term-Batch-format](https://docs.ray.io/en/latest/data/glossary.html#term-Batch-format)

---

<div class="post-metadata">

### Author: ![fengsp](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/fengsp/32/4364_2.png) [@fengsp](https://discuss.ray.io/u/fengsp)
#### Post date: [May 28, 2023, 10:41am UTC](https://discuss.ray.io/t/does-ray-data-dataset-iter-batches-guarantee-order-of-the-original-file/10689/4 "2023-05-28T10:41:02Z")

</div>

It seems this options is not supported in ray==2.2.0, How to get this option for ray 2.2.0? Order is not preserved for all versions of ray?

---

<div class="post-metadata">

### Author: ![gjoliver](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/gjoliver/32/1490_2.png) [@gjoliver](https://discuss.ray.io/u/gjoliver)
#### Post date: [May 28, 2023, 5:00pm UTC](https://discuss.ray.io/t/does-ray-data-dataset-iter-batches-guarantee-order-of-the-original-file/10689/5 "2023-05-28T17:00:01Z")

</div>

ray Dataset is undergoing really active development.  
I’d actually recommend you always use the latest version for performance and functionality improvements.

---

<div class="post-metadata">

### Author: ![fengsp](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/fengsp/32/4364_2.png) [@fengsp](https://discuss.ray.io/u/fengsp)
#### Post date: [May 29, 2023, 3:49am UTC](https://discuss.ray.io/t/does-ray-data-dataset-iter-batches-guarantee-order-of-the-original-file/10689/6 "2023-05-29T03:49:32Z")

</div>

I am using a library that requires ray==2.2.0, anyway, I will implement it myself without using ray dataset api since ray dataset does not work.
